Sign in to Rorobot
Welcome back! Please sign in to continue
or
Don't have an account? Sign up
By continuing, you agree to our Terms of Service and Privacy Policy.
Chat
What makes self-attention different from traditional attention?
RorobotSelf-attention is an attention mechanism relating different positions of a single sequence in order to compute a representation of the sequence.p.2
Flashcards
Flashcard
Q: How is attention output computed?
A: As a weighted sum of the values, where each weight is computed by a compatibility function of the query with the corresponding key.
Source: p.3Due: Tomorrow
Again
Hard
Good
Easy
arXiv Search
transformer attention mechanism
Attention Is All You Need
Vaswani et al. · 2017
BERT: Pre-training of Deep Bidirectional Transformers
Devlin et al. · 2018
An Image is Worth 16x16 Words
Dosovitskiy et al. · 2020