Vaswani et al.'s 'Attention Is All You Need' (2017), section by section. The attention it replaced, scaled dot-product and multi-head attention, the three ways the model uses attention (masked self-attention and encoder-decoder cross-attention), the encoder and decoder stacks with position-wise feed-forward blocks, residuals and layer normalization, shared embedding and softmax weights with sinusoidal positional encodings, the paper's own 'why self-attention' complexity argument, its training recipe (byte-pair data, Adam with the warmup-then-inverse-square-root schedule, dropout and label smoothing), and its results (beam search with length penalty, BLEU, checkpoint averaging). Math cards derive the numbers the paper states, problems implement each component, and the final section builds the full Transformer from scratch.
Learning Resource
Study material to help you master this collection
Model blueprint
Attention Is All You Need · Vaswani et al., 2017