Implement attention(q, k, v, key_padding_mask=None, causal=False) on top of torch.nn.functional.scaled_dot_product_attention.
q, k, v are shaped (batch, heads, seq, head_dim).key_padding_mask is a bool tensor of shape (batch, seq) where True marks a real token and False marks padding that must be ignored.causal=True additionally forbids each query from attending to any later position.Both constraints can apply at once, so you must combine them into a single attn_mask and pass that.
attn_mask, SDPA treats True as "may attend" and False as "masked out" - the opposite of masked_fill's convention.attn_mask must broadcast against (batch, heads, seq_q, seq_k). Reshape the padding mask to (batch, 1, 1, seq).torch.ones(T, T, dtype=torch.bool).tril().is_causal=True together with an attn_mask; build one combined mask instead.attn_mask=None.