The original attention mechanism for encoder-decoder RNNs (Bahdanau et al., 2015) scores each encoder state against the decoder's current state with a tiny neural network rather than a dot product:
Implement additive_attention(query, keys, W_q, W_k, v):
query: (d_q,) decoder state; keys: (n, d_k) encoder states (they also serve as the values)W_q: (d_a, d_q), W_k: (d_a, d_k), v: (d_a,)Return (weights, context) where weights has shape (n,) and sums to 1, and context has shape (d_k,). Use a numerically stable softmax (subtract the maximum score before exponentiating).