Attention compares words with words, so it sees a sentence as a bag: dog bites man and man bites dog look the same to it. Write position_tag(position, length): a list of numbers to add to the word at position (0 is the first word) in a sentence of length words, so the model can tell where each word is. A word is a list of 16 numbers between −1 and 1, so a tag can have at most 16.