Implement a function that applies learned positional embeddings to a batch of token embeddings.
In many transformer architectures (such as BERT and GPT), positional information is injected into the model using a trainable embedding table. Unlike fixed sinusoidal encodings, learned positional embeddings are stored as a matrix where each row corresponds to a specific sequence position and is optimized during training.
Your Task:
Write a function learned_positional_encoding(token_embeddings, position_embedding_table, start_pos) that takes:
token_embeddings: A 3D numpy array of shape (batch_size, seq_len, d_model) representing the embedded token vectors for a batch of sequences.position_embedding_table: A 2D numpy array of shape (max_seq_len, d_model) representing the learned embedding lookup table indexed by position.start_pos: An integer indicating the starting position index in the sequence (default is 0). This is useful in autoregressive generation where decoding may continue from a non-zero position.The function should return a numpy array of the same shape as token_embeddings with the appropriate positional information incorporated.
The same positional embeddings should be applied identically across all samples in the batch.