Implement the greedy policy improvement step used in dynamic programming for reinforcement learning.
Given:
V as a dictionary mapping states to valuestransitions as a dictionary mapping (state, action) to a list of tuples (probability, next_state, reward) representing stochastic dynamicsgammaCompute:
Q(s, a) for every state-action pair present in the transitionsThe function should return a tuple (policy, Q) where:
policy is a dictionary mapping each non-terminal state (that has actions) to its greedy actionQ is a dictionary mapping (state, action) pairs to their computed action-valuesTie-breaking rule: When multiple actions share the same maximum Q-value, select the action that comes first in alphabetical (lexicographic) order.