Implement Sarsa() with accumulating eligibility traces for on-policy control in a deterministic environment.
Implement sarsa_lambda(n_states, n_actions, transitions, terminal_states, gamma, alpha, lam, epsilon, num_episodes, seed=42):
transitions maps (state, action) to (next_state, reward); every non-terminal state has an entry for every action.Randomness (NumPy global RNG, in exactly this order):
np.random.seed(seed) once at the start.S = np.random.choice(non_terminal), where non_terminal is the ascending list of non-terminal states; then choose A ε-greedily.u = np.random.rand(); if u < epsilon return np.random.randint(n_actions), else the greedy action (smallest index on ties). The rand() draw happens even when epsilon = 0.Each step, with :
Return as a 2-D list of shape (n_states, n_actions), rounded to 4 decimals.