Tree-Backup(), TB(), is the eligibility-trace version of the -step tree-backup algorithm. It learns action values for a target policy from data generated by any behaviour policy, and it never uses importance-sampling ratios. Instead, each trace is decayed by the target policy's probability of the action that was actually taken.
Implement tabular TB() over one episode: tree_backup_lambda(Q, episode, pi, alpha, gamma, lam).
Q: action values, shape (n_states, n_actions). Work on a float copy.episode: list of transitions (s, a, r, s_next, done) in time order.pi: target policy, shape (n_states, n_actions), rows summing to 1.Start the trace at zero (same shape as Q). For each transition, in order:
done.Q before this step's update.Return the updated Q.