You’re learning what each square of a board game is worth: all the rewards still to come from it, counted as in “A reward now or later”. You just stepped off a square: you got reward, and you guess the square you landed on is worth guess_next. Write update(guess_here, reward, guess_next, patience, rate): your new guess for the square you just left, without waiting for the game to end. rate (0 to 1) is how far one step may move a guess.
You thought the square you’re on was worth 4. You step off it, get a reward of 1, and land on a square you think is worth 10. What should you now guess the square you left is worth?
update(guess_here, reward, guess_next, patience, rate)| guess_here | your guess of what the square you just left is worth | e.g. 10 |
| reward | what you got for this step | e.g. 1 |
| guess_next | your guess of what the square you landed on is worth (0 if the game ended) | e.g. 10 |
| patience | what a reward one step later is worth, as a share of one now | e.g. 0.9 |
| rate | how far one step may move a guess (0 to 1) | e.g. 0.1 |
| returns | your new guess for the square you just left | |