Theocharous, Thomas and Ghavamzadeh compared a greedy ad policy, which maximises the chance of a click on the current visit, with a life-time value (LTV) policy, which maximises total clicks over a user's repeated visits. Implement both metrics and both policies.
1. ctr_and_ltv(logs). logs is a list with one entry per user, each a list of 0/1 click outcomes, one per visit. Return (ctr, ltv) with CTR = total clicks / total visits and LTV = total clicks / number of users.
2. greedy_vs_ltv(click, trans, ret). A user is in one of engagement stages and starts in stage 0. On each visit the site shows one of offers:
click[k, a]: probability the user clicks offer in stage (reward 1 for a click).trans[a, k, k2]: probability the user moves from stage to stage after being shown offer (rows sum to 1).ret[k2]: probability the user comes back for another visit after landing in stage ; otherwise they never return.A policy picks one offer per stage. The greedy policy picks, in each stage, the offer with the highest click probability. The LTV policy maximises the expected total clicks from every stage (all of these quantities are finite because return probabilities are below 1). Break ties in either policy by the lowest offer index; treat values within as ties.
For each policy compute, for a user starting in stage 0, the expected total clicks (its LTV) and the expected number of visits, and its CTR as the ratio of the two. Return (greedy_policy, ltv_policy, (greedy_ctr, greedy_ltv), (ltv_ctr, ltv_ltv)) with the policies as lists of ints.