The ranking stage scores items independently, so the top of the list is often ten versions of the same thing. A re-ranking stage trades a little relevance for diversity. Implement the two standard greedy methods with NumPy.
mmr(relevance, sim, lam, k)
relevance is a (n,) array, sim an (n, n) similarity matrix with ones on the diagonal, lam in [0, 1]. Pick items one at a time; the score of an unpicked item i is lam * relevance[i] - (1 - lam) * max_{j picked} sim[i, j] (the second term is 0 for the first pick). Return the k picked indices in pick order, ties broken by the lower index.dpp_kernel(relevance, embeddings)
embeddings is (n, d). Normalise each row to unit length, form the cosine similarity matrix S, and return L = diag(relevance) S diag(relevance) rounded to 6 decimals.greedy_dpp(L, k)
L: start with an empty set and at each step add the unpicked item that gives the largest determinant of the sub-kernel of the picked set, ties by lower index. Stop early (return fewer than k) if no candidate increases the determinant above 1e-12. Return the pick order as a list.diversity_tradeoff(relevance, sim, order)
order, return (mean_relevance, mean_pairwise_similarity) of the picked items, both rounded to 4 decimals; the second is 0.0 for fewer than two items.greedy_dpp, np.linalg.det(L[np.ix_(S, S)]) for each candidate set S is fine at interview sizes; the incremental Cholesky version is the production algorithm.lam = 1 MMR reduces to sorting by relevance.