Spooky Sale: 10% off Premium yearly · $118.80 for the first year · until Nov 6Get it

Deep-ML
Learn
Problems1,200+ hands-on ML problemsLearning PathA guided route through the fundamentalsLabsInteractive labs on real techniquesProjectsBuild a GPT, an RL agent, CUDA kernelsCollectionsCurated problem sets and videosMathPen-and-paper math for MLIntuitionInvent the idea behind the formula yourselfPlaylistsYour own problem lists, shareableStudy AssistantAn AI teacher that knows your progress
Interview Prep
Compete
ContestsFresh problem sets, ranked liveLeaderboardWhere you stand, globallyBuddiesSolve with a friend: streaks, a weekly board, chat
Discuss
Deep-Path
Premium
      Intuition
      70 of 130

      Grading its own homework

      A café’s app guesses which customers will come back. Four models were trained, each on some of this month’s 20 customers: for those, it saw the answer. Write honest_score(predictions, answers, trained_on): one model’s score, higher is better, so that the best score goes to the model that will do best on customers it has never seen, like next week’s.

      Four models guessed which of this month’s 20 customers come back; each was trained on some of them (shaded) and saw their answers. Scored on every customer, Parrot comes top, then flops on next week’s new customers: it learned its customers by heart. The goal is scores that rank the models the way next week does, like the bars it ends on. Your honest_score() scores one model at a time.

      A café’s app: will each of 20 customers come back? Four models guessed.this month’s 20 customersnext weekParrot7/12Owl9/12Fox9/12Hare8/12guessed rightwrongtrained on it: it saw the answer
      Your functionhonest_score(predictions, answers, trained_on)
      predictionswhat the model guessed for each of this month's customers, 1 = comes back, 0 = doesn't (a numpy array)e.g. [1, 0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 1, 1, 1,…
      answerswhat each customer really did, 1 = came back, 0 = didn't (a numpy array)e.g. [1, 0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 1, 1, 0,…
      trained_on1 where the model was trained on that customer (it saw the answer), else 0 (a numpy array)e.g. [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0,…
      returnsthe model's score: higher is better
      solution.pyLoading Python
      Sign in for an AI hint
      ← At least one rightMiles vs millimetres →