Spooky Sale: 10% off Premium yearly · $118.80 for the first year · until Nov 6Get it

Deep-ML
Learn
Problems1,200+ hands-on ML problemsLearning PathA guided route through the fundamentalsLabsInteractive labs on real techniquesProjectsBuild a GPT, an RL agent, CUDA kernelsCollectionsCurated problem sets and videosMathPen-and-paper math for MLIntuitionInvent the idea behind the formula yourselfPlaylistsYour own problem lists, shareableStudy AssistantAn AI teacher that knows your progress
Interview Prep
Compete
ContestsFresh problem sets, ranked liveLeaderboardWhere you stand, globallyBuddiesSolve with a friend: streaks, a weekly board, chat
Discuss
Deep-Path
Premium
      Intuition
      45 of 102

      How confused is the model?

      A language model read a text one word at a time and gave each actual next word a chance. Write perplexity(chances): one number for how confused the model was, said as “it was like choosing among k words”.

      A model read “the cat sat” one word at a time, and gave each word the chance it saw it coming. A chance says how unsure the model was: 50% on the right word is as unsure as a fair pick among 2 words. Your function says it for the whole text.

      “the”: the model gave it 50%. That’s as unsure as a fair pick among 2 words.“the cat sat”wordthe model’s chance on itas unsure as a fair pick among…the50%2 wordscat10%10 wordssat25%4 wordsthe whole textlike choosing among?words, every time
      Your functionperplexity(chances)
      chancesthe chance the model gave each actual next worde.g. [1, 1, 1]
      returnshow confused it was, as "like choosing among k words"
      solution.pyLoading Python
      Sign in for an AI hint
      ← Too small to storeNever say never →