Spooky Sale: 10% off Premium yearly · $118.80 for the first year · until Nov 6Get it

Deep-ML
Learn
Problems1,200+ hands-on ML problemsLearning PathA guided route through the fundamentalsLabsInteractive labs on real techniquesProjectsBuild a GPT, an RL agent, CUDA kernelsCollectionsCurated problem sets and videosMathPen-and-paper math for MLIntuitionInvent the idea behind the formula yourselfPlaylistsYour own problem lists, shareableStudy AssistantAn AI teacher that knows your progress
Interview Prep
Compete
ContestsFresh problem sets, ranked liveLeaderboardWhere you stand, globallyBuddiesSolve with a friend: streaks, a weekly board, chat
Discuss
Deep-Path
Premium
      Intuition
      32 of 88

      Cut the long tail

      A language model gives every possible next word a chance, and then picks one at random (“Roll a loaded die”). Most words get a tiny chance, and together that long tail of junk still gets picked now and then. Write keep(chances, share): the indexes of the words to keep before picking. No junk, but enough variety: keep the most likely words until they cover share of the chance.

      chances after “I’d like a cup of”
      watercoffeesoupteamilk
      0.10.40.050.30.15
      share
      0.8
      solution.pyLoading Python
      ← Roll a loaded dieIs the bet worth it? →