Spooky Sale: 10% off Premium yearly · $118.80 for the first year · until Nov 6Get it

Deep-ML
Learn
Problems1,200+ hands-on ML problemsLearning PathA guided route through the fundamentalsLabsInteractive labs on real techniquesProjectsBuild a GPT, an RL agent, CUDA kernelsCollectionsCurated problem sets and videosMathPen-and-paper math for MLIntuitionInvent the idea behind the formula yourselfPlaylistsYour own problem lists, shareableStudy AssistantAn AI teacher that knows your progress
Interview Prep
Compete
ContestsFresh problem sets, ranked liveLeaderboardWhere you stand, globallyBuddiesSolve with a friend: streaks, a weekly board, chat
Discuss
Deep-Path
Premium
      Intuition
      89 of 102

      The greedy trap

      A language model writes two-word sentences. first_chances[i] is the chance it starts with first word i, and second_chances[i][j] is the chance that next word j follows it. Write best_sentence(first_chances, second_chances): return [i, j], the most likely sentence of all.

      A model writes two-word sentences: a first word by its chances, then a next word by the chances after it. Greedy takes the likeliest word, then the likeliest next. Watch the model write, keep a tally, and see whether greedy’s pick is the sentence it writes most.

      A model writes two-word sentences, drawing each word by its chances.first wordnext wordthe60%cat30%dog25%sun25%end20%a40%cat5%dog90%sun3%end2%greedy: “the cat”what the model wrote · 0 sentencesthe most likely sentence:?
      Your functionbest_sentence(first_chances, second_chances)
      first_chancesfirst_chances[i]: chance the sentence starts with word ie.g. [0.3, 0.7]
      second_chancessecond_chances[i][j]: chance word j comes next, after word ie.g. [[0.5, 0.5], [0.2, 0.8]]
      returns[i, j], the most likely sentence
      solution.pyLoading Python
      Sign in for an AI hint
      ← Softmax on the goFind your twin →