Spooky Sale: 10% off Premium yearly · $118.80 for the first year · until Nov 6Get it

Deep-ML
Learn
Problems1,200+ hands-on ML problemsLearning PathA guided route through the fundamentalsLabsInteractive labs on real techniquesProjectsBuild a GPT, an RL agent, CUDA kernelsCollectionsCurated problem sets and videosMathPen-and-paper math for MLIntuitionInvent the idea behind the formula yourselfPlaylistsYour own problem lists, shareableStudy AssistantAn AI teacher that knows your progress
Interview Prep
Compete
ContestsFresh problem sets, ranked liveLeaderboardWhere you stand, globallyBuddiesSolve with a friend: streaks, a weekly board, chat
Discuss
Deep-Path
Premium
      Intuition
      71 of 88

      Don’t peek

      Like in “Who should “it” listen to?”, a word scores how well every word’s key matches its query and turns the scores into shares of its attention. But now the text is being written one word at a time, left to right: when the word at position reads, the words after it don’t exist yet. Write listen(query, keys, position): one share per word in the whole sentence, future words included.

      query: “sat” looks for a place
      animalplace
      01
      keys
      animalplace
      cat20
      sat00
      mat02
      position: “sat” is word 1 (cat is 0)
      1
      solution.pyLoading Python
      ← Listen and collectWord order matters →