Deep-ML
Learn
Problems1,200+ hands-on ML problemsLearning PathA guided route through the fundamentalsLabsInteractive labs on real techniquesProjectsBuild a GPT, an RL agent, CUDA kernelsCollectionsCurated problem sets and videosMathPen-and-paper math for MLPlaylistsYour own problem lists, shareableStudy AssistantNewAn AI teacher that knows your progress
Interview Prep
Compete
ContestsFresh problem sets, ranked liveLeaderboardWhere you stand, globallyBuddiesNewSolve with a friend: streaks, a weekly board, chat
DiscussNew
ResearchNew
Premium

Machine Learning Problem Collections

Curated collections of machine learning problems organized by topic. Track your progress through each collection, complete all problems to earn badges, and follow structured learning sequences covering neural networks, linear algebra, optimization, and more.

      Collections
      DeepSeek-V4 badge

      DeepSeek-V4

      The DeepSeek-V4 technical report (April 2026), section by section. The architecture: manifold-constrained hyper-connections projected onto the Birkhoff polytope with Sinkhorn-Knopp, the CSA/HCA hybrid attention (token-level KV compression, the lightning indexer and sparse selection, heavily compressed attention, partial RoPE, mixed-precision KV entries) and the Muon optimizer with hybrid Newton-Schulz. The infrastructure: expert-parallel communication overlap, batch-invariant and deterministic kernels, ZeRO for Muon, two-stage context parallelism, tensor-level checkpointing, and the heterogeneous KV cache with on-disk storage. Pre-training: data construction and document packing, the batch-size, context-length and dense-to-sparse schedules, DeepSeekMoE routing, anticipatory routing and SwiGLU clipping, FP4 quantization-aware training, multi-token prediction and the evaluation suite. Post-training: GRPO specialists with reasoning-effort modes and the DSML tool format, multi-teacher on-policy distillation with lazy teacher logits, and the preemptible rollout service for million-token RL. Math cards derive the report's numbers, problems implement each mechanism, and two premium projects build the hybrid attention and the training stack from scratch.

      0/115
      0%

      Learning Resource

      Study material to help you master this collection

      Open

      Premium Collection

      Browse the structure here. Upgrade to unlock the problems.

      Upgrade

      1. Introduction and Model Overview (sections 1, 4.2.1)

      4 problems

      2. Manifold-Constrained Hyper-Connections (2.2)

      5 problems

      3. Compressed Sparse Attention (2.3.1)

      7 problems

      4. Heavily Compressed Attention (2.3.2)

      2 problems

      5. Attention Details: Normalization, Partial RoPE and Sliding Window (2.3.3)

      4 problems

      6. Attention Efficiency: Mixed-Precision KV and Cost (2.3.4)

      6 problems

      7. Muon Optimizer (2.4)

      9 problems

      8. Communication-Computation Overlap in Expert Parallelism (3.1)

      3 problems

      9. Batch-Invariant and Deterministic Kernels (3.3)

      3 problems

      10. Training Framework: ZeRO for Muon, Context Parallelism, Checkpointing (3.4)

      6 problems

      11. Inference Framework: KV Cache Structure and On-Disk Storage (3.5)

      8 problems

      12. Pre-Training Data (4.1)

      5 problems

      13. Pre-Training Setups: Schedules, DeepSeekMoE and Routing (4.2.1-4.2.2)

      10 problems

      14. Mitigating Training Instability (4.2.3)

      4 problems

      15. FP4 Quantization-Aware Training (4.2)

      10 problems

      16. Multi-Token Prediction (4.2)

      3 problems

      17. Evaluations (4.3, 5.3)

      4 problems

      18. Specialist Training: GRPO, Rewards and Reasoning Modes (5.1.1)

      12 problems

      19. On-Policy Distillation (5.1.2, 5.2.2)

      5 problems

      20. Post-Training Infrastructure: Rollout Service and Million-Token RL (5.2.3-5.2.4)

      5 problems