In January 2025, DeepSeek published a paper that shook the foundations of the AI industry. And for good reason: they had demonstrated that a language model could learn to reason — to reflect on its own steps, verify its answers, and correct its course — without a single human showing it how. The recipe: pure reinforcement, nothing else. And the results spoke so loudly that the academic community eventually published the work in Nature.
DeepSeek actually released two models, and it’s worth understanding the difference. The first, DeepSeek-R1-Zero, was trained exclusively with reinforcement learning — using an algorithm called GRPO (Group Relative Policy Optimization) — without any supervised fine-tuning. It started at 15.6% accuracy on the demanding AIME 2024 benchmark. After thousands of RL iterations, it reached 71%. With majority voting, it climbed to 86.7%. All of that without ever seeing a single human reasoning example.
The most fascinating part wasn’t the numbers, but what happened during training. The model began exhibiting behaviors no one taught it: it paused to reevaluate its approach, double-checked its calculations, corrected mistakes. The paper calls this “self-reflection,” “self-verification,” and “dynamic strategy adaptation.” In simpler terms: the machine learned to think before answering, and it did so because the reward demanded it, not because a human had shown it how.
Now, the full story has nuance. DeepSeek-R1-Zero had serious problems: poor readability, infinite repetitions, language mixing. That’s why the second model — DeepSeek-R1, the one that actually became famous — used a multi-stage pipeline that included a few thousand “cold-start” examples with human supervision, followed by more RL. DeepSeek-R1 is not “pure RL.” But R1-Zero is, and that’s where the real breakthrough lies.
DeepSeek-R1’s benchmarks are impressive. On AIME 2024 it narrowly beat OpenAI o1-1217 (79.8% vs 79.2%). On MATH-500 it reached 97.3% compared to o1’s 96.4%. On LiveCodeBench, 65.9% against 63.4%. On SWE-bench Verified, 49.2% against 48.9%. On Codeforces, it ranked in the 96.3rd percentile — Elo rating 2029 — competing at the level of elite programmers. All of this with an open-source model under the MIT license.
DeepSeek also distilled the model into smaller versions based on Qwen2.5 and Llama3, ranging from 1.5B to 70B parameters. The distilled 32B version outperforms o1-mini. The 14B version crushes QwQ-32B-Preview. This has enormous implications: anyone with a decent GPU can run a model with reasoning capabilities close to frontier models.
Why does all this matter? Because for years the industry assumed that advanced reasoning required human data — demonstrations of thought, reasoning chains written by people, fine-grained supervision. DeepSeek proved that pure reinforcement can generate reasoning skills more sophisticated than those humans can explicitly demonstrate. It’s a paradigm shift: if RL can produce better reasoners than supervised fine-tuning, the scalability of AI training changes radically.
That said, not everything is perfect. OpenAI o1-1217 still leads on GPQA Diamond (75.7% vs 71.5%) and SimpleQA. DeepSeek-R1-Zero, the pure RL model, wasn’t production-ready due to quality issues. And DeepSeek didn’t disclose exactly how much cold-start data they used (they only say “thousands of samples”), making it difficult to fully replicate the process.
But the central message holds: reasoning emerged from reinforcement, not imitation. And that changes how we think about the future of AI training.
Main source: DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning