A study by the National Bureau of Economic Research (NBER) of over 100,000 GitHub developers has confirmed what many suspected: AI tools generate much more code, but not necessarily more finished software.
The paper, titled “Writing Code vs. Shipping Code: Productivity Effects Across Generations of AI Coding Tools” and published in May 2026 by Demirer, Musolff, and Yang, analyzes the impact of three generations of AI tools:
- Autocomplete (GitHub Copilot): +40% in commits
- Synchronous agents (Cline, Codex): +140% in changes
- Asynchronous agents (Claude Code, autonomous loops): +180% in changes
But the final deliverable — releases, features that reach the user — only increased by ~30%.
The Bottleneck Has Moved
The study’s authors identify that the constraint is no longer in writing code. It is downstream: quality control, approvals, deployment, integration, and support. These stages do not scale with code generation because they involve human review, team coordination, and organizational processes.
The estimated substitution elasticity is only 0.25 — what economists call “strong complementarity.” In plain terms: no matter how much you accelerate code writing, the later stages do not accelerate proportionally.
App marketplace data confirms this: the supply of new apps surged, but usage (downloads, time spent) did not increase at the same rate.
What This Means
The paradox has direct implications for engineering teams:
-
Measuring lines of code or commits no longer works. AI inflates both. The real metric is features shipped and problems solved.
-
The bottleneck is organizational, not technical. Investing in better AI models without investing in QA, CI/CD, and review processes yields diminishing returns.
-
A new role emerges: the “delivery engineer.” Someone who optimizes the entire pipeline, not just code generation. Autonomous agent loops (see our article on AI coding loops) promise to help, but also add complexity.
-
Watch out for technical debt. More AI-generated code means more potential legacy code. Without careful review, generation speed can become technical debt accumulation speed.
For Teams Using AI
The study does not suggest stopping the use of AI tools. It suggests using them intelligently:
- Automate the downstream stages too: automated testing, continuous deployment, AI-assisted code review
- Measure features shipped, not lines written
- Maintain human review at critical points (architecture, security, user experience)
- Prepare for a world where code is cheap but validation remains expensive
The AI productivity paradox is not a criticism of the tools. It is a wake-up call for the industry to evolve its processes at the same pace as its models.