Harvard researchers just analyzed 300 million work events across 700 companies to answer one question: do AI coding agents actually help ship software? Code output jumped 30% after firms adopted them. The amount of software getting done? It barely moved at all.
The problem nobody mentions in the demos
You’ve probably seen the pitch by now. An agent writes the feature while you grab coffee, and development moves at the speed of your ideas. Tools like Copilot, Claude Code, and Codex are everywhere, and GitHub says one in three pull requests now involves an AI agent.
Here’s what the ads leave out. Writing code was never the slow part of software. Deciding whether the code is right, safe, and worth shipping always was. When generation gets 30% faster and review doesn’t, the bottleneck just moves. The new Harvard study puts actual numbers on that trade, and they’re worth a close look before you spend money on agent seats expecting faster shipping.
What the study actually found
Researchers Fiona Chen and James Stratton at Harvard used analytics from Jellyfish, a platform that tracks engineering team output. The dataset covers 300 million work events, over 700,000 employees, and more than 700 software firms from 2021 through March 2026. By the end, 95% of those firms had adopted AI coding agents in some form, from simple autocomplete helpers to systems that write and submit entire pull requests on their own.
The before-and-after numbers look great until you reach the last row:
| What changed after AI agents arrived | Result |
|---|---|
| Lines of code generated | +30% |
| Total commits | +20% |
| Pull requests opened | +23% |
| Review time per pull request | +49% |
| Pull requests needing revisions | Nearly doubled |
| Comments per pull request | +35% |
| Workers spending time on reviews | +14% |
| Issues and features actually resolved | No significant change |
That last row is the whole story. Features shipped per team stayed flat. Meanwhile the human cost of reviewing everything those agents produced quietly grew by double digits.
One distinction the researchers make matters for whatever tool you’re shopping for. A coding assistant mostly autocompletes lines a human is already writing. A coding agent takes a task and produces whole pull requests on its own. The 30% jump in code came from the agent side, and so did the review strain, because that’s where output volume per person explodes. Autocomplete gives you a modest speedup with a modest bill. Hand an agent the whole ticket and the bill arrives with it.
Why the review bottleneck eats the gains
AI coding agents don’t get tired, but the humans checking their work do. After adoption, pull requests took almost 50% longer to review, reviewers left 35% more comments, and more requests came back needing changes. A bigger share of the team ended up doing review work instead of building.
The researchers tried to find companies where this trade actually paid off in output. They couldn’t. Efficiency gained during coding got, in their words, absorbed by downstream constraints. On top of that, the AI doing the reviewing barely helped: by March 2026 about 80% of firms used some form of AI code review, yet agents produced only 23% of review comments and 11% of pull requests. GitHub’s own platform-wide number from this month is higher, one in three pull requests involving an agent, but the direction is the same. Humans still own the review queue, and that queue is what’s growing.
Why does AI review undershoot like this? Watch how it behaves in a real repo and the answer gets obvious. It’s decent at the mechanical stuff: missing null checks, unused variables, style drift. It’s weak at the judgment calls that decide whether code should exist at all, and humans stay the ones who click merge. Review is authority-heavy work, and nobody has delegated authority to a model yet.
Does this mean AI coding is hype? Not even close. It means the gains are real but they’re landing in the wrong place, like a promotion that comes with so much extra paperwork you end the month more behind than before.
How beginners get the speed without the pileup
The study measured big engineering orgs with layers of process. Solo builders and small teams can capture the upside and dodge the review pileup. Four habits make the difference.
1. Ship smaller reviews
Agents are fastest when the task is small and well-scoped. A 200-line change gets a real review in minutes. A 3,000-line mega request sits for days and gets a rubber stamp instead of a look. Split your work so every agent run produces one reviewable piece. You’ll finish faster precisely because each step is slower to check.
2. Define done before you prompt
Decide what “working” means before the agent starts: the button renders, the form saves, the test passes. Then run the thing yourself before it counts as shipped. The study’s flat shipping numbers partly reflect teams accepting generated code as progress. Run code is the only progress that counts. The vibe coding rules from the Linux kernel say the same thing in harsher language.
3. Review like a senior, not a typist
Time your agent saves should go into judgment work: architecture, naming, deciding what deserves to exist. That’s where the market is heading anyway. Indeed’s hiring data shows senior roles made up 71% of software posting growth over the past year, and AI-titled jobs accounted for 37% of it. Companies still pay for people who can tell good code from confident nonsense. Getting sharp at reviewing what AI coding agents produce is quickly becoming the actual job.
4. Track features shipped, not lines written
Agents make code volume meaningless. Measure what matters instead: did the feature ship, how long did it take, what broke after. If your agent usage isn’t moving those numbers, the 30% code boost is just noise with extra commits. Keep a simple log for a month and the picture gets obvious fast.
Shopping for your first setup
If you’re picking tools this month, let the bottleneck pick for you. Fast typists who already know what good code looks like get the most from agents, because their review pass is quick and decisive. If you’re still learning what good code looks like, start with an assistant and treat the agent tier as a graduation. Whatever you buy, budget for the human half too. The study’s 49% review penalty is the invoice most teams never saw coming.
So is AI taking coding jobs?
For working developers, the data says the profession is shifting rather than shrinking. US software development postings are up nearly 15% since early 2025, even while overall postings fell, and the Bureau of Labor Statistics projects 10% growth for the profession through 2035. Demand concentrates on people who can direct and verify AI coding agents. Entry-level pressure is real though: Stanford researchers found employment for early-career developers aged 22 to 25 down about 20%, so newcomers should build AI fluency and review skills together. Our breakdown of why AI skills command a pay premium covers what employers actually reward right now.
The takeaway
AI coding agents didn’t break software development. They exposed which part was ever truly hard, and it was never the typing. Use them aggressively, review seriously, and measure shipped features instead of generated lines. Pick one task this week, split it into reviewable pieces, and let the agent sprint while you stay the editor.
