Blog · ← All posts
We Audited 30 AI-Generated Codebases: The 5 Bugs That Kept Appearing
Last updated: October 2026 · 7 min read
Earlier this year we started offering code audits for teams shipping AI-written code. Not security reviews, exactly, quality reviews. The kind where someone with no stake in the code reads every line and marks what would bite the team in six months.
Thirty codebases later, a pattern had settled in. Different stacks, different companies, different prompts, yet the same bugs kept appearing. Not the same lines of code. The same kinds of mistakes, like the model had five favorite ways to be wrong and rotated through them. Here they are, in order of how often they showed up.
1. Error handling that silently swallows failures (29 of 30)
The most common defect by a wide margin. AI code loves a try/except block with a `pass` in it. Or a catch that logs a generic message and carries on as if nothing happened.
One codebase, a Python data pipeline at a logistics startup, had 47 try/except blocks. Thirty-one of them caught the exception, logged the word "error", and continued. The pipeline had been producing wrong numbers for three weeks before anyone noticed, because nothing ever crashed. The bug was obvious once found. It stayed hidden for three weeks because the error handling did exactly what it was told: nothing.
Why it happens: models are trained to produce code that runs. Unhandled exceptions are a visible failure; swallowed ones aren't. The model optimizes for "no tracebacks," and "no tracebacks" is not the same as "correct."
What to look for: every catch block that doesn't either retry, raise, or return a failure signal upstream. If the only thing in the except clause is a log line, treat it as a bug until proven otherwise.
2. Dead code the model wrote and forgot to delete (27 of 30)
AI-generated code accumulates abandoned drafts. Functions that are never called. Imports that were needed two revisions ago. Entire files that the model replaced but left in place. In one 40-person SaaS team's repo, we found a 600-line module that had zero callers: the model had rewritten the feature in a different file, kept the old one "just in case," and nobody removed it.
This sounds cosmetic. It isn't. Dead code gets read by the next developer, who assumes it matters and builds on top of it. Dead code with a plausible name is a trap: someone will import from it in six months, and the bug report from that will take a full day to untangle.
We now run a simple check on every audit: list all functions and files with no callers. It takes five minutes and it finds something every single time.
3. Magic numbers and hardcoded assumptions (26 of 30)
`if len(users) > 50: paginate()`. A retry loop that tries exactly 3 times, with 3 appearing twice in the function and never explained. A timezone offset of `-5` baked into a date calculation. The team was in Belize, the code would break for any user elsewhere, and the comment said "adjust as needed."
The worst one we found: an e-commerce checkout that applied tax based on a hardcoded list of six state abbreviations. The company sold to fifty states. Forty-four states got charged zero tax. That defect had a dollar value attached to it, and it was introduced by an AI that had no idea which states the company sold to, because nobody told it.
The honest version: hardcoded values aren't always wrong. A constant you name and document is fine. A literal buried in a conditional is a time bomb. AI models produce literals because they're writing the simplest thing that satisfies the prompt, and the simplest thing is always a number where a decision should be.
4. Duplicated logic instead of shared functions (23 of 30)
The model writes a date formatter in one file. Three prompts later it needs the same formatting in another file, doesn't remember the first one, and writes it again, slightly differently. Now there are two formatters that disagree on edge cases, and a bug fix applied to one doesn't reach the other.
In one React codebase we counted the same validation logic in seven components, each copy a few lines different from the others. The team had shipped two contradictory validation messages for the same form field. Users saw "Enter a valid email" and "Invalid email address" on the same screen, from the same codebase, for the same input.
This is the defect that scales worst. One copy is a quirk. Seven copies is a maintenance tax you pay forever. When we built our scanner, we learned that duplication detection catches this mechanically. But the fix is human: someone has to decide which copy is canonical and delete the rest.
5. Functions that do three jobs at once (21 of 30)
A `process_order()` that validates the cart, charges the card, sends the confirmation email, and updates inventory. Eighty lines, four responsibilities, and no test can target one without tripping the other three.
Models do this because prompts do this. "Build the checkout flow" is one prompt, so it becomes one function. The model has no concept of the team that will own this code next quarter. It optimizes for prompt completion, not maintainability.
The fix is cheap at generation time and expensive later. We tell teams: when you ask for a feature, ask for the boundaries too. "Split this into functions that can be tested independently" is one sentence that prevents this entire category.
The cost of "it works"
Here's what we keep telling teams, and what almost nobody believes until the second incident: each of these five defects is cheap to write and expensive to own. The swallowed exception doesn't cost you the afternoon it was written. It costs you the three weeks of wrong data, plus the day of forensic work to find it, plus the argument about whether the numbers can be trusted at all. The 600-line dead module doesn't cost you anything today. It costs you the day, six months from now, when a new hire builds a feature on top of it and you discover it was never wired to anything.
We did rough math on one audit, a Node.js SaaS with about 60,000 lines of generated code. The duplication alone, five copies of the validation logic across the codebase, meant every validation change shipped as five separate edits, each needing its own test run and review. The team estimated it at roughly a day of extra work per release cycle. Over a year, that's a month of engineering time spent servicing a defect that took the model forty seconds to create.
That asymmetry is the whole story. Generation is instant. Ownership is forever. AI code isn't bad because the model is bad; it's expensive because nobody priced the maintenance at generation time.
Why the same five keep appearing
None of these are mysteries. They come from how code generation actually works: the model predicts likely next tokens, trained on millions of repos (most of which were written under deadline, without review). The average codebase has these defects. The model produces average code, fast.
There's a second cause, and it's us. Prompts describe features, not quality bars. Nobody writes "and handle errors properly" because it feels obvious, but the model's defaults for "properly" come from training data, not from your team. Every defect above is what you get when the prompt says what and stays silent on how well. We've written a separate piece on the prompt patterns that produce the worst code, because half of this list is preventable before the first token is generated.
That said, there are two honest caveats. First, human-written code has all five of these defects too, though AI produces them at higher volume and with more confidence. Second, this sample skews toward teams that asked for an audit, which means they suspected something was off. Healthy codebases don't volunteer for review.
What to do with this list
If you're about to ship AI-written code, check these five in this order. They take about an hour on a small codebase and they catch the defects that compound:
- Catch blocks: every one must retry, raise, or return a failure signal, not just log.
- Dead code: list everything with no callers and delete or document it.
- Literals: every hardcoded number or assumption gets a name and a comment, or gets parameterized.
- Duplication: search for repeated logic and consolidate before it multiplies.
- Function size: anything doing more than two jobs gets split before it ships.
This won't catch everything. It's a first pass, not a review. But across thirty codebases, it would have caught the bulk of what we flagged.