AI Writes the Code Now. Here’s Who’s Still Accountable for It.
Somewhere in a meeting recently, someone probably told you the good news: AI writes the code now, so your next system will be faster and cheaper to build. They’re half right, and the missing half is the expensive one. After two decades of building software, here’s the reframe we’d offer before anyone signs off a budget on that basis — the question isn’t whether your build uses AI. Everyone’s does now; refusing it would be like refusing a compiler. The question is who is accountable for what it produces.
That matters because of a gap the 2026 data has made impossible to ignore. Analysis this year found that AI models produce syntactically correct code close to 100% of the time — it compiles, it runs, it demos beautifully — while the average security pass rate sits at just 56% (Pagerly, August 2026). The models got fluent. They didn’t get safe. And fluent-but-unsafe is the most dangerous combination there is, because it looks exactly like finished work.
“Works” and “done” are not the same word
This is the trap we get called in to unwind most often. Code that compiles and passes a demo feels finished, so it ships, and the problem doesn’t surface until a customer, a security researcher or your own logs go looking — by which point it’s in production, holding your data. The 44% that isn’t secure is invisible next to the 56% that is; they look identical on the screen. “It runs” is where the tool’s job ends and ours begins.
The two price tags every system carries
We’ve always told clients that software has two costs: the cost to write it and the cost to own it. The writing is the visible bit — the weeks to get to something that works. The owning is everything after: securing it, changing it, and eventually handing it to whoever comes next. Historically the two moved together, because writing carefully was how you kept it ownable. AI broke that link. It collapsed the cost to write and did nothing to the cost to own — and code nobody fully understands is code nobody can safely change. Generating more of it, faster, doesn’t help if each line adds to a pile no one has read.
The part a straight-talking firm should say out loud
An engineering-led firm should tell you this plainly: this is no longer a hypothetical. Researchers are now tracking flaws traceable directly to AI coding tools — Georgia Tech’s project confirmed dozens of AI-linked CVEs through early 2026, with new ones appearing several times faster each month (Cloud Security Alliance, 2026). And it’s showing up on balance sheets: IBM found that 81% of executives say technical debt is already constraining their AI success, meaning the tool bought to go faster is now the drag. If a vendor’s answer to all of this is “don’t worry, the AI handles it,” they’re selling you speed and quietly handing you the ownership bill.
How we actually use AI on a build
None of this is an argument against AI. We build with it every day and it has genuinely made our senior engineers faster. It’s an argument about where the judgment sits. In practice that means three things, and they’re worth asking any partner about. The architecture is designed by a person before code is generated, so the AI fills in a structure rather than inventing one. Everything generated is reviewed and tested as if roughly half of it has a problem until proven otherwise — because, on the numbers, it might. And someone senior stays accountable for the whole, able to explain in two years why the system is built the way it is. That’s the difference between AI as an accelerant and AI as a slow leak.
The one question to ask before you commission anything
If you take one thing from this: don’t ask a software partner whether they use AI. Ask who’s accountable for what it produces — who designed the architecture, how generated code is reviewed and tested, and who will still be able to explain your system when you need to change it. A team that answers that crisply is using AI the way senior engineers do. A team that can’t is reselling you AI output with a margin on top, and every ownership cost lands on you. That’s the test we’d apply, and it’s the one we’re happy to be held to ourselves.
Frequently asked questions
Is AI-generated code safe to ship?
Not on its own. 2026 analysis found AI produces syntactically correct code nearly 100% of the time but with an average security pass rate of just 56% (Pagerly). Insecure output looks identical to secure output, so it needs senior review before it ships — not blind trust because it runs.
Should we avoid vendors who build with AI, then?
No — everyone builds with AI now. The thing to check is who’s accountable for the output: whether a person designed the architecture up front, whether generated code is reviewed and tested, and whether someone can still explain the system later. Use of AI isn’t the risk; unowned AI is.
What is technical debt, in plain terms?
It’s the future cost of code that was quick to produce but hard to understand, secure or change. In 2026 IBM found 81% of executives say it’s already slowing their AI efforts — the accumulated weight of code shipped faster than anyone could properly own it.
How would we know if our current software is carrying this risk?
Ask whether anyone senior designed the architecture before code was generated, whether what shipped was reviewed, and whether someone could explain it in two years. If the answers are vague, the debt is probably already building. A build review will tell you where you stand.
Commissioning something you need to trust? Book a Free Build Review
Call Eve on: +66 89 354 9916 or visit werehumans.com.