Blog · Benchmarks
Coding agents copy your security mistakes
We spent the last two months watching the newest coding agents write code on apps with a security flaw in them. They're good at writing code that works, and just as good at copying the flaw that was already there.
The numbers are on two results pages, security interview challenges and A.S.E, with the verdicts behind them. Here's how we got there.
Why we started measuring
LeoTrace exists because of a pattern we kept seeing in code written by agents. Ask an agent to add a change-password endpoint "reusing our existing password handling", and it will do exactly that, including the unsalted MD5 the old code used. The agent can write secure code, but nobody asked it to, and the code it was told to follow did it the insecure way.
So we wanted to know how often that happens. Florian, our co-founder and CTO, had written 33 small apps that people use to prepare for application security interviews, each one hiding a single flaw. We wrote one ordinary feature request per app, gave it to GPT-6 Astra, GPT-6.1 Sol and Claude Opus 5.5, three times each, and checked whether their new code brought the flaw along.
It did in 26%, 27% and 13% of runs. Four of the challenges caught all three agents on every single run: the MD5 password hashing, an admin login checked against a password written into the source, an Nginx rule that blocks the exact path but lets slightly different spellings through, and an XSLT report endpoint that reads files from the server. The last one is my favourite, because many runs added a guard against reading files, and none of the guards held.
Then on real code
Our own challenges are a good start, but we wrote them, so we went looking for a benchmark built by someone else. A.S.E, from Tencent and five Chinese universities, takes real open-source projects just before a security fix, removes the function the fix changed, and asks the agent to write it again. Its prompt even tells the agent to "make every effort to avoid introducing security vulnerabilities". Then it runs the original exploit against the result.
Across all 38 tasks we could measure, the exploit still worked in 35% to 40% of runs, for all three agents. What surprised me was which mistakes they made. Code injection, the kind every security course talks about, was rare, though A.S.E doesn't have many injection tasks to begin with. What the agents kept shipping was code that works but is missing a check, like a page that changes data on a plain link click, a file path that can walk out of its folder, or a page without its permission check.
Fixing the benchmark along the way
Running A.S.E properly turned out to be half the work (more than half, really). The removed function was still sitting in the repository's git history, and agents with a shell sometimes found it. In 29 tasks the exploit never really tested the code the agent wrote, so the verdict said nothing about the agent. Two tasks could never produce a verdict at all because of bugs in the harness. We sealed the history, set those tasks aside, fixed what we could, and reported all of it upstream as nine issues and four pull requests. If you run A.S.E yourself, read them before you trust a number from it.
Where LeoPrevent comes in
This is what we built LeoPrevent for. It reviews what the agent wrote when it finishes its turn and tells it what it found, so the agent can fix it before the change lands. The agent decides what to do with it. On A.S.E, with LeoPrevent the exploit worked in 15% of Claude Opus 5.5's runs instead of 40%, and in 19% to 22% of the GPT runs instead of 35% to 38%. No task got worse for any of them. On our own challenges, which are our test set rather than independent evidence, both GPT models shipped the planted flaw well under half as often: 26 runs to 10 for GPT-6 Astra and 27 to 11 for GPT-6.1 Sol. Part of what is left there is a flaw LeoPrevent did catch: on the Nginx challenge it flagged the problem every time, but as a suggestion for the developer, and our runs have no developer to say yes.
What is next
We'll keep adding agents, models and benchmarks we didn't write, and report them the same way, with the analysis written down before the run, three runs per task and the verdicts behind every result published. If you recompute something and get a different answer, I would love to hear about it.
And I am curious: which security mistake have you seen your coding agent copy?