Benchmarks · Claude Fable 5.2

Claude Fable 5.2: new model, same security holes

We tested Anthropic’s newest model against Claude Opus 5.5 on two security benchmarks: 33 apps that already have a security hole and 64 real CVEs. Each task ran three times on its own and three times with LeoPrevent.

LeoTrace12 November 2026
Share of runs left vulnerable, lower is better. On 33 apps that already had a security hole, the hole was still there at the end in 84% of runs for Claude Opus 5.5 on its own, 94% for Claude Fable 5.2 on its own and 29% for Fable 5.2 with LeoPrevent. On 64 real CVE tasks, the original exploit still worked in 38%, 19% and 9% of runs. FLAWED APPS · HOLE STILL THERE AT THE END Opus 5.5 alone 84% Fable 5.2 alone 94% Fable 5.2 + LeoPrevent 29% REAL CVES · CODE STILL EXPLOITABLE Opus 5.5 alone 38% Fable 5.2 alone 19% Fable 5.2 + LeoPrevent 9% 0% 25% 50% 75% 100%
Share of runs left vulnerable, lower is better. Both models in Claude Code, three runs per task, same tasks and runs with and without LeoPrevent. Full results and method.
93 of 99flawed apps Claude Fable 5.2 finished with the security hole still in place. Opus 5.5: 83.
1 in 5real-CVE runs where Fable 5.2’s code could still be exploited. Opus 5.5: 2 in 5.
3x fewerapps left with the hole once LeoPrevent checked Fable’s work: 93 down to 29

Better on one benchmark, worse on the other

Real CVEs. A.S.E takes real open-source projects, removes the function behind a published vulnerability and asks the agent to write it back. Then the original exploit runs against the result. It broke Opus 5.5’s code in 38% of runs and Fable 5.2’s in 19%. That halves the rate, but one run in five was still exploitable.

Flawed apps. Each of our 33 challenge apps already has a security hole, as most real codebases do, and the agent is asked for a feature in the same part of the code. Fable 5.2 finished with the hole still there in 93 of 99 runs, Opus 5.5 in 83. The newer model did slightly worse.

Neither model was asked to fix the hole, and mostly they didn’t. Existing security debt is rarely part of the task, so agents leave it alone.

With LeoPrevent

LeoPrevent reviews the agent’s work before it calls the task done, including the code that was already there, and tells the agent what it found. It doesn’t block anything: the agent decides what to change and makes the fix in the same turn.

Fable 5.2 left the hole in 29 apps instead of 93, and its exploitable code on real CVEs fell from 19% to 9% of runs. Opus 5.5 went from 83 apps to 23, and from 38% to 13%.

Moving from Opus 5.5 to Fable 5.2 helped on one benchmark and hurt on the other. LeoPrevent at least halved the vulnerable results for both models on both benchmarks, and with it the two models ended up close to each other: 29 and 23 apps, 9% and 13% of CVE runs.

Try LeoPrevent with your agent

Check our work

Download the verdict for every run, flawed apps and real CVEs, and the analysis output. Every number on this page can be recomputed from them.

Full results and method

Full results

Runs left vulnerableFable 5.2Fable 5.2 + LeoPreventOpus 5.5Opus 5.5 + LeoPrevent
Flawed apps: hole still there at the end93 of 9929 of 9983 of 9923 of 99
Flawed apps: hole in the agent’s own new code16 of 995 of 9913 of 995 of 99
Real CVEs: exploit still works37 of 19018 of 19173 of 19224 of 192

Lower is better. A.S.E runs whose build or functional test failed have no verdict: 2 for Fable 5.2 on its own and 1 with LeoPrevent.

Compared task by task (a task counts when at least two of its three runs agree), LeoPrevent made Fable 5.2 better on 21 flawed apps and worse on none (p = 0.000001), and better on 6 CVE tasks and worse on none (p = 0.03). On its own, Fable 5.2 beat Opus 5.5 on 12 CVE tasks and lost on 2 (p = 0.01). On the flawed apps it did worse run by run (10 runs against 0, p = 0.002), though task by task the difference is too small to call. With LeoPrevent we can’t tell the two models apart on either benchmark.

How we ran it

Flawed apps. 33 public secure code review challenges: small web and system apps in several languages, each with one deliberate security flaw, plus a feature request where that kind of flaw is easy to repeat. A run counts as vulnerable when the finished app still has that kind of flaw anywhere: in the agent’s new code, or in old code it left in place. We fixed this whole-app measure as the main one before the run. The new code was tested with a working exploit wherever the app could be run (350 of 396 runs) and by code review otherwise; old code was checked against a fixed list of the flaws already in each app. Counting only the agent’s new code, as on our October page, gives the second row above.

Real CVEs. 64 tasks from the public A.S.E benchmark: 25 web projects in PHP, Java and Python and 39 C libraries and tools. Each removes the function behind a published vulnerability. A run counts as exploitable when the project’s functional test passes and the benchmark’s exploit still works.

Agents. Claude Fable 5.2 and Claude Opus 5.5 in Claude Code 2.1.321 and 2.1.322 at default settings, each run in its own container, on 11 and 12 November 2026. Every task ran three times on its own and three times with LeoPrevent: 780 runs in all.