SYNTHETIC: planted kind of flaw anywhere in the finished app (pre-registered primary) Claude Fable 5.2 without LeoPrevent 93/99 (93.9%) Claude Fable 5.2 with LeoPrevent 29/99 (29.3%) Claude Opus 5.5 without LeoPrevent 83/99 (83.8%) Claude Opus 5.5 with LeoPrevent 23/99 (23.2%) Fable: without vs with LeoPrevent: 99 run pairs, vulnerable 93 vs 29; only first 64, only second 0, exact sign test p=1.1e-19; task majority 33 tasks, 21 vs 0, p=9.5e-07 Opus: without vs with LeoPrevent: 99 run pairs, vulnerable 83 vs 23; only first 60, only second 0, exact sign test p=1.7e-18; task majority 33 tasks, 21 vs 0, p=9.5e-07 Without LeoPrevent: Opus vs Fable: 99 run pairs, vulnerable 83 vs 93; only first 0, only second 10, exact sign test p=0.002; task majority 33 tasks, 0 vs 3, p=0.25 With LeoPrevent: Opus vs Fable: 99 run pairs, vulnerable 23 vs 29; only first 6, only second 12, exact sign test p=0.24; task majority 33 tasks, 1 vs 4, p=0.38 SYNTHETIC: planted flaw in the agent's own new code Claude Fable 5.2 without LeoPrevent 16/99 (16.2%) Claude Fable 5.2 with LeoPrevent 5/99 (5.1%) Claude Opus 5.5 without LeoPrevent 13/99 (13.1%) Claude Opus 5.5 with LeoPrevent 5/99 (5.1%) Fable: without vs with LeoPrevent: 99 run pairs, vulnerable 16 vs 5; only first 14, only second 3, exact sign test p=0.013; task majority 33 tasks, 4 vs 0, p=0.12 Opus: without vs with LeoPrevent: 99 run pairs, vulnerable 13 vs 5; only first 9, only second 1, exact sign test p=0.021; task majority 33 tasks, 3 vs 0, p=0.25 Without LeoPrevent: Opus vs Fable: 99 run pairs, vulnerable 13 vs 16; only first 2, only second 5, exact sign test p=0.45; task majority 33 tasks, 0 vs 1, p=1 With LeoPrevent: Opus vs Fable: 99 run pairs, vulnerable 5 vs 5; only first 3, only second 3, exact sign test p=1; task majority 33 tasks, 0 vs 0, p=1 A.S.E: exploit still works against the rebuilt project Claude Fable 5.2 without LeoPrevent 37/190 (19.5%) Claude Fable 5.2 with LeoPrevent 18/191 (9.4%) Claude Opus 5.5 without LeoPrevent 73/192 (38.0%) Claude Opus 5.5 with LeoPrevent 24/192 (12.5%) Fable: without vs with LeoPrevent: 190 run pairs, vulnerable 37 vs 18; only first 21, only second 2, exact sign test p=6.6e-05; task majority 63 tasks, 6 vs 0, p=0.031 Opus: without vs with LeoPrevent: 192 run pairs, vulnerable 73 vs 24; only first 52, only second 3, exact sign test p=1.5e-12; task majority 64 tasks, 16 vs 1, p=0.00027 Without LeoPrevent: Opus vs Fable: 190 run pairs, vulnerable 72 vs 37; only first 40, only second 5, exact sign test p=7.9e-08; task majority 63 tasks, 12 vs 2, p=0.013 With LeoPrevent: Opus vs Fable: 191 run pairs, vulnerable 23 vs 18; only first 8, only second 3, exact sign test p=0.23; task majority 64 tasks, 2 vs 1, p=1 SYNTHETIC grading of the new code: with LeoPrevent, code review: 31, with LeoPrevent, runtime exploit: 167, without LeoPrevent, code review: 15, without LeoPrevent, runtime exploit: 183 A.S.E runs without a verdict: Claude Fable 5.2, with LeoPrevent: 1 BROKEN, Claude Fable 5.2, without LeoPrevent: 2 BROKEN A.S.E web: Fable without 9/74 (12.2%); Fable with 2/75 (2.7%); Opus without 31/75 (41.3%); Opus with 0/75 (0.0%) A.S.E C: Fable without 22/78 (28.2%); Fable with 14/78 (17.9%); Opus without 25/78 (32.1%); Opus with 15/78 (19.2%) A.S.E C held-out: Fable without 6/38 (15.8%); Fable with 2/38 (5.3%); Opus without 17/39 (43.6%); Opus with 9/39 (23.1%)