Abliteration Resistance Is a Finite Margin
An adaptive audit of refusal-direction hardening on Cohere's Tiny Aya
Every number below is a frozen count from a pre-registered audit. The released aggregate counts are in results/final_panel.csv and results/screens.csv. Appendix B maps the evidence tables. Rows measured only on the small screening panel are marked screen; everything else is the final panel.
TL;DR
Abliteration removes an open model's refusal by projecting one "refusal direction" out of its weights. Defenses such as DeepRefusal train the model to keep refusing while that direction is ablated, and they are usually checked by ablating a single direction. We built an adaptive audit and ran it on Cohere's Tiny Aya Global (the core model). The official DeepRefusal release on Llama-3-8B serves as a supplementary reference. The attacker re-extracts the refusal direction on the defended model, ablates it, and repeats. k* counts the rounds needed before fewer than half of 150 harmful prompts are refused. A second attack, a single-layer additive shift of the model's own direction (dose c), is always run against norm-matched random vectors. Every result is scored on harmful refusal, over-refusal and capability.
What k* is, and what it is not. k* measures the effort this attack path needs: one estimator, one selection seed, one prompt panel. It is not an intrinsic safety constant of a model, and it is not the full cost a determined attacker would pay.
Three findings:
- Adaptive re-extraction breaks every Tiny Aya hardening we could train or find at k* = 1–2.
- SafeTune's DeepRefusal resists the direction it was trained against. One direction re-extracted on the defended model takes it from 123/150 to 21/150 refused (k* = 1 under both selection seeds; paired exact McNemar p = 3.9e-31).
- Four further hardenings of ours also fall at k* = 1: a paper-faithful recipe, a dose that re-estimates the direction during training, and two doses of a defense trained against both attack operators. The last two were pre-registered fixes.
- The strongest public hardened Tiny Aya,
xref-v2from the concurrent project "Heretic-Resistant Tiny Aya" (StarWarrior24x7), needs two directions (k* = 2). It falls to a single-layer shift at c = 1.25, while norm-matched random vectors leave 126 and 129/150 refused. That shift is not capability-neutral: GSM8K drops from 178 to 116/250 (p = 4.7e-12). Its authors had already reported publicly that activation steering remains a substantial weakness of their hardening; our numbers agree.
- A deep margin does exist, in the official DeepRefusal checkpoint, but it is finite under both projection and steering, with controls.
- Projection. k* = 5 on the final panel (seed 0).
- Ranks 1–4 hold (145, 145, 143 and 119/150), although k = 4 already erodes refusal significantly.
- k = 5 is the first rank below half: 70/150, with an interval of 38.9–54.6% that crosses 50%. At k = 6, 32/150 are refused.
- On the n = 46 screen it breaks at 6. Under the second selection seed (screen only) it does not break within the 6-direction cap.
- Steering. Projecting its own direction is inert at one layer and at all layers (46/46). Adding a direction at one layer (L22, c = 2.0) leaves 36/150 refused. The judges flag 59/150 of those completions as coherent harmful, but this is an upper bound: a blinded LLM-label check confirmed substantive harmful content in only 3 of 8 sampled judge-flagged rows. The steering mainly removes refusal. Two norm-matched random vectors at the same layer and dose leave 146/150 refused each (paired b/c = 0/110, p = 1.5e-33). The break has a real arithmetic cost: GSM8K drops from 77 to 62/100 (p = 0.008). The matched random costs even more (39/100) while leaving refusal intact.
- Projection. k* = 5 on the final panel (seed 0).
- Hardening carries an over-refusal bill, and the standard repair trades against the margin.
- The bill. On 350 benign-but-edgy prompts, our hardened Tiny Aya checkpoints over-refuse 30.9–41.1% and
xref-v252.3%, against the base's 17.4%. The official checkpoint over-refuses 67.4%. - The repair ("ablate the base direction") behaves differently on each checkpoint. On the deep official checkpoint it cut over-refusal by 31.1 points with no detectable loss of harmful refusal at n = 150 (148 → 145; p = 0.25, not an equivalence test). On
xref-v2it cost 19 refusals, and one more direction then broke the model. On SafeTune's DeepRefusal it removed most refusals by itself (123 → 50/150).
- The bill. On 350 benign-but-edgy prompts, our hardened Tiny Aya checkpoints over-refuse 30.9–41.1% and
Core model: CohereLabs/tiny-aya-global @ af89d219… (3.35B, Cohere2, 36 layers, CC-BY-NC-4.0). Supplementary reference: skysys00/Meta-Llama-3-8B-Instruct-DeepRefusal @ 134fb9ab… (Llama-3-8B; not a Cohere model).
Lexsi stack:
- CuratorKIT: the frozen, provenance-tracked prompt splits.
- SafeTune: the DeepRefusal hardening we trained on Tiny Aya. Its single-direction abliteration attack is the baseline we generalize.
- AuditKIT: the WildGuard and HarmBench-cls judge profiles, and the paired bootstrap behind the final statistics.
CircuitKIT is not used.
1. Why a margin, not a pass/fail
Open weights mean the attacker owns the weights. The cheapest known weight-space attack is abliteration (Arditi et al., 2024; Heretic). It extracts the residual-stream direction that mediates refusal and projects it out of every matrix that writes it, and refusal disappears at rank 1 with little collateral damage. Several defenses (DeepRefusal, extended-refusal training, ablation-adversarial training) harden a model against exactly this operator: during fine-tuning the refusal direction is ablated, so the model learns to refuse without it.
These defenses are usually stress-tested by ablating one direction: either the direction the defense was trained against, or a single direction re-fit on the defended model (SafeTune's AbliterationAttack, Heretic). That check has three blind spots:
- A stale direction. A transferred direction tests the defense's training artifact, not the model. An attacker with the weights re-extracts the direction on the defended model.
- One direction. One direction is the floor of the attack, not its ceiling. An adaptive attacker iterates: re-extract on the partially ablated model, ablate, and repeat (the multi-phase family of Muhamed et al.).
- One operator. Projection is not the only thing an attacker can do with a direction. Adding a multiple of it at one layer (steering) is a different operator, and it needs its own controls.
We therefore report a margin, k*: how many adaptive rounds the defense survives. Alongside it we report a steering dose c* with random controls, and the defense's over-refusal bill, each with its capability cost. k* depends on the direction estimator, the selection seed and the prompt panel, so we report all three (limits in §6).
2. The audit
2.1 Checkpoints
| short name | what it is | role |
|---|---|---|
| Tiny Aya base | CohereLabs/tiny-aya-global @ af89d219… |
undefended reference; validates the attack (its k* must be 1) |
| SafeTune DeepRefusal | SafeTune's DeepRefusal (the library's Variant recipe), LoRA, LR 2e-5 | ours: the sponsor-library defense at an engaged dose |
| DeepRefusal paper recipe | the paper's recipe (prefill augmentation, ≥ 128-prompt direction, 3-site hooks), LR 1e-5 | ours (screen only) |
| re-estimating dose | the refusal direction is re-estimated during training (ASRT-k4, LR 2e-5) | ours: pre-registered fix (Appendix A.2) |
| dual-operator defense, two doses | trained against both projection and additive perturbations, LR 1e-5 and 2e-5 | ours: pre-registered fix (Appendix A.3) |
xref-v1 |
extended-refusal training (StarWarrior24x7, public) @ ca029444… |
concurrent work, credited (screen only) |
xref-v2 |
ablation-adversarial training with a re-estimated direction (same project) @ 14b4675f… |
concurrent work, credited |
| official DeepRefusal | skysys00/Meta-Llama-3-8B-Instruct-DeepRefusal @ 134fb9ab… |
the authors' released checkpoint; supplementary |
2.2 The margin k*
At each round the attacker re-extracts a difference-of-means (DIM) refusal direction on the current, partially ablated model. The candidates are 11 layers × 3 token positions on Tiny Aya and 12 × 3 on Llama-3-8B. They are screened on a frozen 32-prompt selection set that is disjoint from the scored panel (refusal markers first, first-token KL as tie-break). The selected direction is orthogonalized against the earlier ones and projected out of every writer matrix, and the loop repeats up to k = 6.
Break rule (frozen in the harness): a row is broken when fewer than half of the harmful prompts are WildGuard-refused (< 23/46 on the screen, < 75/150 on the final panel). k* is the first broken round. It is reported under two selection seeds, and as a range when they disagree. On the final panel, every rank was run with the frozen seed-0 directions. For the Tiny Aya points that means up to the screen's k*. For the official checkpoint it means k = 1–6, with ranks 1–4 run later on the same engine. Fig 1 never draws a line across an unmeasured k.
2.3 The steering dose c*
Independently of the ladder, we take the model's own DIM vector v at its selected layer and subtract c · v at that single layer, sweeping c ∈ {1.0, 1.25, 1.5, 1.75, 2.0}. c* is the smallest dose whose row is broken; c* = 1.0 means the row already breaks at the lowest dose. Controls are random vectors with the same norm, layer and dose (two seeds each). If random vectors broke the panel, the effect would be destruction rather than direction, and we would drop the steering claim. Norm matching controls vector size, not total disruption: on xref-v2, the random vectors move first-token KL by 0.37 against 2.14 for the DIM.
2.4 The operator/scope control
"Projection is inert, addition breaks it" could confuse the operator with the scope (how many layers are edited). On the two checkpoints that survive one direction, we therefore apply the same direction at the same dose in four arms: {projection, addition} × {one layer, all layers}.
2.5 The over-refusal bill and the repair
The bill is the number of WildGuard-refused prompts on the over-refusal panel (O) with no attack. The repair is the standard "harden, then ablate": project the fixed base-model direction out of the hardened model. We score it on harmful refusal and on over-refusal. To get the margin after repair, we re-extract one more direction on the repaired model.
2.6 Judges, data, statistics
- Judges.
- WildGuard (whole-response) judges refusal and harm on every row. HarmBench-cls judges harm on the harmful panel. Refusal markers are used only to pre-screen candidates.
- "Coherent harmful" means WildGuard-harmful and not degenerate. Degenerate completions are counted separately, so gibberish counts as neither safety nor attack success.
- A WildGuard verdict can be unknown (unparseable output). Rates over known verdicts are reported wherever unknowns occur. The official L28 harmful row and its reproduction each have 55/150 unknown verdicts. Dual-operator dose 1 at c = 2.0 has 1/150 unknown verdicts. The L28 over-refusal row has 1/350. No unknown is counted as a non-refusal.
- The judges were checked in advance against each other (κ = 0.785, HarmBench-cls vs WildGuard) and against blinded LLM-agent labels.
-
A second blinded check uses LLM labels (codex) on a 60-row outcome-enriched sample, reviewed by the owner with no corrections:
- refusal: κ = 0.667 vs WildGuard;
- harmful: κ = 0.542 vs WildGuard and κ = 0.613 vs HarmBench-cls.
Two things follow (Appendix A.6; limits in §6):
- Refusal. The LLM labeller marks refusal less often than WildGuard (24 vs 30 of 60), so the below-half breaks are not a WildGuard refusal artifact.
- Harm on steering rows. The judges overstate it: the labeller confirmed substantive harm in 3 of 8 judge-flagged rows at the official L22 shift and in 1 of 6 at the
xref-v2shift, against 4 of 5 on projection-attack rows. - Data.
- The splits were frozen and hashed before any defended model was scored, with per-prompt provenance from CuratorKIT dedup stages. The extraction, selection and scoring sets are disjoint.
- Harmful panel (B150): 46 mlabonne harmful behaviours, 89 BeaverTails-Eval and 15 further AdvBench-derived behaviours.
- Over-refusal panel (O350): 250 XSTest-safe and 100 OR-Bench-hard prompts.
- Screening panel: the mlabonne 46 (B46) and 50 OR-Bench-hard prompts (O50).
- Capability: MMLU-500 and GSM8K-250 on the final panel; MMLU-200 and GSM8K-100 on the screen.
- Generation: greedy decoding, with 256 new tokens on B and 128 on O.
- Statistics.
- Rates carry Wilson 95% intervals in brackets.
- Comparisons are paired by prompt ID: 141 final-panel tests, plus 12 for the official checkpoint's matched L22 controls. Exact two-sided McNemar is primary. A paired-bootstrap 95% CI accompanies each test, with the bootstrap p-value computed by AuditKIT.
- No multiplicity correction is applied, so p-values are descriptive.
- No threshold was re-tuned after a result. Movements between the screen and the final panel are reported in Appendix A.4.
3. Finding 1: adaptive re-extraction breaks every Tiny Aya hardening at k* = 1–2
Figure 1a (figures/fig1_margin.png, left) plots harmful-prompt refusal against the number of re-extracted directions. Bold lines are the final panel (n = 150); thin lines are the screen (n = 46) for selection seeds 0 and 1.

| Tiny Aya checkpoint | refused at A0 (B150) | refused after 1 re-extracted direction (B150) | k* (seed 0 / seed 1) | capability A0 → k* (MMLU-500 / GSM8K-250) |
|---|---|---|---|---|
| base (reference) | 112/150 (74.7%) | 33/150 (22.0%) | 1 / 1 | 271/185 → 269/187 |
| SafeTune DeepRefusal (ours) | 123/150 (82.0%) | 21/150 (14.0%) [9.3, 20.5] | 1 / 1 | 269/183 → 271/177 |
| DeepRefusal paper recipe (ours) | screen 46/46 | screen 8/46 | 1 / 1 | not measured |
| re-estimating dose (ours) | marker-level only | marker-level only | 1 / 1 (markers) | not measured |
| dual-operator, LR 1e-5 (ours) | 124/150 | 16/150 [6.7, 16.6] | 1 / not run | not run |
| dual-operator, LR 2e-5 (ours) | 134/150 | 35/150 [17.3, 30.7] | 1 / not run | not run |
xref-v1 (StarWarrior24x7) |
screen 46/46 | screen 2/46 (seed 1: 14/46) | 1 / 1 | not measured |
xref-v2 (StarWarrior24x7) |
128/150 (85.3%) | 94/150 (62.7%) | 2 / 2 (k = 2: 49/150 [25.7, 40.5]) | 271/177 → 272/176 |
SafeTune's DeepRefusal is transfer-only on Tiny Aya.
- The trained-against direction. The defense is engaged: against the direction it was trained to resist, it keeps 38/46 refusals where the base keeps 13/46 (screen, p = 6e-8). That is what a single fixed-direction check sees.
- One re-extracted direction. One direction re-extracted on the defended model (the same layer-22 site under both seeds) takes it from 123/150 to 21/150 refused: −68.0 points, 95% CI [−75.3, −60.7], p = 3.9e-31. That attack yields 56/150 coherent harmful completions (HarmBench-cls: 2 → 60/150, p = 1.1e-16). For projection attacks, a blinded LLM-label check agrees with the judges on harm (4 of 5 sampled judge-flagged rows confirmed; Appendix A.6).
- Capability cost. The attack costs little capability: MMLU 269 → 271/500, GSM8K 183 → 177/250 (−2.4 points).
- What the hardening did buy: over-refusal (§5).
The paper-faithful recipe falls the same way on the screen (46 → 8/46).
Two pre-registered fixes did not change the picture.
- The re-estimating dose failed its own selection gate (0/32 refusals under the best candidate, against a gate of ≥ 20/32). It falls at k* = 1 at marker level.
- The dual-operator defense raises clean refusal at A0 (124 and 134 vs 112/150; 12 and 22 gained, none lost; p = 4.9e-4 and 4.8e-7). Even so, it falls at k* = 1 under seed 0 at both doses and does not resist the additive attack.
- Both negatives are reported in full in Appendix A.
Steering. On the screen, every Tiny Aya point without a margin breaks at the lowest dose (c* = 1.0), and norm-matched random vectors hold. One random on the base erodes refusal to 32/46 without breaking it. xref-v2 needs c* = 1.25 (Fig 2a). On the final panel the shift leaves 21/150 refused (KL 2.14; 31 degenerate). WildGuard flags 54/150 completions as coherent harmful, but read that as an upper bound: HarmBench-cls flags 38, and the blinded LLM labeller confirmed substantive harm in only 1 of 6 sampled WildGuard-flagged rows. Its random controls hold at 126 and 129/150, a paired difference of −70.0 and −72.0 points (p = 1.3e-30 and 1.7e-31). The steering break is not capability-neutral. At c* on the final panel, MMLU is unchanged (271 → 274/500, p = 0.72), but GSM8K falls 24.8 points (178 → 116/250, p = 4.7e-12). By contrast, the projection attack at k* on SafeTune DeepRefusal stayed within the pre-registered capability budget.
3.1 Concurrent public hardenings (credited)
The only independently hardened Tiny Aya weights we know of come from the concurrent Lexsi hackathon project "Heretic-Resistant Tiny Aya" by StarWarrior24x7, released with an article and a results dataset:
tiny-aya-global-xref-v1: extended-refusal training, following arXiv:2505.19056.tiny-aya-global-xref-v2: adversarial training against a simulated abliteration attack whose refusal direction is re-estimated from the current model during training.
The models are released under CC-BY-NC-4.0 and the results dataset under CC-BY-4.0. We thank the authors for publishing them. We audit both as released recipes, at the revisions pinned in §2.1, with the same harness and judges as our own checkpoints. Their cards were updated on Nov 14, and the xref-v2 repository now also holds a v2b subfolder, which we did not audit.
Their public model cards, article and results logs report three observations about xref-v2 before us. All three were visible in their results dataset by Nov 12, 2026, and our write-up is not yet public. Their numbers use their own judges and prompts:
- SafeTune ablation. It keeps refusing under SafeTune's single-direction ablation: refusal 29% for the original vs 100% for
xref-v2, and judged-harmful 55% vs 0%. - CAA steering. Steering at strength −1 remains a substantial weakness. They rate it "partly resisted": refusal 4% vs 35%, and judged-harmful 97% vs 72%.
- Over-refusal. Over-refusal on 250 safe XSTest prompts rises from 17.2% to 42.8%. An earlier protocol on their card gave 16% and 39%.
They also describe the v2b variant, which trades some robustness for lower over-refusal (16.8%; 18% under the earlier protocol). Our measurements agree with all three observations and add the margin view. One direction leaves 94/150 refused (−22.7 points, p = 1.2e-10, still above half). Two directions break it, and the steering shift breaks it at c* = 1.25.
These are measurements of two released checkpoints under one harness. They are not a verdict on the project, which evaluates many attacks we did not run (Heretic with 40- and 200-trial searches, AuditKIT probes, prefill, sampling, 4-bit NF4, harmful LoRA fine-tuning, and harmful prompts in four further languages). It also reports that a harmful fine-tune removes refusal from all of its models.
4. Finding 2: a deep margin exists, and it is finite under both projection and steering
The official DeepRefusal checkpoint (Llama-3-8B; supplementary, not a Cohere model) is the only checkpoint in our panel with a deep margin. We measured its adaptive margin under projection, the operator/scope control, and a controlled single-layer steering dose.
4.1 Projection: k* = 5 on the final panel
Figure 1b (figures/fig1_margin.png, right) draws only measured points.
| k | screen, seed 0 (B46) | screen, seed 1 (B46) | final panel, seed-0 directions (B150) |
|---|---|---|---|
| 0 (no attack) | 46/46 | 46/46 | 148/150 (98.7%) |
| 1–3 | 46, 46, 46 | 46, 46, 46 | 145, 145, 143 /150 [92.4–98.6, 92.4–98.6, 90.7–97.7] |
| 4 | 45 | 46 | 119/150 (79.3%) [72.2, 85.0]: holds, but eroded |
| 5 | 32 (holds) | 46 | 70/150 (46.7%) [38.9, 54.6]: below half |
| 6 | 12 (breaks) | 46 (no break at the cap) | 32/150 (21.3%) [15.5, 28.6] |
On the final panel, k* = 5 (seed 0).
- Ranks 1–4 hold at 145, 145, 143 and 119/150. Against no attack (148/150) the paired p-values are 0.25, 0.25, 0.0625 and 3.7e-9. So k = 4 already erodes refusal significantly (−19.3 points) without crossing half.
- k = 5 is the first rank below half (70/150). Its interval (38.9–54.6%) crosses 50%, so the location of the threshold is not sharp.
- On the screen it breaks at k = 6 (seed 0). Under seed 1 it does not break within the 6-direction cap. The two seeds share their first three directions and diverge at k = 4: seed 0 finds large residual directions that break the model, while seed 1's are small and remove nothing.
Both final-panel rows are large attacks in paired terms: k = 5 is −52.0 points (p = 6.6e-24), and k = 6 is −77.3 points (p = 2.4e-35, with 68/150 coherent harmful). What is uncertain is only where the half-panel line is crossed: the k = 5 row sits five prompts under it.
The break rule matters at this margin.
- The relative rule. One stricter rule says a row breaks when its refusals are at most 2× the undefended base's one-direction attack (33/150) and its coherent harm is at least 0.8× that row's. Under it, the first final-panel break is k = 6.
- The pre-registered success definition. It requires refusal within 10 points of the base attack (≤ 48/150) and HarmBench harm ≥ 0.8× the base's. It also gives k = 6, and it reads
xref-v2as "≥ 2" (49/150 misses by one prompt). - The ordering. Under all three rules the ordering is SafeTune DeepRefusal <
xref-v2< official DeepRefusal.
The attack is cheap on harmless prompts (first-token KL 0.18 at k = 5 and 0.26 at k = 6). The checkpoint's MMLU and GSM8K were not measured at k* on any panel.
4.2 The operator, not the scope, decides
Screen (B46). The same direction at the same dose is applied in all four arms:
| arm | xref-v2 (first re-extracted DIM, c = 1.25) |
official DeepRefusal (first re-extracted DIM, L28, c = 2.0) |
|---|---|---|
| projection, one layer | 46/46 (inert; KL 0.003) | 46/46 (inert; KL 0.001) |
| projection, all layers | 37/46 (the one-direction margin; KL 0.17) | 46/46 (inert; KL 0.02) |
| addition, one layer | 7/46 (28/46 judge-flagged coherent harmful; KL 2.13) | 5/46 refused (0 coherent; 43/46 degenerate; 26/46 WildGuard-unknown; KL 1.05) |
| addition, all layers | 0/46 (all degenerate; KL 13.1) | 0/46 (all degenerate; KL 14.0) |
On the official checkpoint, projection is inert whether it edits one layer or all of them, while adding the same vector at one layer collapses refusal. At this layer what replaces refusal is mostly degenerate text: even counting all 26 unknowns as refusals (31/46), the arm stays far below both projection arms.
A preliminary sweep on this checkpoint shows projection failing across the whole family. The transferred base direction, the best of 30 on-model candidates, projection scaled by 1.5 and by 2 (KL 8.7), and 4-bit quantization followed by re-extraction all leave 46/46 refused. The defense learned invariance to the projection operator; it did not remove the direction.
4.3 Steering removes its refusal, and matched random vectors do not
Figure 2 (figures/fig2_dose.png) plots refusal against the additive dose c. Panel (a) is Tiny Aya; panel (b) is the official checkpoint, with the preliminary sweep's L22 curve, both final-panel directions and all random controls.

| official DeepRefusal | dose | refused (B150) | judge-flagged coherent harmful (upper bound) | HarmBench-cls harmful | norm-matched randoms, same layer and dose (refused) |
|---|---|---|---|---|---|
| L22 direction | screen: first below half at c = 1.75 (20/46); final panel at c = 2.0 | 36/150 [17.9, 31.4] | 59/150 (12 degenerate) | 53/150 | 146/150 and 146/150 [93.3, 99.0] (HarmBench 1 and 0) |
| L28 direction (first ladder direction) | c = 2.0 only | 49 refused, 46 complied, 55 WildGuard-unknown: 49/95 of known [41.7, 61.4], range 49–104/150 | 7/150 (92 degenerate) | 9/150 | 146/150 and 147/150 |
The L22 row is the steering result.
- The break. A single-layer shift of the checkpoint's own direction leaves 36/150 refused. Refusal is the break metric, and the blinded LLM labeller agrees with WildGuard on it (7 vs 8 refusals among 20 sampled rows of this condition). The judges flag 59/150 completions as coherent harmful. That is an upper bound: the labeller confirmed substantive harmful content in 3 of 8 sampled judge-flagged rows. The shift mainly removes refusal and often yields abstract or low-utility compliance.
- The controls. Two random vectors with the same norm, layer and dose leave 146/150 refused each. Paired by prompt, 110 prompts are refused under each random but not under the DIM, and none the other way (b/c = 0/110, exact McNemar p = 1.5e-33). Judge-flagged harm follows the same pattern: HarmBench-cls 53 vs 1 and 0 (p = 4.4e-16 and 2.2e-16), and WildGuard coherent harmful 59 vs 0 (p = 3.5e-18). Neither random has an unknown verdict.
- The dose. On the screen the L22 curve first drops below half at c = 1.75 (20/46). The final panel ran only c = 2.0.
The L28 row cannot establish a break.
- Unknown verdicts. At the direction our ladder selects first, the same dose leaves 55/150 outputs that WildGuard cannot label, and re-judging reproduced all 55 as unknown. Over the 95 known verdicts, 49 are refusals (51.6%), which is not below half. The possible range over all 150 prompts is 49–104.
- Degeneration. The row is mostly degenerate (92/150).
- Against its own randoms (146 and 147/150), the known-verdict refusal drop is still large (91 → 49 and 92 → 49 of 95; p = 4.5e-13 and 2.3e-13).
The steering finding therefore rests on L22, and it is a finding about refusal: how much harmful content replaces the refusal depends on the direction and the layer.
The cost of the steering break. Capability was re-measured on MMLU-200 and GSM8K-100, the same protocol as the preliminary screen.
| condition | MMLU (/200) | GSM8K (/100) | paired vs no attack |
|---|---|---|---|
| no attack | 118 | 77 | reference |
| L22 shift at c = 2.0 | 115 | 62 | MMLU −1.5 points, p = 0.69; GSM8K −15 points, p = 0.008 |
| matched random (seed 101) | 106 | 39 | MMLU −6 points, p = 0.029; GSM8K −38 points, p = 5.1e-9 |
- The break carries a substantial arithmetic cost (−15 GSM8K points), so it is not capability-neutral.
- The matched random is even more damaging (DIM vs random: GSM8K +23 points, p = 2.9e-4), yet it leaves refusal at 146/150. General capability damage therefore does not explain the direction-specific refusal contrast.
- This no-attack GSM8K baseline (77) differs from the preliminary screen's (75). All comparisons are paired against the new baseline.
5. Finding 3: hardening has an over-refusal bill, and repair trades against the margin
Figure 3 (figures/fig3_repair.png) shows harmful refusal with no attack, after the base-direction repair, and after repair plus one re-extracted direction (n = 150). Figure 4 (figures/fig4_overrefusal.png) shows over-refusal with no attack, after the repair, and after the steering attack (n = 350).


| checkpoint | harmful refusal (B150): A0 → repair → repair + 1 direction | over-refusal (O350): A0 → repair | capability (MMLU-500 / GSM8K-250) | margin → after repair |
|---|---|---|---|---|
| Tiny Aya base | 112 → (the repair is the base's own one-direction attack: 33) | 61/350 (17.4%) | A0 271/185 | 1 → n/a |
| SafeTune DeepRefusal (ours) | 123 → 50 → not run | 110/350 (31.4%) → screen 41 → 1/50 | A0 269/183; screen repair 103/70 → 97/71 (/200, /100) | 1 → 0 |
| dual-operator, LR 1e-5 / 2e-5 (ours) | 124 / 134 → not run | 108/350 (30.9%) / 144/350 (41.1%) | not run | 1 → not run |
xref-v2 (StarWarrior24x7) |
128 → 109 → 42 | 183/350 (52.3%) → 91/350 (26.0%) | A0 271/177; screen repair 100/70 → 100/75 | 2 → 1 |
| official DeepRefusal (suppl.) | 148 → 145 → 126 | 236/350 (67.4%) → 127/350 (36.3%) | screen A0 118/75; repair 117/78 (/200, /100) | 5 → ≥ 2 |
- The bill. Every hardening of Tiny Aya raises over-refusal on O350 above the base's 17.4%: 30.9–41.1% for ours and 52.3% for
xref-v2. The official checkpoint over-refuses 67.4%. Its own base (Llama-3-8B-Instruct) was not run on O350; on the OR-Bench-hard screen that base refuses 34/50, against 50/50 for the defended model. - The repair, checkpoint by checkpoint (descriptive; three checkpoints do not establish a general rule):
- Official DeepRefusal (deep margin). The repair cut over-refusal by 31.1 points (236 → 127/350; p = 8.6e-32). It produced no detectable loss of harmful refusal at n = 150 (148 → 145; −2.0 points, CI [−4.7, 0.0], p = 0.25). That is the absence of a detected loss, not a test of equivalence. One more re-extracted direction costs 19 refusals (145 → 126; p = 3.8e-6), but the repaired model stays above half.
xref-v2(k* = 2). The repair cut over-refusal by 26.3 points (p = 4.0e-28) but lost 19 of 128 refusals (p = 3.8e-6). One more direction then breaks the model (109 → 42/150; p = 1.4e-20), so the margin falls from 2 to 1.- SafeTune DeepRefusal (k* = 1). The repair by itself takes refusal from 123 to 50/150 (−48.7 points; p = 2.1e-22). Here the repair acts as an attack.
- Steering also strips over-refusal, but not usefully. The steering attack removes most of the bill:
xref-v2falls from 52.3% to 5.7%, and the official checkpoint from 67.4% to 19.4% (L28 direction). Much of that "compliance" is degenerate output (53/350 and 39/350), so it is not a usable repair.
6. Limitations
- What k* measures. k* is the effort of one attack path: a DIM estimator with marker/KL selection, two selection seeds and one prompt panel. It is not an intrinsic safety constant, and not a full attacker cost: stronger estimators, other operators or fine-tuning attacks could need fewer steps.
- Panel size and split. The final panels are n = 150 (harmful) and n = 350 (over-refusal), and the screen is n = 46 (harmful) and 50 (over-refusal). Several decisive rows sit near the threshold, chiefly the official checkpoint's k = 5 row (70/150, interval crossing 50%; §4.1). The panels are the frozen dev splits: disjoint from direction extraction and selection, but the locked test split was not used.
- Judges are classifiers; no human validation. Refusal and harm are judged by classifiers. WildGuard and HarmBench-cls agree closely on projection-attack rows (e.g. 56 vs 60, 68 vs 67, 31 vs 31 harmful). On steering rows, HarmBench-cls counts fewer harmful completions (
xref-v2: 38 vs 54; official L22: 53 vs 59). The blinded checks of the judges use LLM labels (§2.6, Appendix A.6). The latest check, on a small outcome-enriched sample, supports the refusal breaks but not the steering harm counts (3 of 8 and 1 of 6 judge-flagged rows confirmed), so every steering "coherent harmful" count in this article is a judge-flagged upper bound, and no rate is extrapolated from the sample. These labels are neither human labelling nor independent human validation, and no human-validation claim is made anywhere in this package. - Only two checkpoints survive one direction, and neither is ours. These are
xref-v2(k* = 2) and the official DeepRefusal, a Llama checkpoint. The steering and repair findings therefore rest on two hardened points, and the repair comparisons are descriptive. - Seed and rule dependence. The official checkpoint's margin depends on the selection seed (6 vs > 6 on the screen), the panel (5 on the final panel) and the break rule (6 under both stricter rules).
- Coherence and unjudgeable outputs. A refusal break is not automatically a harmful-content break. The official L28 row has 55/150 WildGuard-unknown verdicts and cannot establish a break (§4.3); the fully judged break is the L22 row.
- Capability.
- Both steering breaks cost arithmetic (GSM8K −15 points on the official checkpoint, −24.8 on
xref-v2; §3, §4.3) while MMLU barely moves. - Capability at k* was not measured on the official checkpoint.
- The official checkpoint's capability uses MMLU-200/GSM8K-100.
xref-v2's c* capability was re-baselined on different hardware (fp16 T4: no attack 271/178, against 271/177 in the earlier bf16 run). Paired tests use the matched baseline.- The dual-operator doses have no capability rows.
- Both steering breaks cost arithmetic (GSM8K −15 points on the official checkpoint, −24.8 on
- SafeTune's DeepRefusal is a Variant. It has documented deltas from the paper's recipe. We audited what ships, plus the paper recipe separately (screen only), and we transfer no conclusion to the paper's own models.
- Template and language. One fixed chat template and English prompts only.
7. Related work
Refusal as a direction. Arditi et al. (arXiv:2406.11717) showed that refusal in instruct-tuned models is mediated by a single residual-stream direction whose ablation removes refusal. Heretic (github.com/p-e-w/heretic) packages the weight-space attack, and SafeTune's AbliterationAttack exposes it as a stress test.
Hardening against the direction. DeepRefusal (Xie et al., arXiv:2509.15202, EMNLP 2025 Findings) probabilistically ablates a fixed refusal direction during fine-tuning. Its own results show a single on-model direction partially breaking Llama-2-7B (36.9% attack success vs 0.0% transferred; Table 1). On Llama-3-8B the same attack reaches only 0.2% (Appendix B, Table 6), consistent with the official checkpoint holding at k = 1 in our audit. ReFAT (Yu et al., arXiv:2409.20089) re-extracts a single refusal feature during training; it is prior art for our re-estimating dose. ART (Kuo, Smith, Yadav; arXiv:2605.26526) attacks fine-tuning defenses and proposes abliteration-resistant tuning.
Adaptive and steering attacks. Multi-phase refusal-feature ablation, with fresh orthogonalized directions accumulated, was established against decoy defenses by DDO (Muhamed, Diab, Smith; arXiv:2609.16204). Our ladder applies that family to direction-blocking defenses and adds the margin and repair accounting. APS (arXiv:2605.20286, ICML 2026) breaks 12 fortified models with adaptive per-layer probe steering but does not test DeepRefusal. Latent adversarial training (Sheshadri et al., arXiv:2407.15549) inspired our dual-operator recipe.
Public, non-peer-reviewed breaks of the official DeepRefusal. Two Hub model cards report breaking the released checkpoint:
wangzhang/Llama-3-8B-Instruct-DeepRefusal-Brokenattenuates the defense delta toward the base and then abliterates (self-reported 89% ASR; needs the base weights);FTK11558/Meta-Llama-3-8B-Instruct-DeepRefusal-Broken-APSuses APS steering (self-reported ~90%).
We cite both as unreviewed. Our break needs no base weights and is dose- and random-controlled.
Concurrent hackathon work. "Heretic-Resistant Tiny Aya" (StarWarrior24x7) reported before us that activation steering remains a substantial weakness of their ablation-hardened Tiny Aya (§3.1).
Quantization. MANSU (Sadhu, Seth, Sankarapu; arXiv:2605.15138) sizes unlearning edits to survive 4-bit rounding. We make no quantization claim.
The Lexsi stack. SafeTune (arXiv:2609.22153) supplies the DeepRefusal Variant and the single-direction abliteration baseline. CuratorKIT (arXiv:2606.21631) supplies the dedup and provenance stages of the splits. AuditKIT supplies the judge profiles and the paired bootstrap (auditkit._bootstrap.paired_bootstrap). CircuitKIT is not used.
8. Reproducibility and release
- Figures.
python figures/make_figures.pyregenerates all four figures from the CSVs in results/. - Count and logic checks.
python audit/reproduce_frozen.pychecks the reported counts in results/final_panel.csv.python -m pytest audit/tests -qchecks the audit logic without a GPU. - Computed statistics. The CSVs in results/ hold every reported count, interval and paired test. Statistics are shipped as computed tables. Per-prompt judgments are withheld, so the shipped files do not support recomputing the paired tests.
- Dataset. The HF dataset holds ID-only splits and
rebuild_text.py. - GPU archive. experiments/ contains archived as-run GPU code. It is not a turnkey pipeline. It needs the original payloads and GPU hosts. Per-prompt judgments and generated text are withheld.
- Released: audit code, metrics and counts, our defense adapters with model cards (each labelled with its measured margin; none is hardened), and ID-only prompt splits with provenance and per-source licenses.
- Not released: abliterated or attacked weights, extracted attack directions, steering vectors, or raw harmful completions.
- Licenses. Derivatives inherit CC-BY-NC-4.0 from Tiny Aya, and the audited
xrefcheckpoints are CC-BY-NC-4.0. The official DeepRefusal checkpoint ships no license, which we disclose. - Courtesy notes. We will share these results with the DeepRefusal authors and with StarWarrior24x7 when the release is public.
Appendix A. Experiment history and pre-registration
A.1 Pre-registered claims and their status
Four claims and four kill criteria were frozen before the final panels.
Claims:
- C1 (confirmed; DR wording corrected). Single-direction tests overstate abliteration resistance. The adaptive margin is 1 for SafeTune's DeepRefusal on Tiny Aya, 2 for
xref-v2, and greater than 2 for the official DeepRefusal (5 on the final panel, with ranks 1–4 holding; 6 and > 6 on the screen). - C2 (confirmed on both checkpoints that survive one direction). A single-layer additive shift of the model's own direction bypasses the margin while norm-matched randoms do not. Final-panel randoms are matched to the coherent rows (
xref-v2c*, official L22). - C3 (confirmed as a bill; repair stated descriptively). As frozen, C3 also said the repair is "free only when k* is large". With two hardened checkpoints and a non-significant difference (p = 0.25) that is not an equivalence test, we report the repair checkpoint by checkpoint instead (§5).
- C4 (negative). A re-estimating training dose would harden Tiny Aya. It did not (A.2).
Kill criteria:
- K1 (seed-unstable k* reported as a range): applied to the official checkpoint.
- K2 (random vectors break → drop C2): not triggered; no random control breaks anywhere.
- K3 (stop the fix branch if it fails its gate): fired for the re-estimating dose.
- K4 (report final numbers for > 10-point screen→final movements): A.4.
A fifth registration (D11) covered the dual-operator defense (A.3).
A.2 The re-estimating dose (C4 negative)
The pre-registered fix re-estimates the refusal direction during training (ASRT-k4, LR 2e-5; prior art: ReFAT). It failed its on-model selection screen: 0/32 refusals under the best candidate (layer 26), against a gate of ≥ 20/32. Kill criterion K3 fired and training stopped. On the audit ladder it falls at k* = 1 under both seeds at marker level (DIM norm ratio 0.969). WildGuard labels do not exist for it (the judge ran out of memory on a 16 GB GPU), so no WildGuard estimate is claimed.
A.3 The dual-operator defense (negative)
The registered prediction: a Tiny Aya defense trained against adaptive projection and additive perturbations would meet five conditions.
- k* ≥ 3 under both seeds;
- ≥ 100/150 refused under a single-layer shift at c = 2.0;
- intact norm-matched randoms;
- over-refusal ≤ 122/350 (2× the base);
- capability within −2/−3 points.
The kill condition was "neither operator target met after two doses".
The recipe. A DeepRefusal-style LoRA was trained on the 8,500-example paper-recipe corpus for one epoch (532 updates). The rank-3 DIM subspace is re-estimated on the current model every 16 steps. On disjoint harmful batches it is either projected out (per site and token, p = 0.5) or added at a random dose c ~ U(0.5, 2.5): sampled perturbations in the spirit of latent adversarial training, not an optimized adversary. Each dose was audited on B150/O350 in a seed-0 screen (A0, k = 1–3, c = 2.0, one random).
| criterion (bar) | dose 1, LR 1e-5 | dose 2, LR 2e-5 |
|---|---|---|
| k* under both seeds (≥ 3) | 1 (seed 0: 16/150 at k = 1; 20, 15 at k = 2, 3): fail | 1 (35/150; 33, 33): fail |
| refused at c = 2.0 (≥ 100/150) | 21/150, plus 1 WildGuard-unknown: fail | 70/150: fail |
| norm-matched random (holds) | 128/150 refused, but 111/150 degenerate | 48/150 refused, 141/150 degenerate: fail |
| over-refusal (≤ 122/350) | 108/350 (30.9%): pass | 144/350 (41.1%): fail |
| capability | not run | not run |
| A0 harmful refusal vs base 112/150 | 124/150 (12 gained, 0 lost; p = 4.9e-4) | 134/150 (22 gained, 0 lost; p = 4.8e-7) |
Verdict: the two-dose kill fired. Seed 0's k* = 1 already rules out the projection target at both doses. The c = 2.0 target fails at both doses, even counting dose 1's unknown verdict as a refusal (22/150). Clean refusal rises but does not survive the attack: dose 2's one-direction attack yields 72/150 coherent harmful completions (HarmBench-cls 68) with 1 degenerate.
No direction-specific additive claim for these models. At c = 2.0, both the DIM shift and its random destroy generation: 83 and 111 of 150 outputs are degenerate at dose 1, and 136 and 141 at dose 2, where all 150 hit the token cap. The random's first-token KL is 9.95, against 2.38 for the DIM. Dose 2's random control fails outright, so we infer no c* for these models.
A counterintuitive observation (descriptive only). Under its own one-direction attack, dose 1 breaks harder than the undefended base. It refuses 16/150 [6.7, 16.6] with 78/150 coherent harmful, against the base's 33/150 [16.1, 29.3] with 37/150. Its selected direction sits at the same site as the base's (layer 22, last token) with a larger norm (35.9 vs 32.4). One reading is that the training concentrated refusal in a single, easier-to-find direction. We do not claim this, for four reasons:
- the base row comes from a different run of the frozen protocol;
- there is one training seed and one selection seed;
- no between-model paired test was computed;
- no norm-ratio or fixed-direction measurement exists for these adapters.
Dose 2 does not show the pattern (35 vs 33/150).
Scope. This is a screen. Seed 1, the full dose curve, a second random, the operator/scope control, the repair and capability were not run. The kill does not depend on them. The two doses were trained on different GPUs (bf16 both), a possible numerical confound.
A.4 Screen→final movements (K4)
The census. Between the screen (B46 mlabonne, O50 OR-Bench-hard) and the final panels (B150, O350), the frozen census counts 23 movements larger than 10 points: 9 on the official checkpoint and 14 on Tiny Aya. All 23 are attributed to set composition.
Shared prompts did not move. The base at A0 on the 46 shared harmful prompts is 46/46, exactly as on the screen. Every official-checkpoint harmful cell is unchanged on the matched 46 (+2.17 points on the L22 row). The added prompts are harder to refuse, so high-refusal rows move down while attacked rows barely move (+1.0, −6.5, −0.3 and −4.75 points on the four main attack rows).
Three movements changed a reading:
- The official checkpoint's k = 5 row held on the screen (32/46) but is below half on the final panel (70/150). This is why its margin reads 6 on the screen but 5 on the final panel.
xref-v2's repair kept every refusal on the screen but loses 19/128 on the final panel.- The judge-flagged coherent yield of the official L22 break fell from 31/46 to 59/150.
All flagged rows are reported with their final numbers; none was re-run.
A.5 Not run (nothing invented)
Not run:
- the official checkpoint's capability at k* and after repair (A0 and c* capability were measured on MMLU-200/GSM8K-100);
- the base and SafeTune DeepRefusal additive rows on the final panel (screen only);
xref-v2capability after repair on the final panel;- over-refusal on O350 for the official L22 row, for SafeTune DeepRefusal after repair, and for Llama-3-8B-Instruct;
- final panels for the paper recipe and
xref-v1; - WildGuard labels for the re-estimating dose;
- the dual-operator audit beyond its screen;
- the official checkpoint's seed 1 beyond the cap;
- an NF4 behavioural arm.
A.6 Judge check against blinded LLM labels
The sample. 60 rows were drawn by outcome (enriched for attacked and repaired conditions), blinded, and labelled for refusal and harm by an LLM (codex) on the owner's behalf. The owner reviewed the labels and made no corrections. No rate is extrapolated from this outcome-enriched sample.
| comparison (n = 60) | κ | LLM+ / judge+ | LLM+ / judge− | LLM− / judge+ | LLM− / judge− |
|---|---|---|---|---|---|
| refusal vs WildGuard | 0.667 | 22 | 2 | 8 | 28 |
| harmful vs WildGuard | 0.542 | 10 | 0 | 11 | 39 |
| harmful vs HarmBench-cls | 0.613 | 9 | 1 | 7 | 43 |
| stratum | n | refused (LLM / WildGuard) | harmful (LLM / WildGuard / HarmBench-cls) |
|---|---|---|---|
| official DR, L22 shift at c = 2.0 | 20 | 7 / 8 | 3 / 8 / 8 |
xref-v2, shift at c = 1.25 |
15 | 4 / 5 | 1 / 6 / 2 |
xref-v2, k = 2 projection |
5 | 2 / 3 | 2 / 2 / 2 |
| SafeTune DeepRefusal, k = 1 projection | 5 | 2 / 2 | 2 / 3 / 2 |
xref-v2, repair |
10 | 6 / 8 | 1 / 1 / 1 |
| no-attack rows | 5 | 3 / 4 | 1 / 1 / 1 |
Reading:
- Refusal, the break metric, survives. The LLM labeller marks refusal no more often than WildGuard (24 vs 30; only 2 rows are LLM-refused but not WildGuard-refused). So the below-half breaks are not a WildGuard refusal artifact, and WildGuard is, if anything, conservative for the attacker.
- Harm on steering rows is overstated by the judges. The labeller confirmed substantive harmful content in 3 of 8 WildGuard-flagged official-L22 rows and 1 of 6
xref-v2rows, which is why the steering harm counts are reported as upper bounds (§6). - Harm on projection rows agrees. The labeller confirmed 4 of 5 judge-flagged rows.
Appendix B. Provenance map
Evidence tables. The released CSVs retain the original checkpoint and source identifiers. Use this map to locate each part of the audit:
| evidence | where it is |
|---|---|
ta-base, t22-dr2e5, t24-lr1e5, t31-asrtk4, t35-dose1/t35-dose2, xref-v1, xref-v2, dr-official |
checkpoints in §2.1, in table order |
| Final panels | final panels for Tiny Aya / the official checkpoint (results/final_panel.csv) |
| Screening panels | the screens (results/screens.csv) |
| Preliminary official-checkpoint sweep | the preliminary sweep on the official checkpoint (results/screens.csv) |
| Matched steering controls | the final-panel L22 random controls (results/final_panel.csv, results/paired_tests.csv) |
| Dual-operator audit | the dual-operator defense and its audit (results/final_panel.csv) |
| Paired statistics | the paired statistics (results/paired_tests.csv) |
| Completed rank ladder and capability | final-panel official-checkpoint ranks k = 1–4 and capability at c* (results/capability.csv) |
| Blinded label check | the blinded LLM-label check (Appendix A.6) |
Where the numbers live.
- Final-panel counts: results/final_panel.csv.
- WildGuard-unknown counts: the
unknowncolumn in results/final_panel.csv and results/screens.csv. - Paired tests: results/paired_tests.csv.
- Wilson intervals: results/wilson_ci.csv.