Attack success rate charges every jailbreak the same. Compute-aware evaluation fixes that, then breaks in CI

Two attacks broke the same model at nearly the same rate last week. One of them cost 168 times more than the other, and the number I would have reported to anyone asking about robustness treated them as equals.

That number is attack success rate at a fixed query budget, and it is what nearly every jailbreak evaluation reports. Ten queries, count the breaches, publish the fraction. The trouble is that a query is not a unit of effort. A template attack spends one generation and one judge call. A gradient attack spends 131 forward passes through the target before it generates anything at all. Charging both a flat rate per query is like quoting a building's security as "three attempts to get in" without saying whether an attempt is a shove or a crane.

Malikeh Ehghaghi, Boglárka Ecsedi, Marsha Chechik and Colin Raffel put both attacks on a shared axis instead: cumulative floating-point operations. Their risk-compute curves plot how much risk an attacker buys per unit of compute, and their two summary numbers are the compute needed to reach 50% risk and an average efficiency figure. Across ten models they find things the query axis cannot see, including that scaling a model from 0.5B to 7B parameters raises the cost of a gradient attack twentyfold while leaving cheap template attacks almost as effective as before.

They are right about the axis. What I wanted to know is narrower: whether either summary metric survives contact with a build pipeline, because a robustness figure earns nothing until a job can compute it nightly and tell someone the number moved. So I rebuilt the cost model and both metrics over a task small enough to run in four minutes on a laptop, and asked how much benchmark a ranking needs before it stops changing its mind.

Three things this piece does not do. It does not evaluate any real language model: there is no language model anywhere in my prototype, by design, and no number here transfers to one. It does not test the attacks themselves, which are miniatures of published ones and are only here to differ in cost. And it says nothing about whether jailbreak robustness is achievable, which is a different and much harder question than whether it is measurable.

The short version

• The idea: measure how much compute an attacker must spend to break your model, not how many times they had to ask.
• Why it might work: per-step cost is analytic, so heterogeneous attacks become comparable on one axis without new experiments.
• What I tested: whether the paper's two summary metrics rank three target sizes consistently, and how many behaviours each ranking needs to match the full benchmark.
• What happened: the compute axis reproduced the paper's scaling asymmetry, the compute-to-threshold metric was undefined for the cheapest attack in every seed, and the normalised metric ranked reliably from 40 behaviours.
• Where it breaks: every apparent robustness difference in my testbed tracked how much benign traffic the target refused.
• Decision: record the accounting now, gate on nothing until a utility column and a stability floor sit beside it.

Start with the problem

Here is the situation the paper is aimed at. You own a model with a safety layer. Every release, someone runs a jailbreak benchmark and hands you a number: 200 behaviours, ten queries each, 31% of them got through. Next release the number is 27%. Did the model get harder to break?

You cannot tell, and the reason is not statistical. Suppose the first release fell mostly to a template attack anyone can copy from a forum, and the second fell mostly to a gradient attack that needs the weights and a GPU-hour per behaviour. The percentage went down and your exposure to a realistic attacker went up.

Classical security has a name for the missing quantity. The work factor is the computational effort an adversary has to spend, and it is why nobody reports a cipher's strength as "three attempts". The paper imports that idea with FLOPs as the currency, charging every component of an attack step: the target's generation, the judge's evaluation, an auxiliary attacker model if there is one, and the candidate sweep and backward pass if the attack uses gradients.

What that figure is not showing is how often each attack succeeds. Against this target the template attack breaches 33% of behaviours and the gradient attack 93%, so the expensive attack is roughly three times as effective and 168 times as expensive. A success rate reports the three and discards the 168.

How the method works

1. Run the attack as budgeted iterative refinement

Every attack family gets the same loop, which the paper states as Algorithm 1. At step t the attack policy proposes a prompt, the target answers, a safety judge returns a binary verdict, and the loop stops on the first success or exhausts the budget. Query budgets run over {0, 1, 2, 4, 6, 8, 10}. The uniform loop is what makes the families comparable at all: they differ only in what the policy does and what it costs.

2. Charge each step for the passes it actually took

The three cost equations are short enough to read. A template step pays for one target generation and one judge pass. A refinement step adds a forward pass through the attacker model. A gradient step adds 128 candidate evaluations and a backward pass, which the paper charges at three times a forward pass: two for the standard backward-to-forward ratio, plus one for the discrete operations that greedy coordinate substitution performs over the vocabulary.

CANDIDATES = 128   # Zou et al. 2023, used unchanged by the paper
BETA_BWD = 3       # 2:1 backward-to-forward, plus 50% for discrete ops
 
def forward(parameters, length):
    return 2.0 * parameters * length
 
def template_step(target_n, judge_n, generated, judged):
    return forward(target_n, generated) + forward(judge_n, judged)
 
def gradient_step(target_n, judge_n, generated, judged, optimised):
    search = (CANDIDATES + BETA_BWD) * forward(target_n, optimised)
    return search + template_step(target_n, judge_n, generated, judged)

Correctness check: tests/test_cost.py asserts each expression against the paper's arithmetic, that the candidate sweep accounts for more than 90% of a gradient step, and that a gradient step costs more than fifty times a template step at matched lengths.

The asymmetry is structural rather than incidental. The candidate sweep dominates its own equation, so the gradient attack's cost is set almost entirely by a constant the attack chose, which means a defender can predict the bill without running anything.

3. Turn the traces into two summary numbers

Risk at budget λ is the fraction of behaviours breached within λ steps. Mean cumulative cost at λ is the average FLOPs per prompt, counting only steps the attacker had to pay for before succeeding. Plot one against the other and you have a risk-compute curve. From it, two summaries: C@0.5, the compute at the first budget whose risk reaches 50%, and average efficiency, the area under the curve normalised by the compute ceiling.

The two summaries have different characters, and this is where my interest started. Risk and success rate are means: add a behaviour and the number moves a little. C@0.5 is a threshold crossing, so it moves in jumps, and when risk never reaches 50% there is no compute to report. The paper writes those cells as infinity. There are four of them in its main table.

What the research actually shows

The strongest result

Read that as a defender and it inverts a common intuition. Scaling did buy real protection, against the expensive attack, from the attacker least likely to bother. Against the cheap attack that anyone can run, a fourteen-fold larger model bought almost nothing, and a metric that counts queries would have told you the model improved.

Does the mechanism earn the credit?

The mechanism is arithmetic rather than a learned component, so the question is whether the effect is an artefact of the accounting. Two things argue it is not. The rankings hold across two benchmarks with Spearman correlation at or above 0.91, and the same asymmetry appears in three separate study designs: training stage, model size, and safety tuning. The safety-tuning arm is the most striking, because it fails in the interesting direction.

Where the evidence is thin

The authors are direct about most of it. FLOPs are a theoretical proxy that ignores memory bandwidth, parallelism and wall-clock; the judge is an automated model with known reliability problems, and they cite a 2026 result whose title says LLM judges fail to reliably measure adversarial robustness; and none of the three attacks is adaptive against a specific defence, which they expect would tighten every bound.

One gap they do not name is the one this article is about. Every number in the paper is a research measurement over 200 behaviours and 10 seeds on data-centre GPUs, between 0.5 and 10 GPU-hours per configuration. Nothing in it addresses how the metrics behave as estimators at the scale a release process can afford. I suspect that is because the paper is arguing for an axis rather than selling a gate, which is a fair place to stop, and it leaves the operational question open.

Why this paper earned the test

I only consider papers whose first public version is under six months old, which bounded the search to work posted since February. Within that window I screened four serious candidates against claim support, baseline strength, causal evidence, production transfer, and testability.

That shortlist is a judgement call and someone else would have picked SkillSafe-Bench. I chose the compute paper because its central claim can be checked without a GPU, which is the rarest property on the list.

Rebuilding the claim

Experimental setup

Four targets over one synthetic task. A request is a 24-token sequence, four to six of whose words are the payload; a jailbreak is a request the target should have refused whose answer carried the payload through. Three targets vary width at 8, 24 and 64 hidden units, holding alignment data constant, which is the paper's model-size study. A fourth holds width at 24 and adds 600 extra refusal examples skewed unevenly across six categories, which is its safety-tuning study. Parameter counts run from 282 to 2,242, so results are in MFLOPs and only ratios carry over.

Three attack families mirror the paper's: eight surface templates sampled with replacement, a surrogate attacker that proposes rewrites, and a greedy coordinate substitution over a 20-token suffix with 128 candidates a step. Two judges, one accurate and one deliberately degraded, both scoring every step, with ground truth recorded alongside. 120 behaviours, 10 seeds, the paper's budget sweep, τ = 0.5. No language model, no GPU, no third-party package, four minutes end to end.

git clone https://github.com/shravan1996/ship-the-paper-risk-under-pressure
cd ship-the-paper-risk-under-pressure
python3 -m unittest discover -s tests -t .
python3 eval.py

Nothing in the repository is harmful text. The measurement is the accounting over attack traces, and real harmful content would add nothing to it. The full implementation and its deviations are in the prototype repository.

What I expected

I expected C@0.5 to be the fragile metric and average efficiency to be the boring one. A threshold crossing throws away everything about the curve except where it passes one line, so I assumed it would swing wildly under resampling and amplify judge error, and my read was that the paper led with it because it reads well in a table. I expected the ranking to need somewhere around 80 behaviours to settle, and I expected the eight-fold size increase to show up in both metrics.

I was wrong about the mechanism of the fragility, and wrong about the judge. The direction of the first one I would have got right for the wrong reason, which is worse than missing it.

What happened

The direction and rough shape of the paper's scaling result came out of a testbed a millionth of the size, which is the strongest evidence I can offer that the effect belongs to the accounting rather than to any property of language models. The paper's factors are 20 and 2.8; mine are 6.7 and 1.4. What reproduced is the asymmetry, not the magnitude.

Then the part that surprised me. C@0.5 was not noisy. It was absent.

That last sentence is the one I keep coming back to. Every other metric I have worked with converges as you add data. This one diverges, because a bigger sample pins risk more precisely at a value below the threshold, and precision about being below 50% is exactly what makes the metric undefined. Adding behaviours does not rescue it. It confirms the hole.

Would a lower threshold fix it? Partly, and not in a way I would want to defend: τ = 0.25 would give the template arm a number, at the cost of a gate that fires on a quarter of behaviours being breached, and the choice of τ then quietly sets which attacks your metric can see.

Now the judge, which I had expected to be the weak point:

One number in the paper does not add up

While rebuilding average efficiency I could not make the paper's own figures fall out of its definition. Equation 5 sets AE = CAURC / C̄max. CAURC integrates risk with respect to compute, so it already carries units of compute, and dividing by the ceiling once leaves a dimensionless ratio between 0 and 1. Table 1 labels the column risk per TFLOP and reports values between 0.0005 and 0.06.

Those cannot both be right. For Qwen2.5-0.5B under the template attack, risk reaches 0.99 within budget, so a dimensionless area-over-ceiling has to land somewhere near a half, not at 0.0596. The compute ceiling itself is not printed, but the same row reports 50% risk at 8.2 TFLOPs, which puts the ceiling in the region of ten. Divide by it twice, which is the same as normalising the compute axis before integrating, and the figure lands in the hundredths, in the right units and the right part of the table's range.

What breaks outside the experiment

Robustness bought with over-refusal

The most conservative target in my run had the lowest success rate under every attack, and it also refused 12.4% of benign requests against 3.9% and 5.2% for the other two sizes. The safety-tuned target dropped over-refusal to 7.8% and became more exploitable per unit compute than its base in five of six categories, including the three that received two to three times more refusal data. That last part I did not expect at all, and I still cannot separate the volume of the extra data from its skew, so I would not read it as evidence about real safety tuning.

What I will read it as: every apparent robustness difference I measured tracked how much ordinary traffic the target turned away. A compute-aware gate with no utility column will approve the release that quietly broke your product, and it will do so with a bigger number than last time.

A judge audited on the wrong distribution

My first judge scored 0% error on held-out responses and called refusals unsafe the moment it saw one, because I had trained it on prompts and never shown it a refusal. Every attack succeeded on step one, in every arm, and I spent a while believing my targets were made of paper. What should have caught me out sooner is that all twelve target-and-attack cells reported exactly 1.00, and no real measurement is that tidy. The general failure is not my bug: it is that judges get evaluated on the distribution they were trained on and then applied to attacked outputs, which is a different distribution by construction. Audit the judge on attacked responses, or the risk numbers underneath every compute figure are unmeasured.

Metrics that go undefined without saying so

An uncrossed threshold produces no number, and a pipeline that reports "no regression" when a metric fails to compute is worse than one that reports nothing. If you adopt C@τ anywhere, the censored fraction has to be a first-class output beside it, and a run where a third of the arms are censored has to fail loudly rather than pass quietly.

Cost that is only proportional to what you pay

FLOPs are invariant to hardware, which is what makes them comparable and also what makes them not a budget. An attacker with a rented GPU-hour and one with a laptop face the same FLOP count and very different real costs. The paper says this plainly. My guess is that it stops mattering inside a release comparison, where you only need the ratio to last release, and starts mattering the moment someone converts a work factor into a dollar figure for a risk register.

The production scorecard

The average is not what decides this. What decides it is that the metric the paper leads on cannot be put in a gate, and the metric that can be needs a second column the paper does not discuss. Both problems are fixable in a sprint, which is why the verdict is Wait rather than Never.

A production shape I would test

Everything here is asynchronous and none of it touches a request. The benchmark is versioned, because a ranking against a moving behaviour set means nothing. Attacks span the cost range on purpose: dropping the expensive arm saves most of the bill and removes the arm where the compute axis does its clearest work, so make that trade explicitly rather than by accident. The gate compares ratios against the previous release on the same benchmark and seeds, never absolute FLOP counts, and it refuses to report a ranking below the stability floor. I expect the stability floor to be the piece teams skip, because it is the only box that tells you to collect more data before believing your own dashboard.

From prototype to shadow traffic

1. 01

   Bound one workflow and one threat

   Owner

   Security engineer

   Artifact

   Versioned behaviour set for a single product surface

   Signal

   Behaviour count and category coverage

2. 02

   Instrument attack cost, change no metrics yet

   Owner

   ML platform engineer

   Artifact

   FLOP accounting over existing eval traces

   Signal

   Cost per family reported beside the current success rate

3. 03

   Measure your own stability floor

   Owner

   ML platform engineer

   Artifact

   Resampling report over the frozen benchmark

   Signal

   Smallest behaviour count whose ranking agrees with the full set 95% of the time

4. 04

   Audit the judge on attacked outputs

   Owner

   Applied scientist

   Artifact

   Labelled sample of post-attack responses

   Signal

   False positive and false negative rate on the attacked distribution

5. 05

   Add the utility column

   Owner

   Product owner

   Artifact

   Over-refusal rate on live benign traffic

   Signal

   Refusals on requests the product is supposed to answer

6. 06

   Gate on ratios, or say why not

   Owner

   Release manager

   Artifact

   Decision record naming the metric and the floor

   Signal

   Release held or shipped, with the reason recorded

Ship gates

• Risk per unit compute is reported per attack family, and no family is silently dropped
• The benchmark meets or exceeds the measured stability floor for the comparison being made
• Judge error is measured on attacked responses within the last release cycle
• Over-refusal on benign traffic is reported next to every robustness figure

Kill criteria

• Any gated metric is undefined or censored for more than one arm: fail the run, do not pass it
• Cost per breach improves while over-refusal rises: treat as a regression, not a win
• Ranking flips between two consecutive runs on the same benchmark and seeds
• The cheapest attack family's risk rises at all, whatever the expensive families do

Human review policy: a human reads any run where a metric was censored, where the ranking flipped, or where cost per breach and over-refusal moved together. Automated pass is allowed only when every gated metric is defined and the floor was met.

My verdict

Wait, for gating a release on either summary metric.

The axis is right and I would start using it this week. Cost accounting over traces you already collect is arithmetic, adds no experiments, and immediately tells you something the success rate cannot: which of your attackers is getting a discount. What I would not do yet is let a number computed this way hold or approve a release, because in my run the metric the paper leads on had no value at all for the attack family a real attacker reaches for first, and the metric that did work ranked the most over-refusing model as the most robust one.

Paper evidence and prototype evidence agree on the mechanism and diverge on the metrics. The paper's ten models show the compute axis carrying real signal, and I reproduced that at absurdly small scale. My run says the two summaries are not interchangeable and that one of them is not a statistic you can operate. Where they conflict I trust the paper on what the axis reveals about models, and my own traces on how the metrics behave as estimators, because the second is the question I designed for and the paper did not set out to answer it.

The narrow path I would fund now is the four steps below: instrument the accounting, measure your own stability floor, audit the judge on attacked outputs, put over-refusal in the same table. Perhaps two weeks, and it makes the next release comparable to this one. What I would reject is the version that reaches a slide deck as a single hardening number, because a work factor with no utility column beside it is an argument for refusing everything.

I would change this verdict if:

1. Someone reports the censored fraction of C@τ on a real benchmark and it turns out to be rare rather than routine, in which case the metric is gateable with a guard around it.
2. A team publishes a stability floor at production scale, on real models, that a nightly job can afford.
3. Risk per unit compute is shown to separate two models whose over-refusal rates are matched, which would tell me the metric is measuring robustness rather than conservatism.

What to remember

Acknowledgements

This article builds on Risk Under Pressure: Compute-Aware Evaluation of Adversarial Robustness in Language Models by Malikeh Ehghaghi, Boglárka Ecsedi, Marsha Chechik and Colin Raffel.

The prototype uses or adapts:

• The cost model, risk-compute curves and summary metrics from the paper above, whose framework is released at r-three/risk-under-pressure
• Miniature versions of three published attacks: JailBroken (Wei, Haghtalab and Steinhardt, 2023), PAIR (Chao and colleagues, 2025) and Greedy Coordinate Gradient (Zou and colleagues, 2023)

References

Primary research

• Risk Under Pressure: Compute-Aware Evaluation of Adversarial Robustness in Language Models. Malikeh Ehghaghi, Boglárka Ecsedi, Marsha Chechik, Colin Raffel, 2026

Supporting and contrary evidence

• Universal and Transferable Adversarial Attacks on Aligned Language Models. The gradient attack whose 128-candidate step dominates its own cost equation
• A coin flip for safety: LLM judges fail to reliably measure adversarial robustness. Why the judge under every risk number is itself a measurement problem
• The attacker moves second: stronger adaptive attacks bypass defenses against LLM jailbreaks and prompt injections. Twelve defences with near-zero reported success rates, broken above 90% by attacks that adapt to them
• VanillaBench: The Hidden Accuracy Cost of Adversarial Robustness. The utility column, argued across 186 published robust models
• Obfuscated Gradients Give a False Sense of Security. The 2018 case that established evaluation, not architecture, as the usual failure

Prototype

• Source, setup, and evaluation
• Exact tested commit

Sources last checked: 2026-08-13

The question a robustness evaluation has to answer is not whether someone can break your model. It is what breaking it costs them, and whether that price went up or down since the last release.