The brittleness is specific and mechanical: a model trained to refuse harmful requests in a single turn will still cave to the same request if an attacker spreads it across a conversation, and a new paper called ShallowDefend, from researchers at UC Davis and Microsoft Research, argues this isn't a training-data gap but a structural one. According to the paper, posted to arXiv, "alignment is shallow, with safety behavior concentrated in a model's first few output tokens, and existing defenses inherit the weakness." Once a response opens compliantly, the paper states, the aligned distribution collapses back toward the base model's — which is exactly why prefilling an affirmative prefix defeats refusal, and why a multi-turn attacker only has to steer the conversation until the locally natural opener is compliant.
ShallowDefend's fix is to make the model commit to a decision before it starts answering, rather than letting that decision leak implicitly into a refusal prefix. The framework, called Assess → Decide → Respond, has the defender generate three explicit parts at every turn: an assessment of the user's underlying intent, a parseable decision line reading COMPLY, REFUSE, or (in one variant) SAFE-REDIRECT, and only then a reply conditioned on that decision. Because the decision is an externalized token span rather than something buried in the tone of a reply, the paper notes, "it can therefore be read by a rule rather than judged by a model" — and because it precedes the reply within the same generation, it's binding: changing the decision changes the response.
That externalization is what lets the reward function skip the judge model entirely. The paper describes a self-verified reward that simply string-matches the model's committed label against the known ground truth of the training conversation — refuse on dialogues grown from harmful seeds, comply on benign ones — with no classifier, no preference labeler, and no annotation pipeline in the loop. A separate external-reward variant still uses an LLM-as-judge (the paper used the HarmBench-13B classifier during training, and gpt-5-mini for evaluation) over a three-way decision space that adds safe-redirect. Training uses GRPO, Shao et al.'s reinforcement learning algorithm, applying the reward only at the final turn of each conversation — the "payload turn" — rather than spreading credit across every turn, which the authors found otherwise lets the policy pad early, easy turns until the token budget runs out on the hard ones.
The evaluation spans two model families and fourteen benchmarks, as the paper states, covering safety, utility, and helpfulness, and the defenders are built on Llama3.1-8B-Instruct and Qwen3-8B. On Llama3.1-8B, the self-verified "aligned" defender hits a 90.4 average SAFE% across five safety suites (AdvBench, HarmBench, MultiBreak, CoSafe, and MHJ), against a 59.9 average for the undefended base model and 86.3 for the strongest baseline, X-Boundary. On Qwen3-8B the self-verified defender reaches 93.4 average SAFE%, versus 46.1 for the base model. Utility — averaged across MMLU, HellaSwag, Winogrande, GSM8K, TruthfulQA, and IFEval — stays close to baseline on both backbones, which the paper attributes to those benchmarks being scored without the chat template and therefore largely insensitive to safety training.
The cost shows up in helpfulness, not capability. On Llama3.1-8B, over-refusal on XSTest climbs to 20.4 for the self-verified aligned model versus 10.0 for the base, and on XSB it climbs to 54.4 versus 30.0 — a stricter decision policy that the paper flags directly as "a modest cost to helpfulness." MT-Bench scores dip too, from 6.20 for the Llama base model to 5.86 for the self-verified aligned version. The paper's ablations found that removing the decision line entirely — leaving just assessment and response — was the single most damaging change tested: safety on the suite fell below the undefended base model under human red-teaming, even with the assessment, reward, and data all left intact, which the authors use to argue that the committed decision, not the deliberation around it, is what carries the defense.
The threat model is also explicitly bounded: the paper states its attacks are "scripted multi-turn attacks replayed statically," and lists adaptive adversaries as an open limitation it does not address. It also flags that the judges used to score safety disagree with each other on a substantial share of individual samples — a dispersion the authors say "cautions against reading any single-judge attack-success rate as ground truth," which is as much an admission about the reliability of safety benchmarking generally as it is about this method specifically.
For a team hardening a chat agent against distributed-attack jailbreaks, the actionable piece here isn't the specific SAFE% numbers, which are backbone- and benchmark-dependent — it's the recipe: externalize a parseable safety decision as a token span the model must commit to before it replies, and reward only that span against a known label at the final turn, rather than trying to shape the tone of the whole response with a judge model in the loop.