Prefill Jailbreak Bypasses LLM Refusal in Early Response; Harm Representation Unchanged

Jul 17, 2026

A new preprint demonstrates that inserting a simple prefill phrase (such as "Sure, here is") at the start of a prompt can bypass refusal mechanisms in aligned large language models (LLMs). The study finds that while the model's internal representation of harm remains high, behavioral refusal drops to chance, and this effect is localized to the first half of the response. The underlying mechanism is shown to be generic autoregressive conditioning rather than a safety-specific suppression, and the vulnerability is consistent across multiple model families and sizes.

Why it matters: This work exposes a structural vulnerability in current LLM safety alignment, highlighting the challenge of defending against response-site attacks that exploit generic model mechanisms rather than safety-specific features.

Full story at: arXiv Computation and Language ↗