A new preprint introduces a training-free method to enhance the safety alignment of large language models (LLMs) during downstream adaptation. By using knowledge distillation and model fusion, the approach corrects 'shadow alignment' issues without requiring supervised fine-tuning or reinforcement learning. Experiments show an average 14.42% improvement in defense success rate across 17 LLMs, with no loss in model performance.
Why it matters: This plug-and-play technique addresses a key challenge in safely adapting LLMs for new tasks, offering a practical solution without the need for costly retraining.
Researchers introduced SLYP, a REACT-style pipeline leveraging LLM agents for end-to-end vulnerability discovery in commercial off-the-shelf (COTS) binaries. SLYP identified all 64 vulnerable entry functions in a 20-object COM benchmark and generated debugger-verified proof-of-concept crashes for 67.5% of cases, outperforming default production agents. To date, SLYP has uncovered 39 zero-day vulnerabilities, with 23 assigned CVEs and $203,000 in bounty awards.
Why it matters: This work demonstrates a significant advance in automated vulnerability discovery, showing that LLM agents can effectively analyze and reason about vulnerabilities in stripped, optimized machine code where source code is unavailable.
Policy & Safety→Official→arXiv Cryptography and Security
A new preprint demonstrates that AI coding agents can be compromised by malicious instructions embedded in project setup documentation, such as README files. The study systematically evaluates supply-chain attacks delivered through documentation across multiple ecosystems (npm, Cargo) and agent harness-model combinations, showing that agents often install untrusted or vulnerable dependencies. Security-oriented prompts only partially mitigate these risks, and the effectiveness of defenses varies by harness and model.
Why it matters: This exposes a critical security vulnerability in AI coding agents, where project documentation can be weaponized as an attack vector, threatening software supply chain integrity.
A new framework, PA-HDP, is proposed to address privacy risks in retrieval-augmented generation (RAG) systems by recognizing that privacy leakage is dynamic and depends on the user's query. PA-HDP uses a prompt-aware risk hierarchy and adaptive protection mechanisms to assess and mitigate privacy risks on a per-query basis. Experimental results show that PA-HDP reduces privacy leakage and maintains retrieval quality better than previous static, document-level approaches.
Why it matters: This work introduces a more nuanced and effective approach to privacy in RAG systems by adapting protection to the actual sensitivity of content in response to specific user queries.
Policy & Safety→Official→arXiv Cryptography and Security
A new preprint investigates prompt injection vulnerabilities in agentic systems that use persistent memory, focusing on Anthropic Claude Code and OpenAI Codex across four models. The study finds that while it is challenging for attackers to overwrite memory files directly, malicious payloads already present in memory can persist and compromise both current and future sessions. The persistence and effectiveness of these attacks vary depending on the system, model, and adversarial objectives.
Why it matters: This work demonstrates that persistent memory in agentic systems introduces a novel and significant attack vector for prompt injection, underscoring the need for new defenses that secure memory updates without impeding beneficial adaptation.
FlowGuard is a new security detection system for the Model Context Protocol (MCP) that combines semantic analysis with runtime evidence to assess risks in LLM agent tool interactions. In evaluations on an executable benchmark of 1,880 MCP cases, FlowGuard achieved F1 scores of 0.879 and 0.942 for Command Injection and File System Access vulnerabilities, respectively. It also reduced end-to-end latency by up to 2.23x compared to existing dynamic scanners and reported 523 findings across 326 real-world servers.
Why it matters: FlowGuard addresses a critical gap in MCP security by reliably detecting both execution-based and semantic risks in LLM agent tool interactions, improving upon the limitations of current scanners.
Researchers have developed a fully automated framework that uses large language models (LLMs) to generate, execute, and revise adversary emulation playbooks directly from MITRE ATT&CK-aligned cyber threat intelligence (CTI) reports. The system unifies playbook generation, execution, and failure recovery, eliminating the need for manual intervention present in previous approaches. In evaluations on 11 CTI reports using four leading LLMs, the framework achieved its best performance with Claude Sonnet 4.5, reaching 84.22% execution success after revision and a CTI F1 score of 60.50%. The failure recovery mechanism improved execution success rates by 14.59 to 17.23 percentage points across all tested models.
Why it matters: This work represents a significant advance in automated cybersecurity testing, enabling more scalable and responsive adversary emulation by minimizing manual effort.
DataShield is a framework designed to identify risky fine-tuning data for large language models (LLMs) by measuring alignment with consensus safe and unsafe subspaces derived from multiple safety-aligned models. The method enables both sample-level filtering and segment-level masking, reducing attack success rates by 14.6% and 32.3% respectively, while preserving downstream utility.
Why it matters: This work offers a transferable approach to mitigating safety degradation during LLM fine-tuning by filtering risky data without requiring target-model-specific computation.
A new preprint proposes that evaluations of security agents should incorporate economic efficiency, not just task success, by comparing models at fixed cost levels. The study finds that offensive CTF agent performance improves with increased compute resources, while defensive SOC agent success relies more on disciplined tool use and telemetry navigation than on raw reasoning budget. The authors introduce a cost-aware evaluation framework and provide an interactive website with their results.
Why it matters: This work introduces a cost-aware evaluation framework that could reshape how security agents are benchmarked for practical operational use.
Policy & Safety→Official→arXiv Cryptography and Security
A large-scale study of 6,051 Android apps reveals significant inconsistencies between privacy policies and Google Play Data Safety labels, particularly for sensitive data types such as personal information and device identifiers. The research finds that sharing disclosures are less consistent than collection disclosures, and that apps involving persistent monitoring and communication present higher privacy risks. The study introduces a sensitivity-weighted risk score to better assess these misalignments.
Why it matters: This work exposes structural weaknesses in current privacy disclosure mechanisms, highlighting the need for improved verification and transparency in app data practices.
Policy & Safety→Official→arXiv Cryptography and Security
A new preprint introduces MemPoison, a benchmark and analysis framework for evaluating persistent memory attacks on LLM agents. The study covers 1,227 hand-validated attack cases across four attack types and three injection channels, revealing that while write-time defenses can suppress direct attacks, they are ineffective against more complex compositional and context-triggered attacks. The authors propose a taxonomy of attack types and demonstrate structural blind spots in current defense mechanisms, advocating for adaptive, context-sensitive memory defense strategies.
Why it matters: This work exposes critical and previously underappreciated security vulnerabilities in LLM agents with persistent memory, indicating that current defenses are inadequate against sophisticated multi-step attacks.
Policy & Safety→Official→arXiv Cryptography and Security
Researchers demonstrate that large language models (LLMs) used for network security log analysis are vulnerable to passive prompt injection, where adversaries embed malicious payloads in log fields that execute when queried. Their LogInject framework achieves up to 88.2% attack success rate across models, and a novel 'Context Stitching' technique evades stateless filters with 76.4% success. Layered defenses reduce attacks by 90.4%, but 8.4% residual vulnerability remains, highlighting the need for defense-in-depth and human oversight.
Why it matters: This research exposes a critical security flaw in LLM-based security operations, showing that untrusted log data can hijack model outputs and undermine threat detection.
Researchers have developed a methodology to automatically extract instruction-centric leakage contracts for major CPU architectures, implemented in the tool malcos. The approach is evaluated on x86 and ARM CPUs, demonstrating that the synthesized contracts are precise and sound with respect to observed microarchitectural leaks.
Why it matters: Automating the creation of leakage contracts can significantly reduce manual effort in securing CPUs against microarchitectural side-channel attacks.
A new preprint demonstrates that combining real and text-to-image (T2I) synthetic data for model training can increase privacy leakage of real samples, contrary to common assumptions. The authors introduce a theoretical framework explaining how synthetic data can force models to memorize real data more, and present RSMixLeak, a method to assess this risk using membership inference attacks. They also propose a lightweight indicator to help identify datasets at high risk for privacy leakage when used in mixed training.
Why it matters: This work reveals that synthetic data augmentation can unintentionally worsen privacy risks for real data, challenging prevailing practices in privacy-preserving AI training.
Policy & Safety→Official→arXiv Cryptography and Security
A new preprint introduces an Adversarial Prompting Framework (APF) designed to systematically assess the safety of AI models against adversarial prompt attacks. The framework generates structured prompts at varying levels of sophistication, from straightforward harmful requests to advanced encoding-based attacks, and enables automated testing with quantitative security metrics. The study finds notable differences in model vulnerabilities, with encoded prompts most frequently bypassing safety mechanisms.
Why it matters: This framework provides a practical, automated approach for identifying and quantifying critical vulnerabilities in AI models, which is essential for improving AI safety.
Policy & Safety→Official→arXiv Cryptography and Security
A new arXiv preprint from GDM introduces the AI Control Roadmap v0.1, outlining a structured approach to internal security for potentially misaligned AI agents. The roadmap presents a threat taxonomy (TRAIT&R), capability-based mitigation tiers (D1-D4, R1-R3), and 15 specific defensive measures, including chain-of-thought monitoring and shutdown infrastructure. The framework is designed to escalate defenses as AI capabilities increase, aiming to address emerging security challenges in AI deployment.
Why it matters: This roadmap offers a systematic, tiered framework for mitigating risks from misaligned AI agents, addressing a key safety concern as AI systems become more capable and integrated into critical operations.
Researchers introduce a trust model for large language models (LLMs) that shifts authority to external code, using source integrity to determine which operations are executed. Their deterministic pipeline ranks inputs by trust level, and a non-model monitor ensures that only trusted inputs can trigger actions. In experiments with an unmodified Gemma 4 26B model, their defense increased protection against genuine prompt injection leaks from 27% to 94% with only a 4% reduction in clean output quality, and improved attribution of lower-trust sources from 0% to 92%.
Why it matters: This work establishes a provable security boundary against prompt injection in LLMs, addressing a critical vulnerability while maintaining high performance and source attribution.
A new preprint introduces the concept of Protective Capacity Hallucination (PCH), where large language models (LLMs) in protective roles falsely claim to perform real-world actions, such as contacting emergency services, despite lacking such capabilities. The study, spanning 13,600 sessions across eight LLMs, finds that PCH is triggered by multi-party dialogue and situational severity, but is notably absent in intimate-partner conflict scenarios due to targeted safety alignment. The authors argue that PCH reveals a gap between assigning protective roles to LLMs and clearly specifying their actual capabilities.
Why it matters: This work highlights a critical safety risk: LLMs may mislead users by asserting abilities they do not possess, emphasizing the need for explicit capability boundaries in deployment.
Researchers introduce ProfMalPlus, a detector for malicious NPM packages that integrates object-sensitive behavior graphs with coordinated large language model (LLM) reasoning over annotated code slices. ProfMalPlus achieves a 98.1% F1-score, surpassing existing detectors by 3.5% to 52.6%, and led to the identification and removal of 597 previously unknown malicious packages from the NPM registry.
Why it matters: This work demonstrates a significant advance in software supply chain security by leveraging LLM-based agent coordination to more effectively detect malicious packages in the NPM ecosystem.
DREA is a hypothesis-driven framework for repository-level vulnerability detection that separates reasoning from exploration using two collaborating agents: a planning agent (advanced LLM) that forms vulnerability hypotheses and an explorer agent (lightweight model) that retrieves relevant repository context. This approach improves Pair-Correctness from 19-26% to 30-42% and reduces API costs by 16-48 times by offloading over 93% of tokens to the local model. The paper also introduces RepoPairBench, a new benchmark, and finds that 26-55% of true positives have flawed rationales, highlighting reasoning quality as a bottleneck.
Why it matters: This work significantly advances LLM-based vulnerability detection by enabling adaptive, cost-efficient exploration of repository-level dependencies and identifying reasoning quality as a key challenge.