Large-Scale Study Reveals LLM Agents Hallucinate Skill Names, Enabling Supply-Chain Attacks

Jul 15, 2026

A large-scale study analyzing 15,000 prompts across 12 LLM and agent configurations found that hallucination of skill names is widespread, with rates averaging 36.0% for standalone LLMs and 36.9% for agents, and rising to 43.1% on real-world developer questions. These hallucinated names can be exploited by adversaries who pre-register malicious skills, enabling supply-chain attacks. The study evaluated four defenses and found that the strongest, retrieval grounding, reduced hallucination to 3.2% but significantly reduced the system's usefulness, with correct skill recommendations dropping to about one in six.

Why it matters: This vulnerability exposes LLM agent ecosystems to easy supply-chain attacks, and current defenses severely compromise usability, highlighting the need for structural changes to registries and recommendation pipelines.

Full story at: arXiv Cryptography and Security ↗