Hollow-LLM Attack Shows Zero-Knowledge Verification Can Be Tricked by Ghost Weights
A new preprint introduces the Hollow-LLM Attack, demonstrating that zero-knowledge (ZK) verification methods for large language model (LLM) inference can be circumvented by using specially constructed 'ghost weights.' These weights allow a provider to produce outputs that pass ZK verification while only performing the computation of a much smaller model, undermining the intended guarantee that the advertised model was actually run. The work highlights a gap between proof of correct output and proof of computational effort in current ZK-based LLM verification schemes.
Why it matters: This reveals a fundamental vulnerability in proposed cryptographic verification of remote LLM inference, raising concerns for trust and accountability in AI-as-a-service settings.
Full story at: arXiv Cryptography and Security ↗