Privacy-Centric Edge-Cloud Collaborative LLM Inference Framework

Jul 16, 2026

A new framework proposes privacy-preserving large language model (LLM) inference by splitting computation between edge devices and the cloud. It uses authenticated key-value (KV) cache and AES-GCM encryption to protect user data, while distributing tasks such as preprocessing, embedding, and partial decoding to the edge, and more intensive inference to the cloud. Evaluations show reductions in per-token latency by up to 46.1% and downlink payloads by up to 67.4% compared to baseline split inference, with performance comparable to full cloud inference.

Why it matters: This framework offers a practical solution to balancing latency, hardware limitations, and privacy for LLM inference on consumer and embedded devices.

Full story at: arXiv Cryptography and Security ↗