Evidence-in-the-Loop: Trace-Driven Optimization for Customer-Service LLM Agents
A new preprint introduces an evidence-grounded workflow for deploying large language model (LLM) agents in real-world customer service. The system integrates hybrid retrieval-augmented generation (RAG) techniques—including BM25, vector recall, cross-encoder reranking, and rule-based evidence—within a LangGraph DAG for policy-compliant orchestration. The authors propose three reusable deployment patterns: hybrid RAG evidence construction, evidence-grounded decision making, and trace-driven RAG improvement, all demonstrated in a production setting.
Why it matters: This work offers a practical and auditable framework for deploying LLM agents in customer service, emphasizing evidence boundaries and policy compliance.
Full story at: arXiv Information Retrieval ↗