Structured Feedback Improves Repair in an LLM Agent Loop

Jul 17, 2026

A new preprint introduces VeriHarness, a code-controlled agent loop designed to test how structured feedback affects LLM agent performance. In experiments with 50 TextWorld games, providing feedback that included failure location, observed value, and admissible alternatives significantly improved task completion rates for both Qwen2.5-Coder-14B and Llama-3.1-8B models. The study found that the admissible alternatives field was the main contributor to these gains, and that presenting feedback in prose was nearly as effective as using JSON format.

Why it matters: This work offers empirical evidence on optimizing feedback mechanisms for LLM agents, which could enhance their reliability in automated code generation and complex task execution.

Full story at: arXiv Software Engineering ↗