Vendor-Neutral Metric Assesses Reconstructability of Agent Safety Evidence

Jul 15, 2026

A new preprint introduces a vendor-neutral metric designed to evaluate whether evidence from agent-safety evaluations can reconstruct the decisions underlying safety claims. The metric assesses reconstructability across eight decision-property classes and includes a cross-harness adapter for generating Evidence Sufficiency Cards. Tests on public traces show sufficiency scores between 0.458 and 0.833, with replay preconditions unmet in all scored traces.

Why it matters: This work provides a standardized approach to evaluating the validity of agent safety claims, addressing a key challenge in building trustworthy AI systems.

Full story at: arXiv Software Engineering ↗