Equal Accuracy, Unequal Evidence: Search APIs as Decision Surfaces for Tool-Using Agents

Jul 14, 2026

A new preprint argues that commercial search APIs should be understood as decision surfaces for tool-using agents, not just as recall tools. In tests with a frozen GPT-5.4 agent answering 100 questions using three different search providers (Brave, Tavily, Firecrawl), the study found similar answer accuracy across providers but significant differences in the quality and distribution of evidence, such as snippet informativeness and exploration patterns.

Why it matters: The findings suggest that the choice of search API affects not only answer accuracy but also the efficiency and strategy of agent decision-making, with implications for retrieval budgets and agent design.

Full story at: arXiv Computation and Language ↗