FinResearchBench II: A Deep Research Benchmark with Consensus-Derived Gold Rubrics for Distinguishing Financial Report Quality

Jul 15, 2026

Researchers introduce a scalable pipeline for generating high-quality evaluation rubrics for financial report assessment without requiring human experts in the final evaluation loop. Using 104 real-world queries and 14,450 candidate rubrics, they demonstrate that LLM-based evaluation achieves 98.67% label-level agreement with human experts on jointly unanimous items. The process yields a final set of 2,600 consensus-derived gold rubrics, enabling differentiated rankings across 10 deep research systems.

Why it matters: This work enables large-scale, automated evaluation of AI-generated financial reports, reducing reliance on human experts and facilitating more efficient system comparison and improvement.

Full story at: arXiv Computation and Language ↗