DrawingVQA: A Real-World Benchmark for Multi-Depth Visual-Textual Reasoning on Construction Drawings
DrawingVQA introduces the first benchmark specifically designed to evaluate multimodal large language models on real-world construction drawings. The dataset includes 33 authentic construction drawings and 92 expert-curated question-answer pairs, covering three levels of reasoning complexity. Evaluations show a significant performance gap between current models and human experts, especially at more advanced reasoning levels.
Why it matters: This benchmark provides a crucial resource for advancing AI capabilities in domain-specific multimodal reasoning, with direct relevance to engineering and construction workflows.
Full story at: arXiv AI/ML ↗