Study Reveals Gaps in Test Coverage for AI-Generated Pull Requests
A new analysis of 4,882 agent-generated pull requests from the AIDev dataset finds that AI coding agents include test changes in only 49.6% of PRs that modify code under test. Existing tests cover just 61.5% of changed executable lines in Java and 27.0% in Python, with error-handling constructs being the most consistently under-tested, with miss rates up to 86.0% in Java and 81.0% in Python. Agent-written tests improve coverage in only a minority of cases, highlighting persistent gaps in automated test practices.
Why it matters: As AI coding agents become more autonomous, understanding and improving test coverage is critical to preventing software regressions and ensuring code reliability.
Full story at: arXiv Software Engineering ↗