SemaDiff: Identifying Semantic-Changing Commits with Generated Code and Tests

Jul 16, 2026

SemaDiff is a novel approach that uses large language model (LLM)-generated code and tests to distinguish between semantic-preserving and behavior-changing commits in software repositories. By generating additional dependent classes and tests for both pre- and post-commit code versions, SemaDiff can detect behavioral differences. In evaluation on a manually annotated dataset of 183 Java commits, SemaDiff achieved 76% accuracy and 100% precision in identifying semantic-changing commits.

Why it matters: This method advances software repository mining by enabling more reliable identification of purely refactoring commits, which is important for tasks such as debugging, fault localization, and constructing bug datasets.

Full story at: arXiv Software Engineering ↗