Code-MUE: Black-Box Uncertainty Estimation for Code LLMs via Execution-Based Semantic Graphs

Jul 15, 2026

Researchers introduce Code-MUE, a black-box framework for measuring uncertainty in code large language models (LLMs) by analyzing execution-based semantic interaction graphs. The method quantifies semantic diversity using Von Neumann entropy and demonstrates a strong negative correlation with functional correctness (Spearman's up to -0.98) across eight models. Code-MUE outperforms lexical and embedding-based baselines for risk detection and selective prediction, addressing limitations of existing black-box uncertainty metrics for code.

Why it matters: This work provides a practical and effective approach to uncertainty estimation for closed-source code LLMs, supporting safer and more reliable automation in software engineering.

Full story at: arXiv Software Engineering ↗