Text and language model news — Page 23

Language models and text-based AI systems, including reasoning, generation, and understanding of written language.

ResearchOfficialarXiv Software Engineering

Knowledge-Guided Synthetic Bug Feedback Improves LLM-Based Unit Test Generation

Researchers propose a framework that leverages historical real-bug mechanisms to generate synthetic bugs, which are then used as feedback to guide large language model (LLM)-based unit test generation. By constructing structural and semantic representations of bug records and retrieving relevant mechanisms, the system instantiates synthetic bugs as feedback targets for iterative test improvement. Evaluation on the Defects4J benchmark shows that this approach improves real-bug detection compared to several baseline methods.

Why it matters: This work demonstrates a novel and effective method for enhancing the ability of LLM-generated unit tests to detect real software bugs, potentially increasing software reliability.

Jul 14, 2026

ResearchOfficialarXiv Software Engineering

Model Instability in Software Analytics Can Be Quantified and Reduced, Study Finds

A new preprint reports that model instability is a significant issue in software analytics, with repeated runs of the same optimizer agreeing on only 13.7% of test cases across 127 multi-objective software engineering optimization problems. The researchers show that by adjusting label allocation, model complexity, and scoring splits, agreement can be improved by a factor of 4.8 and the standard deviation of optimization error reduced by 22% on average, without degrading recommendation quality. The study argues that instability is not just noise but a measurable and manageable property, and recommends treating it as a standard evaluation axis.

Why it matters: This work provides a practical framework for measuring and managing model instability, which could enhance the reliability and trustworthiness of software engineering optimization tools.

Jul 14, 2026

ResearchOfficialarXiv Software Engineering

ACQUIRE: QA-Driven Framework Improves LLM-Based Code Repair via Pre-Repair Knowledge Acquisition

A new framework called ACQUIRE is proposed to enhance LLM-based code repair by explicitly acquiring repository knowledge before generating patches. The system uses a Questioner-Answerer collaboration to identify and fill knowledge gaps, resulting in more accurate repairs. On the SWE-bench Verified benchmark, ACQUIRE improves Pass@1 accuracy by up to 4.4 percentage points compared to prior pre-repair methods, with only modest overhead.

Why it matters: This approach addresses a major source of factual errors in LLM coding agents by systematically improving their understanding of code repositories before attempting repairs.

Jul 14, 2026

ResearchOfficialarXiv Software Engineering

RepTran: Search-Based Repair of Transformer Models

Researchers introduce RepTran, a search-based method designed to repair Transformer models by focusing on their feed-forward networks. RepTran identifies suspicious weights using a combination of variance-based and bidirectional scores, then optimizes these weights through differential evolution. Evaluated on 18 fault benchmarks derived from CIFAR-100 and Tiny-ImageNet, RepTran achieved a 74.7% average repair rate, statistically outperforming existing DNN repair methods such as Arachne.

Why it matters: This work presents a novel approach for automated repair of Transformer models, potentially improving the reliability of AI-enabled software systems.

Jul 14, 2026

ResearchOfficialarXiv Software Engineering

OpsMem: Dual-Memory Framework for LLM-Based Failure Diagnosis

Researchers introduce OpsMem, a dual-memory framework that coordinates short-term diagnostic state with long-term operational experience using cross-memory resonance. On a real-world Huawei microservice failure diagnosis dataset, OpsMem outperforms agentic-reasoning and knowledge-augmented baselines, improving diagnosis accuracy metrics by up to 46.88% and 18.39%.

Why it matters: OpsMem demonstrates a significant advance in LLM-based failure diagnosis by effectively integrating short- and long-term memory, leading to notably improved diagnostic performance in complex software systems.

Jul 14, 2026

ResearchOfficialarXiv Software Engineering

LLMs Fail to Generate Compilable Unity Game Scenes in Single Pass, Study Finds

A new preprint evaluates large language models' (LLMs) ability to generate Unity C# game scenes in a single pass, without iterative repair. Across 10,400 generations, none of the outputs compiled successfully. The study categorizes over 90,000 compiler errors, finding that most failures stem from invented or misused Unity APIs rather than general code structure, highlighting a key limitation in LLMs' domain-specific knowledge.

Why it matters: This work demonstrates a fundamental barrier to using LLMs for domain-specific code generation without iterative feedback, clarifying the gap between LLMs' general knowledge and practical software engineering needs.

Jul 14, 2026

ResearchOfficialarXiv Statistical ML

Edge Cluster Expansion with Radial Rotary Attention Improves Machine Learning Interatomic Potentials

A new study introduces two novel interaction building blocks for machine learning interatomic potentials: the Edge Complex Product Basis and Radial Rotary Complex Attention. The proposed model, TECE-OAM-RRA-1.0, demonstrates state-of-the-art performance on the Matbench Discovery benchmark, surpassing previous methods. The work also presents improvements to the Atomic Cluster Expansion module and evaluates the approach on multiple datasets.

Why it matters: Improved accuracy in machine learning interatomic potentials can significantly enhance materials science simulations and discovery.

Jul 14, 2026

ResearchOfficialarXiv Software Engineering

Foundation Models of Human Cognition Improve Prediction of Program Comprehension

A new study evaluates Centaur, a foundation model trained on data from 160 general psychological experiments, on nine program-comprehension tasks. Centaur's predictions of human responses align more closely with actual human data than those of its base model, Llama 3.1. The model relies less on prior trial information and more on task-specific context, indicating that cognitive patterns learned from general psychology can transfer to complex software engineering tasks.

Why it matters: This research demonstrates that foundation models of human cognition can be leveraged to better predict developer behavior, potentially informing the design of improved tools for code understanding and developer support.

Jul 14, 2026

ResearchOfficialarXiv Software Engineering

Agentic Code Requires More Post-Merge Maintenance and Introduces More Vulnerabilities

A longitudinal study of 182 repositories finds that while agentic and human code contributions have similar overall maintenance rates, agentic code requires significantly more corrective maintenance and introduces more security weaknesses and dependency vulnerabilities. The study also finds that a 10 percentage-point increase in a project's no-review rate is associated with a 6% increase in agentic maintenance burden.

Why it matters: As AI coding agents become more common, this research shows that merge success is not enough; agentic tools must be designed to produce code that remains secure and maintainable over time.

Jul 14, 2026

ResearchOfficialarXiv Software Engineering

AfterVibe: Recovering Natural-Language Specs from Vibe Coding Sessions

AfterVibe is a framework that extracts abstract natural-language specifications from AI-assisted coding sessions by analyzing both the code and the conversation that produced it. The framework validates these specs through a regeneration test, where a separate AI agent re-implements the code from the spec alone, achieving a mean regeneration score of 5.06 out of 6.0 on 72 real-world projects. The specs are shown to be both abstract and strong, outperforming existing human-authored descriptions and supporting iterative refinement.

Why it matters: AfterVibe demonstrates that natural-language specifications can effectively capture developer intent and may serve as a primary review artifact as AI-generated code becomes more prevalent.

Jul 14, 2026

ResearchOfficialarXiv Software Engineering

LLMs with Error Mitigation Techniques Effectively Classify Static-Analysis Alerts

A new study evaluates large language models (LLMs) for classifying static-analysis alerts as real bugs or false alarms, applying consistency checks and reasoning evaluation to reduce errors. Mid-tier reasoning LLMs achieved at least 98% recall and 94.8% specificity across three benchmark suites. The researchers also used LLM-generated trigger programs to provide independent evidence of real flaws, finding that valid triggers reliably indicated true positives.

Why it matters: This work suggests LLMs, when combined with error mitigation strategies, could significantly reduce the manual effort required to review static-analysis alerts in software security.

Jul 14, 2026

ResearchOfficialarXiv Multiagent Systems

Multi-Agent Routing as Set-Valued Prediction: A WildChat Benchmark and Cost-Aware Evaluation

Researchers introduce a new benchmark based on WildChat for evaluating multi-agent routing from natural-language prompts as a set-valued prediction problem. The study demonstrates that supervised routing models, including a fine-tuned encoder and a linear multilabel classifier, significantly outperform nearest-neighbor and zero-shot LLM baselines. The benchmark includes a reproducible evaluation protocol with set-level metrics and cost-aware constraints, enabling systematic study of accuracy-cost trade-offs in multi-agent systems.

Why it matters: This work enables reproducible and systematic evaluation of multi-agent routing strategies, which is important for developing efficient and cost-effective multi-agent systems.

Jul 14, 2026

ResearchOfficialarXiv Multiagent Systems

Interlat: Enabling Agents to Communicate Entirely in Latent Space

Researchers introduce Interlat, a paradigm where LLM-based agents communicate using continuous latent states rather than natural language. Inspired by telepathy, this approach allows agents to share internal representations directly, bypassing the limitations of token-based communication. Experiments show that Interlat outperforms chain-of-thought prompting and single-agent baselines, and that compressing latent messages can accelerate inference by up to 24x while maintaining competitive performance.

Why it matters: This work demonstrates a novel and potentially more efficient method for multi-agent communication, challenging the dominance of natural language and opening new directions for collaborative AI systems.

Jul 14, 2026

ResearchOfficialarXiv Multiagent Systems

Dynamics of Learning under User Choice: Overspecialization and Peer-Model Probing

A recent preprint demonstrates that machine learning models trained on data from users who self-select platforms can become overspecialized, leading to poor performance on the broader population. The authors identify this 'overspecialization trap' and introduce a probing algorithm that leverages predictions from peer models to mitigate the issue. Theoretical analysis and experiments on real-world datasets support the effectiveness of this approach under certain conditions.

Why it matters: This work exposes a key limitation in current multi-platform learning systems and proposes a theoretically grounded solution with practical implications for improving model generalization.

Jul 14, 2026

ResearchOfficialarXiv Multiagent Systems

PerspectiveGap: A Benchmark for Multi-Agent Orchestration Prompting

Researchers have introduced PerspectiveGap, a benchmark designed to evaluate large language models' (LLMs) ability to compose orchestration prompts for multi-agent systems. The benchmark features 110 scenarios across 10 topologies, testing models on role-fragment assignment and free-form prompt writing. Results show that even the best-performing model (GPT-5.5) achieves only a 62% pass rate, while the average combined pass rate across 33 models is 17.2%.

Why it matters: PerspectiveGap highlights a significant and under-explored gap in current LLM capabilities for orchestrating multi-agent systems, providing a new foundation for systematic evaluation and improvement.

Jul 14, 2026

ResearchOfficialarXiv Multiagent Systems

INFORM: Disentangling Intrinsic Importance from Emergent Structure in Multi-Expert Orchestration

A new interpretability method called INFORM decouples expert interaction structure from functional importance in multi-expert LLM systems. The method demonstrates that experts frequently selected by routing policies may have limited actual influence, while less frequently used experts can be structurally critical. INFORM uses gradient sensitivity to identify intrinsically important experts, whose removal leads to a disproportionate collapse in the system's interaction structure. This approach reveals functional and structural dependencies that are not captured by standard routing or accuracy metrics.

Why it matters: This work offers a principled framework for understanding and optimizing multi-expert LLM systems, uncovering hidden dependencies that can inform more robust orchestration strategies.

Jul 14, 2026

ResearchOfficialarXiv Multiagent Systems

Hierarchical Memory Architecture Enables Long-Horizon Multi-Agent LLM Workflows

A new preprint introduces Ensemble QSP, a multi-agent framework featuring a three-layer hierarchical memory architecture that maintains bounded and constant context throughout extended computational modeling tasks. The system coordinates five specialist worker agents under domain-expert principal investigators, using structured checklists and domain knowledge to enforce constraints. Benchmarking shows robust autonomous pharmacokinetic-pharmacodynamic model selection, consistent quality across different LLMs, and improved parameter recovery compared to single-agent baselines, all without human intervention.

Why it matters: This work demonstrates a practical solution to the context window limitations of LLMs, enabling fully autonomous, multi-session scientific modeling workflows.

Jul 14, 2026

ResearchOfficialarXiv Multiagent Systems

StructAgent: A Unified Causal Structure Framework for Long-Horizon Digital Agents

Researchers introduce StructAgent, a framework that organizes digital agent state and workflow around a unified causal representation of task progress. This approach enables explicit progress checkpointing and verifier-backed state transitions, leading to substantial improvements in success rates for long-horizon computer-use tasks. StructAgent boosts Qwen3.5-9B's performance on OSWorld-Verified from 27.0% to 46.9%, and achieves 78.9% with MiniMax-M3, while also generalizing to environments like Minecraft.

Why it matters: StructAgent represents a significant advance in enabling digital agents to reliably execute complex, long-horizon tasks by introducing structured, verifiable progress tracking.

Jul 14, 2026

ResearchOfficialarXiv Multiagent Systems

WattCouncil: Context-Aware Household Energy Scenario Generation With Governed LLMs

Researchers introduce WattCouncil, a framework that uses multiple LLM-based agents in specialized roles to generate synthetic household energy demand data. The system incorporates cultural, temporal, and physical constraints to create context-sensitive daily routines, and its outputs are evaluated against the CER dataset of 4,232 households. The framework also includes ablation studies to assess consistency.

Why it matters: This work offers a novel approach to generating realistic, privacy-preserving household energy data, addressing a key bottleneck in smart-grid research.

Jul 14, 2026

ResearchOfficialarXiv Machine Learning

When Data Imbalance Helps: Robust Generalization Through Shortcut Saturation

A new preprint reports that, contrary to conventional wisdom, data imbalance can actually promote robust generalization in sufficiently capable models when spurious correlations are present. In a synthetic task, a 2-layer transformer achieved 100% adversarial accuracy in 77% of training runs at a high spurious ratio (0.90), compared to 0% at a balanced ratio (0.50). This effect was not observed in 1-layer models, where data imbalance led models to rely on the shortcut feature instead.

Why it matters: This finding challenges standard assumptions about data balance and suggests new strategies for training models to resist spurious correlations.

Jul 14, 2026