Researchers present AgentScore, a method that leverages large language model (LLM) agents to automatically generate clinical scoring systems composed of interpretable decision rules. AgentScore outperforms existing score-generation methods across eight clinical prediction tasks and achieves AUROC comparable to more flexible interpretable models, while adhering to deployability constraints. The method also demonstrates higher discrimination than established guideline-based scores on two externally validated tasks.
Why it matters: This work advances the automation of clinically deployable, interpretable scoring systems, potentially improving the translation of machine learning models into routine clinical practice.
Researchers investigated a frozen 2.6B parameter looped transformer (Ouro-RLTT) and found that its hidden states can predict task success on GSM8K (AUROC 0.797) and other properties, such as branch survival and content ranking. However, attempts to intervene in these internal states did not yield any validated improvement in model capability. The authors term this phenomenon 'operational proto-introspection,' where internal signals are accessible for readout but not yet controllable for steering outcomes.
Why it matters: This work highlights a significant gap between being able to interpret a language model's internal processes and being able to control or improve its behavior based on that interpretability.
Researchers introduce a decision-aware weak-to-strong (W2S) learning framework that addresses the challenge of scarce labeled data in operational decision-making. The approach first trains a weak model on limited labeled data, then uses its predictions to generate soft labels for abundant unlabeled contexts, enabling the training of a stronger model. Theoretical analysis provides bounds on decision risk, and experiments on synthetic and real-world tasks support the method's effectiveness.
Why it matters: This work offers a principled way to leverage abundant unlabeled data to improve decision-making when labeled data are limited, addressing a common challenge in real-world applications.
A new preprint introduces the concept of the 'information shadow'—a set of phenomena that text-trained language models cannot learn, regardless of model scale. The authors formally identify three types of structural limits: (1) information that language cannot express, (2) functions that are statistically non-identifiable from training data, and (3) functions that are representable but unreachable by gradient-based training. Each limit is demonstrated with provable probes and controls that rule out capacity or modality artifacts.
Why it matters: This work provides a formal framework for understanding fundamental, scale-independent limits of language models, informing benchmark design and capability auditing.
Researchers introduce a multi-agent simulation framework where large language model (LLM) agents interact, evaluate each other, and adapt their behavior through in-context learning enhanced by a coaching signal. By designing behavioral reward functions that reflect key drivers of online engagement, the study finds that these agents develop stable interaction patterns and network structures resembling those found in real online communities.
Why it matters: This work provides a principled testbed for studying collective dynamics in LLM populations and offers insights into how artificial agents can approximate or diverge from human-like social behavior.
Researchers introduce Reference-Relative Policy Optimization (RRPO), a method that generalizes Group Relative Policy Optimization (GRPO) by using reference-relative contrastive comparisons instead of correctness-based advantage construction. RRPO employs stratified conditional rollouts to create positive and negative anchor sets, then trains a metric projection head with a set-contrastive objective to define contrastive advantages for policy optimization. The approach is shown to be competitive with verifier-based optimization across tasks such as verifiable reasoning, open-ended generation, and post-supervised fine-tuning settings.
Why it matters: RRPO broadens the applicability of group-relative optimization methods to tasks lacking a single correctness criterion, potentially enabling reinforcement learning in more diverse and open-ended domains.
Researchers propose Signed Rectified Flow (Signed RF), a generalization of Rectified Flow that enables generative models to both promote desired distributions and suppress undesired ones by leveraging a signed measure. The method provides a principled way to incorporate negative information and exclusion constraints into generative modeling. Experiments show that Signed RF improves the fidelity-diversity trade-off on ImageNet, reduces memorization, and decreases nudity in Stable Diffusion 3.5 without degrading output quality.
Why it matters: This work introduces a novel framework for controlling generative models with exclusion constraints, offering practical advances in safety and content moderation.
A new method called domain-conditional position offsets, which involves adding a single learned vector to the first token embeddings, has been shown to reduce the cold-start penalty in autoregressive language models. This approach trains in minutes on about 100 documents, requires no changes to model weights, and reduces in-domain perplexity by up to 27% across models ranging from 410M to 8B parameters. It outperforms direct logit-bias correction and is more lightweight than LoRA or soft prompts.
Why it matters: This technique provides a practical, low-cost way to improve language model accuracy on short, domain-specific inputs without retraining or added latency.
A new preprint identifies a 'selection bottleneck' in multi-agent large language model (LLM) pipelines, demonstrating that the quality of the selection mechanism can be more critical than team diversity for output quality. In experiments across 42 tasks, judge-based selection achieved a 0.810 win rate against a single-model baseline, while homogeneous teams performed near chance (0.512). The study suggests that improving the selector may have a greater impact than increasing generator diversity in such pipelines.
Why it matters: This work challenges the common assumption that diversity is always beneficial in multi-agent LLM systems, highlighting the decisive role of aggregation methods in pipeline performance.
Researchers introduce SAFE, a multi-agent reinforcement learning (MARL) framework that leverages a counterfactual baseline conditioned on a self-evolving default action derived from each agent's experience buffer. This method enables unbiased counterfactual credit assignment in continuous action spaces and ensures convergence to local optima. Experiments on cooperative vehicular tasks indicate that SAFE outperforms existing state-of-the-art MARL models.
Why it matters: The approach addresses a major challenge in multi-agent RL by enabling unbiased credit assignment in continuous action spaces, which is important for real-world cooperative applications such as autonomous vehicles.
A new preprint introduces an agentic calibration approach where a large language model (LLM) serves as the optimizer for grey-box simulation models. The method is evaluated against Nelder-Mead and Bayesian Optimization on an anal cancer simulation model, showing lower error and requiring fewer model evaluations in unconstrained settings, and achieving comparable performance under constraints. Constraint handling is simplified by using plain-language prompts, though the approach incurs higher per-iteration inference costs.
Why it matters: This work suggests that LLMs can act as effective and auditable optimizers for complex simulation models, potentially simplifying calibration workflows and reducing the need for specialized constraint-handling techniques.
A systematic comparison of two geospatial foundation models, TerraMind and THOR, finds that architectural choices—particularly patch size and decoder type—account for more performance variance than the specific model identity. The study, conducted under the European Space Agency's Φ-lab, also highlights that TerraMind and THOR represent complementary strategies: TerraMind emphasizes pretraining-time scale, while THOR focuses on inference-time tokenization. The authors propose a diagnostic ablation methodology for understanding model differences across diverse geospatial tasks.
Why it matters: This research offers a new methodology for diagnosing and interpreting performance differences in geospatial AI models, moving beyond aggregate leaderboards to inform future model development.
Assistax is a new GPU-accelerated reinforcement learning benchmark for assistive robotics, built on JAX and MuJoCo-MJX. It supports multi-agent RL with a simulated human partner and introduces an ad-hoc teamwork pipeline for evaluating robot policies with unseen human policies. The benchmark achieves up to 412x faster simulation than comparable CPU-based environments and provides pre-trained humanoid policies for research use.
Why it matters: Assistax enables high-throughput, multi-agent RL research in assistive robotics, addressing a key bottleneck in developing and evaluating human-robot interaction policies.
A new preprint presents a systematic comparison of Monte Carlo Dropout and Deep Ensembles for uncertainty quantification in AI-driven crash simulation surrogates, using an open-source bumper beam benchmark. The study leverages concrete dropout from NVIDIA PhysicsNeMo to eliminate manual hyperparameter tuning and evaluates both methods on accuracy, calibration, and computational cost. Results reveal a trade-off between accuracy and calibration, challenging the assumption that deep ensembles are always the gold standard, and show that well-calibrated, hyperparameter-free uncertainty estimates can be achieved at lower computational cost.
Why it matters: This work advances the reliability and efficiency of uncertainty quantification in safety-critical engineering simulations, potentially improving trust and adoption of AI surrogates in engineering workflows.
Interactive Training 2 is an open-source control plane designed to steer live model training runs through a shared protocol. It allows both humans and automated controllers to submit requests, which are validated and applied at safe control points during training. The system features a customized Aim workspace that integrates live metrics, controls, and a chronological record of requests and outcomes. Demonstrations were conducted across five NLP and reinforcement-learning workflows.
Why it matters: This system offers a reusable and auditable foundation for human- and agent-guided training, potentially enhancing transparency and control in live model training processes.
Researchers have introduced FALCON-Discover, a post-hoc, model-agnostic framework designed to detect regions in prediction space where AI models are confidently wrong. By combining multiple discrepancy signals—such as confidence, local support, neighborhood agreement, and perturbation stability—the method outperforms standard calibration and trust-scoring techniques in identifying concentrated areas of false confidence. The study finds that dangerous overconfidence is recurrent but varies by regime, and that different detection strategies excel depending on the dataset.
Why it matters: This work addresses a critical safety issue by providing a more effective way to detect and mitigate localized overconfidence in AI models, which can lead to high-risk errors.
Researchers introduce FLITE, a federated fine-tuning method that leverages a low-rank, seed-regenerable factorization to reduce per-client communication to just 1,280 floats (~5 KB) per round—an 8,718-fold reduction compared to full-weight FedAvg. FLITE achieves 74.67% accuracy on CIFAR-100 with ResNet-18, within approximately 0.5 percentage points of FedAvg, and matches or exceeds FedAvg performance under strong non-IID data skew. The method can further compress communication to 648 bytes per round using int4 latents without loss of accuracy.
Why it matters: FLITE significantly lowers the communication requirements for federated learning, enabling practical fine-tuning of large models on devices with limited bandwidth.
A new preprint reveals that as language models increase in size, they not only become more capable but also more susceptible to compounding errors through a hidden auto-regressive risk regime. The study demonstrates that while the knowledge gap narrows with scale, knowledge degradation accelerates significantly, leading to a self-perpetuating failure mode that the model cannot detect on its own. The researchers also show that this risk regime is causal and can be mitigated, but standard self-monitoring techniques often fail to identify it.
Why it matters: This work highlights a fundamental and previously underappreciated reliability issue in large language models that intensifies with scale, challenging the notion that bigger models are inherently more reliable.
Researchers introduce GA-AMLS, a rare-event Monte Carlo method that operates in the continuous activation space of language models to estimate extremely low probabilities, addressing issues like zero-estimate collapse and systematic bias found in previous approaches. They also propose the Shifted-Power Bregman Loss, a proper scoring rule that remains finite for zero estimates and allows for tunable asymmetry between underestimation and overestimation penalties. Experiments on small transformer models show that GA-AMLS achieves the lowest loss under symmetric evaluation, while methods with overestimation bias perform better under asymmetric penalties. The work demonstrates that estimator choice should be matched to deployment context and establishes activation space as a tractable domain for rare-event estimation in language models.
Why it matters: This work advances the robustness and reliability of rare-event risk estimation in language models, which is crucial for ensuring safety in large-scale and adversarial applications.
A new preprint formalizes and empirically investigates the limits of support-preserving alignment and bounded filtering in eliminating harmful outputs from large language models (LLMs). The authors provide theoretical arguments and test multiple models, showing that harmful output rates decrease with increased filtering but consistently plateau above zero, regardless of filtering compute. This suggests that current alignment and filtering methods cannot fully eliminate harmful behavior in LLMs.
Why it matters: The findings challenge the assumption that alignment and filtering can drive harmful LLM outputs to zero, raising important questions about the safety guarantees of deployed language models.