A new preprint investigates the premise that quantization loss in large language models can be effectively predicted from per-layer sensitivities. The authors show that per-layer effects account for 85–93% of loss variance at 4-bit precision, and introduce a coverage model and additive predictor that achieve low KL divergence on models ranging from 30B to 355B parameters. Notably, their approach maintains performance on code and reasoning tasks below four bits, outperforming gradient-based methods in these regimes.
Why it matters: This work offers a theoretically grounded and empirically validated approach for mixed-precision quantization, potentially enabling more efficient deployment of large language models with minimal accuracy loss.
COMPASS introduces a method that uses empirical mobility scaling laws as feedback to guide large language model (LLM)-based human mobility simulation. By integrating population-level statistical regularities into the prompt construction process, COMPASS bridges the gap between individual trajectory realism and collective mobility patterns. The approach demonstrates improved performance over existing LLM-based simulators on two public datasets.
Why it matters: This work advances LLM-based geospatial simulation by enabling both realistic individual behaviors and accurate population-level mobility patterns, addressing a key limitation of prior methods.
A new preprint challenges the prevailing belief that information dependency, such as rote memorization, is responsible for training data exposure in image reconstruction attacks. The authors demonstrate that adversarially non-robust features are the actual cause and introduce Anti Adversarial Training (AT-AT), a method that intentionally learns such features to improve both privacy against model inversion attacks and model accuracy.
Why it matters: This work revises the understanding of training data privacy in machine learning and highlights a new tradeoff between privacy and adversarial robustness.
Researchers present conDitar-dev, a conditional diffusion-based framework for structure-based drug design that generates ligands with strong binding affinities and improved ADMET properties. The model outperforms state-of-the-art baselines on a benchmark of human disease targets, achieving an average binding score of -8.85 kcal/mol and up to 73% improvement in ADMET properties. Experimental validation on PD-L1 and CSF1R targets yielded molecules with micromolar binding affinities and nanomolar IC50 values, demonstrating real-world applicability.
Why it matters: This work advances AI-driven drug discovery by demonstrating a method that generates molecules with both computational and experimental validation, potentially accelerating the development of new therapeutics.
A new study demonstrates that the widely used Gale-Shapley stable marriage algorithm is vulnerable to privacy attacks when the proposer side is malicious. The researchers show that, under these conditions, it is possible to recover the full preference lists of honest agents, including in real-world scenarios such as the National Resident Matching Program. The findings suggest that current implementations may expose sensitive information and highlight the need for privacy-preserving alternatives.
Why it matters: The results reveal critical privacy risks in matching algorithms used in sensitive applications, emphasizing the urgency for improved privacy protections.
A new study proposes a feature-guided zero-shot framework that leverages large language models (LLMs) for early chronic kidney disease (CKD) screening without the need for dataset-specific training. By selecting a compact set of clinically meaningful and readily available features, the researchers evaluated four LLMs across three heterogeneous CKD datasets from different countries. The models achieved consistent and statistically significant improvements in balanced accuracy and probability estimates, reaching performance levels suitable for screening purposes.
Why it matters: This work shows that LLMs can enable training-free, clinically meaningful CKD screening using minimal, community-accessible features, potentially improving access to screening in resource-limited settings.
Researchers have introduced JoPMol, a generative model that unifies gene expression profiles, molecular structure text, and chemical property data to enable precision molecular design. JoPMol demonstrates superior performance compared to state-of-the-art methods across several evaluation metrics and exhibits strong generalization in transfer tasks and biologically grounded simulations. The model's code is publicly available.
Why it matters: This work represents a notable advance in AI-driven drug discovery by enabling coordinated control over both biological and chemical conditions for personalized molecular design.
A new preprint systematically studies how the training duration of domain expert models affects the quality of merged multi-task models. The authors fine-tuned experts on five domains and three model sizes, evaluating five merging methods at various training checkpoints. They found that simple averaging performs worse as experts overfit, while sparsification-based merging methods achieve their best results well past the validation loss optimum. The study concludes that training duration and merging method should be selected together for optimal results.
Why it matters: This work provides actionable insights for practitioners seeking to improve multi-task model merging, a key technique for combining specialized models without co-training.
A new preprint introduces a white-box instrument based on hidden deterministic finite automata to separately measure reward attainment and latent-state learning in reinforcement learning (RL) agents. The study demonstrates that high reward does not necessarily indicate that an agent has learned the underlying task state, distinguishing between 'perception gaps' (where latent state is not recoverable from observations) and 'planning gaps' (where state is recoverable but not used). The authors show that optimizer strength, task structure, and observation informativeness each influence the relationship between reward and state learning.
Why it matters: This work challenges the common assumption that high reward in RL implies genuine task understanding, providing a method to diagnose when agents exploit shortcuts rather than learning true latent states.
CARE-LoRA is a method designed to reduce the memory bottleneck of activations during LoRA fine-tuning by replacing full input activations with low-rank compressed activations from the LoRA branch. It introduces a lightweight reconstruction matrix computed during the forward pass, enabling gradient reconstruction in backpropagation while keeping LoRA matrices fully trainable. Experiments across various models and tasks show that CARE-LoRA achieves competitive or superior performance compared to standard LoRA and its variants, with a substantially reduced memory footprint.
Why it matters: This approach addresses a major memory limitation in fine-tuning large models, potentially making such processes more feasible on hardware with limited memory.
Researchers present Support Vector Attention (SV-Attention), a novel max-margin memory mechanism for attention models that enables certified token eviction and exact unlearning. SV-Attention achieves higher rare-item recall (0.86 vs. 0.32) and better retention (0.80 vs. 0.05 deterioration hours) on MIMIC-IV data streams compared to baselines, and demonstrates improved compression (2.178 BPC vs. 2.383) on enwik8 relative to a sliding-window Transformer. The method also supports surgical forgetting, exact editing, and patient-record deletion in real-world scenarios.
Why it matters: This work introduces a principled and practical approach to certified unlearning in attention mechanisms, addressing a key challenge in machine learning privacy and data management.
A new training-free attribution method is introduced to analyze the dependencies of feedforward network (FFN) neurons in Transformer models. The study finds that small, sparse subsets of upstream activations and attention outputs can preserve neuron activations with high fidelity, even when other inputs are masked. This reveals that FFNs, despite their dense parameterization, have sparse and structured inter-layer dependencies at the neuron level. The method is scalable and can be used for circuit-level interpretability and identifying sparse pathways for potential efficiency gains.
Why it matters: This work offers a practical and scalable approach to understanding and potentially optimizing the internal structure of Transformer models, which could lead to more efficient inference and improved interpretability.
Researchers introduce the first pseudo-polynomial-time exact algorithm for computing Data Shapley values in weighted k-nearest-neighbor (KNN) regression, addressing a longstanding computational barrier. The work also presents a certified fully polynomial-time approximation scheme (FPTAS) with machine-checkable error bounds and extends the approach to soft-label multi-class prediction. An open-source implementation and the first exact ground-truth dataset for weighted-regression Data Shapley are provided.
Why it matters: This advance enables deterministic, certified data valuation in weighted KNN regression, providing a reliable reference for auditing and improving approximate methods.
A new preprint introduces an agentic AI scientific community composed of virtual laboratories that autonomously discover neural operator architectures. Each lab uses LLM agents for planning, training, and peer review, operating under a citation-based economy. The system demonstrates the ability to find high-accuracy, low-parameter-count neural operator designs across five PDE benchmark problems, outperforming rule-based alternatives in maintaining architectural diversity.
Why it matters: This work represents a significant advance in automating scientific discovery using LLM-driven agentic systems, highlighting the potential for AI to autonomously innovate in complex domains.
Airbnb has developed and deployed Proximity Features, a privacy-compliant system that addresses the cold-start personalization problem by grouping users based on geographic proximity using geo-IP data and adaptive clustering. This approach aggregates signals for groups of around 1,000 users, avoiding the need for persistent individual identifiers and operating within consent-gated privacy controls. Online A/B tests in production show statistically significant increases in bookings, particularly for users lacking recent or any history.
Why it matters: This work provides a scalable, privacy-preserving solution to cold-start personalization with demonstrated real-world impact at a major online platform.
Researchers introduce LiteTopK, a fused GPU kernel designed to accelerate Indexer-TopK operations in long-context sparse attention by leveraging the curse of dimensionality. LiteTopK samples data to estimate score ranges and partitions candidates into bins, reducing memory traffic and overhead while maintaining exact top-k correctness. Experiments show a 1.2x speedup in the prefill stage of GLM 5.2 during real-world deployment, along with lower memory usage.
Why it matters: This work offers a practical advance in the efficiency of sparse attention for large language models, enabling faster and more memory-efficient processing of long contexts.
A new preprint demonstrates that up to 77% of features recovered by sparse autoencoders (SAEs) with high cosine similarity to ground-truth directions are causally inert, meaning the matched atom does not activate when the feature is present. The authors introduce sae-causal-audit, a model-agnostic tool for causal validation, and identify two types of inertness: structural (due to antipodal-pair geometry) and competitive (arising from TopK pathologies in degraded SAEs).
Why it matters: This work challenges the use of correlational metrics alone for evaluating SAE interpretability, showing that high recovery scores can mask a lack of causal relevance, which is important for mechanistic interpretability research.
A new audit of six KV-cache compression methods finds that their performance rankings reverse when compression is performed before seeing the query (query-agnostic) compared to after (query-aware). In the more realistic query-agnostic setting, only KeyDiff consistently outperforms trivial baselines, while the widely used SnapKV method underperforms a simple 'keep start and recent window' baseline.
Why it matters: This study challenges the validity of current evaluation protocols for KV-cache compression, suggesting that some popular methods may be less effective in real-world LLM deployments than previously thought.
A new study demonstrates that applying Principal Component Analysis (PCA) to compress query embeddings can enhance retrieval performance in specialized domains. The approach was tested across 9 dense retrievers and 14 MTEB datasets, showing improved NDCG@10 scores in 75.4% of model-dataset pairs compared to standard embeddings. This method preserves domain-relevant features while removing non-discriminative components, providing a lightweight, training-free alternative to traditional domain adaptation methods.
Why it matters: This technique offers a simple and efficient way to improve dense retrieval in specialized domains without the need for costly annotation or retraining.
A new method equips speculative execution in LLM agents with three online memory systems—a contrastive transition table, episodic memory, and confusion tracker—to improve prediction quality by learning from past agent trajectories. The approach achieves 19–39% relative accuracy improvement on action prediction and up to 2.5× increase on observation prediction tasks, with gains increasing as memory accumulates. All speculation occurs during idle time, resulting in no added wall-clock cost and preserving identical agent trajectories compared to non-speculative execution.
Why it matters: This work demonstrates a way for LLM agents to learn from experience and improve efficiency without sacrificing correctness, potentially advancing the deployment of faster and more capable agentic systems.