Researchers introduce DAG-FM, a novel foundation model for causal discovery that decomposes the process into two auto-regressive stages using specialized Transformer-based modules. The model incorporates a Mixture-of-Leaf-Experts approach to adapt to diverse causal mechanisms and demonstrates state-of-the-art performance on both synthetic and real-world datasets, outperforming existing classical and foundation model approaches.
Why it matters: This work represents a significant advance in scalable causal discovery, enabling more accurate inference of cause-effect relationships from complex observational data.
A new preprint audits distributional reinforcement learning (RL) agents—specifically QR-DQN, C51, and IQN—and finds that 40-95% of their strongest risk trade-off claims are statistically refuted at 95% confidence. The study shows that the 'risk' learned by these agents is largely a training artifact, not a reflection of true environment stochasticity, and that following the agents' risk-based advice can sometimes lead to outcomes worse than random choice. The authors provide a statistical toolkit for auditing such claims and highlight pitfalls that can lead to misleading audit results.
Why it matters: This work raises significant concerns about the reliability of risk estimates from distributional RL agents, which are increasingly considered for safety-critical applications.
Researchers introduce WSqD, a new learning rate schedule that replaces the constant phase of the Warmup-Stable-Decay (WSD) schedule with a shifted inverse-square-root base, making it independent of the training horizon. The method achieves minimax-optimal convergence in stochastic convex optimization and, in language model pretraining experiments, matches or outperforms tuned WSD across different training horizons using a single peak learning rate.
Why it matters: WSqD could reduce the need for repeated learning rate tuning when extending training, potentially saving computational resources in large-scale model training.
A new preprint provides theoretical analysis showing that diffusion models such as DDIM and DDPM can efficiently adapt to data with unknown low-dimensional structure. The authors prove that, under exact score functions, the number of sampling iterations needed to achieve a given total variation distance scales with the intrinsic dimension of the data, rather than the ambient dimension. The results are extended to cases where the score function is learned from data, with kernel-based estimators shown to maintain this adaptivity under certain conditions.
Why it matters: This work offers the first rigorous theoretical evidence that diffusion models can efficiently sample from high-dimensional data with low intrinsic dimension, supporting and explaining their empirical success.
Researchers introduce MAD-Path, a new Markov Chain Monte Carlo (MCMC) method that samples from multimodal distributions by interpolating along diffusion paths rather than using traditional tempering. This approach preserves the relative weights of modes and improves mixing, with a Metropolis adjustment ensuring the target distribution remains invariant even with approximate intermediate scores. Experimental results demonstrate that MAD-Path achieves better exploration and more accurate mode-weight estimation compared to tempering-based MCMC methods.
Why it matters: MAD-Path addresses a longstanding challenge in statistical inference by providing a more reliable and principled method for sampling from complex, multimodal Bayesian posteriors.
A new study introduces two novel interaction building blocks for machine learning interatomic potentials: the Edge Complex Product Basis and Radial Rotary Complex Attention. The proposed model, TECE-OAM-RRA-1.0, demonstrates state-of-the-art performance on the Matbench Discovery benchmark, surpassing previous methods. The work also presents improvements to the Atomic Cluster Expansion module and evaluates the approach on multiple datasets.
Why it matters: Improved accuracy in machine learning interatomic potentials can significantly enhance materials science simulations and discovery.