Value Drifts: Tracing Value Alignment During LLM Post-Training

Jul 16, 2026

A new preprint investigates how large language models (LLMs) acquire and adjust to human values during post-training. The study finds that supervised fine-tuning (SFT) largely determines a model's value alignment, while subsequent preference optimization rarely changes these values significantly. Experiments with Llama-3 and Qwen-3 models further show that different preference optimization algorithms can result in different value alignment outcomes, even when using the same data.

Why it matters: Understanding when and how LLMs learn human values can guide better data curation and algorithm choices for improved model alignment.

Full story at: arXiv Computers and Society ↗