OpenLanguageModel: Readable and Composable Small-Language-Model Pretraining for Education and Research
OpenLanguageModel (OLM) is an open-source PyTorch library designed for building and pretraining small language models with readable and modular code. It offers 27 presets across nine model families, supports CPU, single-GPU, and multi-GPU execution, and demonstrates 90.6% four-GPU weak-scaling efficiency on a 348M-parameter workload. OLM is MIT-licensed and available via PyPI, GitHub, and its documentation site.
Why it matters: OLM lowers the barrier for education and research in language model pretraining by providing transparent, modular code that is accessible for both teaching and experimentation.
Full story at: arXiv Computation and Language ↗