Building a European Multilingual Evaluation Dataset: The MMLU Localisation Project within the EMT Network
A collaboration between the European Commission's Directorate-General for Translation and the European Master's in Translation network has localized the MMLU dataset into 11 European languages. The project not only aims to create a more inclusive benchmark for evaluating large language models (LLMs), but also provides master's students with authentic, project-based professional training in translation and related skills.
Why it matters: This initiative addresses the lack of multilingual evaluation benchmarks for LLMs in European languages and demonstrates a novel approach to dataset creation through educational partnerships.
Full story at: arXiv Computation and Language ↗