LLM-based essay scoring shows robust cross-prompt generalization but systematic first-language bias

Jul 17, 2026

A preprint study evaluated a LoRA-adapted Gemma-3-27B-it language model on the full TOEFL11 essay corpus, achieving 77.79% band agreement and demonstrating strong cross-prompt generalization across eight unseen prompts. Despite this, the model showed a systematic scoring bias, consistently awarding higher scores to essays from European-language backgrounds compared to East-Asian-language backgrounds within each proficiency band.

Why it matters: This is the first large-scale fairness analysis of a fine-tuned open-weight LLM for automated essay scoring, highlighting a first-language bias with potential implications for educational assessment fairness.

Full story at: arXiv Computation and Language ↗