Massive Multitask Language Understanding (MMLU) dataset

The MMLU dataset is a benchmark for measuring the behavior of large language models on a number of tasks. It consists of 15908 multiple choice questions distributed across 57 subject areas.

Data and Resources

Cite this as

Mark Steyvers, Heliodoro Tejeda, Aakriti Kumar, Catarina Belem, Sheer Karny, Xinyue Hu, Lukas Mayer, Padhraic Smyth (2025). Dataset: Massive Multitask Language Understanding (MMLU) dataset. https://doi.org/10.57702/mcam7zcl

DOI retrieved: January 2, 2025

Additional Info

Field Value
Created January 2, 2025
Last update January 2, 2025
Defined In https://doi.org/10.48550/arXiv.2406.11634
Citation
  • https://doi.org/10.48550/arXiv.2401.13835
Author Mark Steyvers
More Authors
Heliodoro Tejeda
Aakriti Kumar
Catarina Belem
Sheer Karny
Xinyue Hu
Lukas Mayer
Padhraic Smyth
Homepage https://huggingface.co/datasets/Massive-Multitask-Language-Understanding