{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9ef1f5fe-7f53-53b2-bdc0-ac588264d5dc","task_key":"default--test--4876","task_revision_id":"1","upstream_id":"4876","short_description":"The peak of Minoan civilization, from 3650 to 3420 B.P., was bracketed by what…","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['the introduction of Linear A script and the adoption of Linear B script', 'an earthquake and a volcanic eruption', 'the end of the Stone Age and the beginning of the Bronze Age', 'the discovery of bronze on the island, and the replacement of bronze by iron', 'the construction of the Palace of Knossos and its destruction', 'the establishment of the first maritime trade routes and the decline of these routes', 'the rise of the Pharaohs in Egypt and the fall of the Hittite Empire.', 'the invasion of Troy, and the surrender to the Romans', 'the establishment of the first Minoan Palace and the invasion of the Dorians', 'the founding of the Minos dynasty and the invasion of the Mycenaeans']\",\"question\":\"The peak of Minoan civilization, from 3650 to 3420 B.P., was bracketed by what two events?\",\"src\":\"ori_mmlu-prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}