{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"155d3705-817c-5242-bac3-f376a0a9b443","task_key":"default--test--4816","task_revision_id":"1","upstream_id":"4816","short_description":"What distinguishes the trade patterns of the Mesolithic from those of the Upper…","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['in the Mesolithic, goods traveled less far and fewer goods were exchanged', 'in the Mesolithic, goods traveled farther but the same number of goods were exchanged', 'in the Mesolithic, goods traveled less far but the same number of goods were exchanged', 'in the Mesolithic, goods traveled the same distance but more goods were exchanged', 'in the Mesolithic, goods traveled farther but fewer goods were exchanged', 'in the Upper Paleolithic, goods traveled less far but more goods were exchanged', 'in the Mesolithic, goods traveled the same distance but fewer goods were exchanged', 'in the Mesolithic, goods traveled less far but more goods were exchanged', 'in the Mesolithic, goods traveled farther and more goods were exchanged', 'in the Mesolithic, goods traveled the same distance and the same number of goods were exchanged']\",\"question\":\"What distinguishes the trade patterns of the Mesolithic from those of the Upper Paleolithic?\",\"src\":\"ori_mmlu-prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}