{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"35ac3b56-f52f-5d29-b0c6-0caa5259cf65","task_key":"default--test--5705","task_revision_id":"1","upstream_id":"5705","short_description":"The true length of a block of wood is 1.010 cm. Three measurements of this block…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['They are somewhat precise and somewhat accurate.', 'They are exceptionally accurate but not precise.', 'They are precise but not accurate.', 'They are accurate and slightly precise.', 'They are precise and accurate.', 'They are neither precise nor accurate.', 'They are slightly precise but very accurate.', 'They are very precise but slightly accurate.', 'They are accurate but not precise.', 'They are exceptionally precise but not accurate.']\",\"question\":\"The true length of a block of wood is 1.010 cm. Three measurements of this block produced the following values: 1.4 cm, 1.2 cm, and 0.9 cm. Which of the following statements is true concerning these measurements?\",\"src\":\"ori_mmlu-miscellaneous\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}