{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"69ddc931-4cb1-5ee1-b3c8-dc96bb6f86a6","task_key":"default--test--5752","task_revision_id":"1","upstream_id":"5752","short_description":"Consider the following statements: (1) As countries get richer, the population…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['The first statement is unproven, the second is true', 'Neither statement has been definitively proven true or false', 'False, False', 'Both statements are partially true', 'The first statement is partially true, the second is false', 'The first statement is true, the second is unproven', 'True, False', 'True, True', 'False, True', 'The first statement is false, the second is partially true']\",\"question\":\"Consider the following statements: (1) As countries get richer, the population tends to report higher average life satisfaction. (2) Important life events such as marriage or divorce do affect our happiness, but have surprisingly little long-term impact. Which of these statements is True/False, as of 2020?\",\"src\":\"ori_mmlu-global_facts\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}