{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b4f275cc-16bb-57b4-a46f-d90a70156071","task_key":"default--test--2171","task_revision_id":"1","upstream_id":"2171","short_description":"Which of the following explanations of why a 17-year-old drives his car at or…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['\\\"I don\\\\'t want to waste gas.\\\"', '\\\"I\\\\'m not in a rush.\\\"', '\\\"I want to impress my friends.\\\"', '\\\"I find it relaxing to drive slowly.\\\"', '\\\"It\\\\'s the law.\\\"', '\\\"I want my parents to approve of my driving.\\\"', '\\\"I don\\\\'t want to get any tickets.\\\"', '\\\"I don\\\\'t want to crash my car.\\\"', '\\\"I\\\\'m afraid of going fast.\\\"', '\\\"I want to be a good example for my younger siblings.\\\"']\",\"question\":\"Which of the following explanations of why a 17-year-old drives his car at or below the speed limit best illustrates Kohlberg's conventional level of morality?\",\"src\":\"ori_mmlu-high_school_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}