{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f038e9f4-edc6-51cf-8523-8f042e32bcd0","task_key":"default--test--5872","task_revision_id":"1","upstream_id":"5872","short_description":"A major car accident occurs. One man stops his car and starts helping people.…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['Self-fulfilling Prophecy', 'Social Identity', 'Cognitive Dissonance', 'Convergence', 'Emergent-norm', 'Deindividuation', 'Value-added', 'Social Facilitation', 'Bystander Effect', 'Contagion']\",\"question\":\"A major car accident occurs. One man stops his car and starts helping people. After seeing him stop, others also stop and start helping. Which of the following theories best explains the scenario?\",\"src\":\"ori_mmlu-sociology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}