{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"86924186-17bc-55ce-8366-84cdd2e09134","task_key":"default--test--2462","task_revision_id":"1","upstream_id":"2462","short_description":"Which of the following best summarizes why people tend to stay in a slightly…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Their neurons continue to fire at a rapid pace, keeping them in a state of high alert.', 'Their neurons remain in a state of graded potentiality even after they have fired.', 'Their heart rate remains elevated due to the stress of the crisis.', 'Their muscles remain tense due to the physical exertion of the crisis.', 'Their pituitary gland continues to release stress hormones.', 'Their bloodstream continues to contain elevated levels of adrenaline.']\",\"question\":\"Which of the following best summarizes why people tend to stay in a slightly elevated state of arousal after a crisis has occurred?\",\"src\":\"ori_mmlu-high_school_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}