{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"83f91742-a4b2-543b-a2a4-69cd584c727b","task_key":"default--test--2650","task_revision_id":"1","upstream_id":"2650","short_description":"According to attribution theorists, an error observers make in formulating…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['overattributing to person-situation interactions', 'overattributing to personal characteristics', 'overattributing to situations', 'overattributing to unpredictable events', 'underattributing to predictable events', 'underattributing to persons', 'underattributing to internal factors', 'underattributing to situations', 'underattributing to external environment factors', 'overattributing to external factors']\",\"question\":\"According to attribution theorists, an error observers make in formulating causal attributions about the behavior of others is\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}