{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b97cfe98-f493-5eca-819c-ab90d1845cd8","task_key":"default--test--5153","task_revision_id":"1","upstream_id":"5153","short_description":"Becker proclaimed that cannabis use was:","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['a dangerous pastime that young people should not be allowed to pursue', 'a behavior that could be easily unlearned with proper guidance', 'a gateway drug that leads to use of harder substances', 'not as enjoyable as other sociologists had suggested', 'increasing throughout all sections of youth in the 1970s', 'decreasing in popularity among youth in the 1970s', 'learned gradually through the social processes of a deviant career', 'something that should be strictly regulated by the government', 'a healthy alternative to alcohol and other dangerous substances', 'a form of rebellion against societal norms']\",\"question\":\"Becker proclaimed that cannabis use was:\",\"src\":\"ori_mmlu-sociology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}