{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bcd9046d-bbaf-5cae-abe6-4a540a95aabc","task_key":"default--test--5896","task_revision_id":"1","upstream_id":"5896","short_description":"Which of the following is the most reasonable action to take for an artist whose…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['Making sure no children are present when using the product', \\\"Reading the product's label and proceeding according to the label directions\\\", 'Only using the product sparingly and infrequently', 'Diluting the product with water to reduce its hazard', 'Checking with a qualified toxicologist before using the material', 'Ignoring the hazard and using the product as usual', 'Wearing a mask and gloves while using the product', 'Getting a friend to use the product instead', 'Using the product only in large, well-ventilated spaces', 'Finding a nontoxic product to use and adapting the art-making process as necessary']\",\"question\":\"Which of the following is the most reasonable action to take for an artist whose work requires the use of a specific hazardous product?\",\"src\":\"ori_mmlu-miscellaneous\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}