{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7e40a229-08af-543b-be9b-a083297cb8bd","task_key":"default--test--3879","task_revision_id":"1","upstream_id":"3879","short_description":"How many ounces of silver alloy which is 28% silver must be mixed with 24 ounces…","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['48 ounces of silver', '42 ounces of silver', '54 ounces of silver', '72 ounces of silver', '60 ounces of silver', '36 ounces of silver', '18 ounces of silver', '30 ounces of silver', '24 ounces of silver', '12 ounces of silver']\",\"question\":\"How many ounces of silver alloy which is 28% silver must be mixed with 24 ounces of silver which is 8% silver to produce a new alloy which is 20% silver?\",\"src\":\"stemez-Chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}