{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c9d85014-8f91-5f5e-b1c3-3334eb47a2f0","task_key":"default--test--6707","task_revision_id":"1","upstream_id":"6707","short_description":"Toxicity and dose-response data from animal studies, together with data of which…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['The price of the food to which the substance is added', 'The concentration of the substance in food', 'The amount of the substance that is excreted by the human body', 'The amount of the substance that is naturally present in the human body', 'The quantity of the substance sold in supermarkets', 'The yearly intake of the substance by animals consuming the food', 'The daily intake of the substance by humans consuming the food', 'The total amount of the substance a person will consume over a lifetime']\",\"question\":\"Toxicity and dose-response data from animal studies, together with data of which of the following types, are used to evaluate the safety of a food substance?\\n\",\"src\":\"ori_mmlu-nutrition\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}