{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f02240ce-9625-5b77-961f-7811a3638b67","task_key":"default--test--4676","task_revision_id":"1","upstream_id":"4676","short_description":"Australian and America megafauna were probably wiped out by:","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['natural disasters such as volcanic eruptions or earthquakes.', 'both humans and environmental changes.', 'the spread of invasive plant species which altered their habitat.', 'a sudden drastic climate change due to global warming.', 'humans who carried diseases over the land bridge.', 'humans.', 'diseases transmitted by other animal species.', 'a comet or asteroid impact.', 'environmental changes.', 'competition with other animal species.']\",\"question\":\"Australian and America megafauna were probably wiped out by:\",\"src\":\"ori_mmlu-prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}