{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"780cfdb1-737f-5d54-bc35-6c2c2f96e827","task_key":"default--test--6016","task_revision_id":"1","upstream_id":"6016","short_description":"The docility hypothesis is that","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Those who experience low environmental demands become less docile', 'Those with less capability are less vulnerable to environmental demands', 'Those with low capabilities are more vulnerable to environmental demands', 'Those with high capabilities become less vulnerable to environmental demands', 'Those with high capabilities are more vulnerable to environmental demands', 'Those who experience high environmental demands become less docile', 'Those with high capability become less docile in low environmental demands', 'Those who experience low environmental demands become more docile']\",\"question\":\"The docility hypothesis is that\",\"src\":\"ori_mmlu-human_aging\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}