{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c63be599-e5f1-5f3e-80f7-cc8c98cb2017","task_key":"default--test--6404","task_revision_id":"1","upstream_id":"6404","short_description":"With respect to marital satisfaction, most long lasting marriages","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Show lower levels during the years when children are present', 'Show constant low satisfaction throughout the marriage', 'Show high satisfaction through all the years of marriage', \\\"Show a steady increase in satisfaction throughout the marriage, regardless of children's presence\\\", 'Show ups and downs throughout the marriage', 'Show low satisfaction in the early years, which increases as the years pass', 'Show high satisfaction only during the years when children are present', 'Show high satisfaction that steadily declines as the years pass', 'Show high satisfaction only after children have left the home', 'Show high satisfaction in the early years, which gradually increases over time']\",\"question\":\"With respect to marital satisfaction, most long lasting marriages\",\"src\":\"ori_mmlu-human_aging\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}