{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c176b429-d453-5519-83c4-2c0aa838e77a","task_key":"default--test--7904","task_revision_id":"1","upstream_id":"7904","short_description":"To study the effect of music on piecework output at a clothing manufacturer, two…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['A third group that hears day-long R & B music', 'A third group that hears a mix of classical, light rock, and other genres of music', 'A third group that hears day-long country music', 'A third group that hears day-long pop music', 'A third group that hears day-long jazz music', 'A third group that hears classical music every morning and light rock every afternoon', 'A third group that hears different music each day from various genres', 'A third group that randomly hears either classical or light rock music each day', 'A third group for which no music is played', 'A third group that hears music for half the day']\",\"question\":\"To study the effect of music on piecework output at a clothing manufacturer, two experimental treatments are planned: day-long classical music for one group versus day-long light rock music for another. Which one of the following groups would serve best as a control for this study?\",\"src\":\"ori_mmlu-high_school_statistics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}