{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"04387e49-4001-5388-8a2f-e4559f1872b8","task_key":"default--test--4756","task_revision_id":"1","upstream_id":"4756","short_description":"How did Aurignacian technology differ from Mousterian technology?","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['Mousterian technology was focused on the production of scrapers.', 'Aurignacian technology was primarily used for creating stone tools.', 'Aurignacian technology was less advanced than Mousterian technology.', 'Mousterian technology produced more usable blade surface.', 'Aurignacian technology produced more usable blade surface.', 'Mousterian technology produced less usable blade surface.', 'Mousterian technology was mainly used for creating hand axes.', 'Mousterian technology produced more projectile points.']\",\"question\":\"How did Aurignacian technology differ from Mousterian technology?\",\"src\":\"ori_mmlu-prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}