{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0f5643e8-2387-56f1-bc79-936e6f85beba","task_key":"default--test--4710","task_revision_id":"1","upstream_id":"4710","short_description":"The appropriateness and usefulness of pedestrian survey depends on:","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['the presence of a nearby museum or storage facility.', 'naturally exposed layers in which to search for artifacts, ecofacts, or bone.', 'the accessibility of the site by public transportation.', 'the presence of primary and secondary refuse.', 'the existence of a cache or collection of artifacts.', 'the availability of an experienced archaeological team.', 'the likelihood of finding gold or other valuable materials.', 'associations that can be established between artifacts, ecofacts, and bone.', 'the weather conditions at the time of the survey.', 'the political stability of the region where the site is located.']\",\"question\":\"The appropriateness and usefulness of pedestrian survey depends on:\",\"src\":\"ori_mmlu-prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}