{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"78268dc9-5d5a-5a99-a0b1-f3f59830637f","task_key":"default--test--4865","task_revision_id":"1","upstream_id":"4865","short_description":"Archaeological evidence for the domestication of cats suggests it occurred when…","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['after 11,000 years ago, as an unintentional consequence of food production and storage', 'before 15,000 years ago, as an unintentional result of the cultivation of crops', 'before 22,000 years ago, as a result of the shift from opportunistic foraging to logistical collecting', 'before 13,000 years ago, due to intentional breeding for pest control purposes', 'before 11,000 years ago, as a direct result of artificial selection', 'after 14,000 years ago, as an unintentional consequence of the domestication of birds', 'after 12,000 years ago, due to intentional breeding for hunting purposes', 'before 10,000 years ago, as a direct result of artificial selection for companionship', 'after 9,000 years ago, as an unintentional result of the domestication of cattle and production of milk and cheese', 'after 7,000 years ago, as a direct result of the domestication of dogs']\",\"question\":\"Archaeological evidence for the domestication of cats suggests it occurred when and why?\",\"src\":\"ori_mmlu-prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}