{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a559a42a-80d6-5174-bb6c-6cfd6190f946","task_key":"default--test--4936","task_revision_id":"1","upstream_id":"4936","short_description":"By analyzing the levels of 13C in bones, researchers can establish whether an…","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['a diet high in calcium or low in calcium.', 'a diet high in saturated fats or low in saturated fats.', 'mostly grains or mostly nuts and fruits.', 'mostly root vegetables or mostly leafy greens.', 'a diet rich in sugar or low in sugar.', 'a certain type of honey produced only in sub-Saharan Africa.', 'mostly meat or mostly vegetables.', 'mostly fish or mostly poultry.', 'no dairy or lots of dairy.', 'mostly processed foods or mostly organic foods.']\",\"question\":\"By analyzing the levels of 13C in bones, researchers can establish whether an individual ate:\",\"src\":\"ori_mmlu-prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}