{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"65f6eeb5-353c-5e6e-85d2-a55d655ee91e","task_key":"default--test--10810","task_revision_id":"1","upstream_id":"10810","short_description":"According to Epicurus, the wealth required for a good life is:","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['easily procured.', 'available only to the educated.', 'available only to the rich.', 'obtained only through the accumulation of material possessions.', 'achievable only through constant labor and struggle.', 'attainable for everyone, but only through great effort.', 'gained only through inheritance or luck.', 'not necessary for a good life.', 'available only to the gods.', 'requiring a lifetime of pursuit.']\",\"question\":\"According to Epicurus, the wealth required for a good life is:\",\"src\":\"ori_mmlu-philosophy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}