{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"01cc3248-da24-5c2a-97e9-f3f75167adbd","task_key":"default--test--10896","task_revision_id":"1","upstream_id":"10896","short_description":"Mill claims that the principle of utility:","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['can only be proven through extensive observation and study.', 'can be proven from principles that are known empirically.', 'can be proven, but only under certain conditions.', 'is inherently flawed and therefore cannot be proven.', 'is a theory that has been widely disproven.', 'can be proven, but this is unique to the principle of utility.', 'cannot be proven, but this is common to all first principles.', 'can be proven, but only by using a complex mathematical model.', 'cannot be proven, and this is a unique problem for the theory.', 'can be proven from self-evident principles.']\",\"question\":\"Mill claims that the principle of utility:\",\"src\":\"ori_mmlu-philosophy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}