{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e7065585-85bd-55c1-a168-89fd214bc053","task_key":"default--test--10852","task_revision_id":"1","upstream_id":"10852","short_description":"According to Rawls, the two parts of his theory:","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['can be proven false with logical reasoning.', 'are independent: You can accept either part without accepting the other.', 'are based purely on empirical evidence.', 'are not open to debate or disagreement.', 'cannot be justified via rational means.', 'can only be justified through emotional appeal.', 'are based on personal preferences and biases.', 'are dependent on each other, but can be accepted separately.', 'are interdependent: You cannot accept one part without accepting the other.', 'are provable from self-evident premises.']\",\"question\":\"According to Rawls, the two parts of his theory:\",\"src\":\"ori_mmlu-philosophy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}