{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1143c80e-fc15-59a0-9b7e-558fb644dbd1","task_key":"default--test--1572","task_revision_id":"1","upstream_id":"1572","short_description":"Hart argues that secondary rules are 'power-conferring' but this seems…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['It is not a social rule.', 'It both confers power and imposes duties on judges to decide cases.', 'It is a primary rule, not a secondary rule.', 'It is applicable only in the case of federal constitutions.', 'It fails to specify the source of authority.', 'It does not confer power but only recognizes it.', 'It applies only in the case of unitary constitutions.', 'It is more of a procedural rule than a power-conferring rule.', 'It only imposes duties on judges without conferring power.', 'It confers power on the legislature, not the judiciary.']\",\"question\":\"Hart argues that secondary rules are 'power-conferring' but this seems unconvincing in the case of the rule of recognition because:\",\"src\":\"ori_mmlu-jurisprudence\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}