{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3b0bf7bb-2148-51d9-85b5-1cae85968086","task_key":"default--test--1663","task_revision_id":"1","upstream_id":"1663","short_description":"What is the relationship between the formal sources of international law?","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['Treaties supersede custom', 'Treaties and General Principles of Law are of equal importance, but both supersede custom', 'General Principles of Law supersede both custom and treaties', 'Judicial decisions and teachings of the most highly qualified publicists are the only sources of international law.', 'Judicial decisions and teachings of the most highly qualified publicists supersede both custom and treaties', 'Custom and General Principles of Law are of equal importance, but both supersede treaties', 'There is a hierarchy, but it varies from case to case', 'Custom supersedes treaties', 'Treaties and custom are of equal importance, but General Principles of Law supersede both', 'There is no hierarchy between the formal sources of international law']\",\"question\":\"What is the relationship between the formal sources of international law?\",\"src\":\"ori_mmlu-international_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}