{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ed8cdd40-985a-561e-a39f-533fd82befe7","task_key":"default--test--1664","task_revision_id":"1","upstream_id":"1664","short_description":"What is the relationship between the formal sources of international law?","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['Custom and General Principles of Law are of equal importance, but both supersede treaties', 'Judicial decisions and teachings of the most highly qualified publicists supersede both custom and treaties', 'There is no hierarchy between the formal sources of international law', 'General Principles of Law supersede both custom and treaties', 'Treaties and General Principles of Law are of equal importance, but both supersede custom', 'Custom supersedes treaties', 'There is a hierarchy, but it varies from case to case', 'Treaties supersede custom', 'Judicial decisions and teachings of the most highly qualified publicists are the only sources of international law.', 'Treaties and custom are of equal importance, but General Principles of Law supersede both']\",\"question\":\"What is the relationship between the formal sources of international law? \",\"src\":\"ori_mmlu-international_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}