{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"18365b69-951a-522b-8561-358114558ad7","task_key":"default--test--7651","task_revision_id":"1","upstream_id":"7651","short_description":"Which of the following statements accurately describes the relationship between…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['MP intersects AP at the minimum of MP.', 'AP rises when MP is above it and falls when MP is below it.', 'MP is always higher than AP regardless of the level of labor.', 'AP and MP are inversely related to each other.', 'AP falls when MP is above it and rises when MP is below it.', 'AP always remains constant regardless of the MP.', 'MP intersects AP at the maximum of MP.', 'AP and MP are either both rising or both falling at all levels of labor.', 'AP and MP intersect only once at the maximum level of labor.', 'AP and MP are always parallel to each other.']\",\"question\":\"Which of the following statements accurately describes the relationship between average product (AP) and marginal product (MP) of labor?\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}