{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2c40ba80-e416-56c5-b941-56cd51962e44","task_key":"default--test--6907","task_revision_id":"1","upstream_id":"6907","short_description":"When opportunity cost is constant across all production levels, the productions…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['an irregular zig-zag line.', 'concave to the origin.', 'a straight horizontal line.', 'convex to the origin.', 'a curved line sloping upward from left to right.', 'a straight vertical line.', 'a straight diagonal line sloping upward from left to right.', 'a straight diagonal line sloping downward from right to left.', 'a straight diagonal line sloping downward from left to right.', 'undefined.']\",\"question\":\"When opportunity cost is constant across all production levels, the productions possibilities frontier is\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}