{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6bfa1f9b-f62e-50e8-a0a9-c67561b8e5ea","task_key":"default--test--2684","task_revision_id":"1","upstream_id":"2684","short_description":"Consider the distribution of the previous problem. Let us add 2 relatively high…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Mean is unaffected, median is somewhat affected, mode is most affected', 'Median is most affected, mean is somewhat affected, mode is unaffected', 'All measures are unaffected', 'Mean and median are equally affected, mode is most affected', 'All measures are equally affected', 'Mean is most affected, median is somewhat affected, mode is unaffected', 'Median is unaffected, mean and mode are equally affected', 'Mode and median are equally affected, mean is unaffected', 'Mode is unaffected, mean is somewhat affected, median is most affected', 'Mode is most affected, median is somewhat affected, mean is unaffected']\",\"question\":\"Consider the distribution of the previous problem. Let us add 2 relatively high values in order to observe the differential effecton the three measures of central tendency. Use the followingeight ordered values: 3, 5, 6, 6, 7, 9, 16, 20.\",\"src\":\"stemez-Psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}