{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f5e9a979-37a1-58dd-b656-c8996d8055f6","task_key":"default--test--726","task_revision_id":"1","upstream_id":"726","short_description":"Mary O' Grady recently worked 44 hours during a regular (Monday-Friday) week. In…","config":"default","split":"test","body":"{\"category\":\"business\",\"options\":\"['$210.35', '$187.90', '$194.25', '$182.13', '$170.50', '$165.80', '$160.00', '$150.00', '$200.00', '$176.45']\",\"question\":\"Mary O' Grady recently worked 44 hours during a regular (Monday-Friday) week. In addition, she worked 4(1/2) hours on Saturday and 3 hours on Sunday. Time and a half is paid for all hours worked over 40 during the regular week and for all Saturday work. Double time is paid for all hours worked on Sunday. What were her gross earnings for the week if her regular rate is $3.10 an hour?\",\"src\":\"stemez-Business\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}