{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9125e6b0-9c8c-59e8-8ee4-613694dd1d42","task_key":"default--test--10657","task_revision_id":"1","upstream_id":"10657","short_description":"Assume that any assignment statement can be executed in unit time. If as many…","config":"default","split":"test","body":"{\"category\":\"computer science\",\"options\":\"['4', '10', '9', '7', '3', '2', '1', '8', '6', '5']\",\"question\":\"Assume that any assignment statement can be executed in unit time. If as many identical processors as needed are used, what is the minimum number of time units needed to execute the assignments\\nA := B + C\\nB := A - B\\nC := A * E\\nD := A/F\\nE := B - C\\nF := A + B\\nwith the same result as if the assignments were executed in the order shown?\",\"src\":\"ori_mmlu-college_computer_science\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}