{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"070538ca-7bf2-560a-95c3-d22ec8221f8c","task_key":"default--test--10626","task_revision_id":"1","upstream_id":"10626","short_description":"Two computers communicate with each other by sending data packets across a local…","config":"default","split":"test","body":"{\"category\":\"computer science\",\"options\":\"['10,000 bytes/second', '200,000 bytes/second', '750,000 bytes/second', '50,000 bytes/second', '1,000,000 bytes/second', '25,000 bytes/ second', '2,500,000 bytes/second', '100,000 bytes/ second', '5,000,000 bytes/second', '500,000 bytes/ second']\",\"question\":\"Two computers communicate with each other by sending data packets across a local area network. The size of these packets is 1,000 bytes. The network has the capacity to carry 1,000 packets per second. The CPU time required to execute the network protocol to send one packet is 10 milliseconds. The maximum rate at which one computer can send data to another is approximately\",\"src\":\"ori_mmlu-college_computer_science\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}