{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"01684251-9218-51d6-9473-a8b5090ed761","task_key":"default--test--10482","task_revision_id":"1","upstream_id":"10482","short_description":"The IP protocol is primarily concerned with","config":"default","split":"test","body":"{\"category\":\"computer science\",\"options\":\"['Reliable delivery of large (multi-packet) messages between machines that are not necessarily directly connected', 'Providing a user interface for network management', 'Coordinating the hardware components of the network', 'Routing packets through the network', 'Controlling the power consumption of network devices', 'Reliable delivery of packets between directly connected machines', 'Ensuring compatibility between different network technologies', 'Monitoring the network traffic speed', 'Dealing with differences among operating system architectures', 'Managing the security of data transmission']\",\"question\":\"The IP protocol is primarily concerned with\",\"src\":\"ori_mmlu-college_computer_science\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}