{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"43c39a57-f9e9-5fba-a0f9-4f92f7a4e7c5","task_key":"default--test--9600","task_revision_id":"1","upstream_id":"9600","short_description":"A spaceship from earth enters a circular orbit 22,000 km above the surface of…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['4.8 km/s', '4.0 km/s', '3.5 km/s', '5.2 km/s', '7.1 km/s', '5.6 km/s', '6.5 km/s', '2.9 km/s', '6.0 km/s', '3.2 km/s']\",\"question\":\"A spaceship from earth enters a circular orbit 22,000 km above the surface of Mars at the equator, which allows it to rotate at the same speed as Mars and thus to stay always above the same point on the planet's surface. The Martians set up a guided missile directly beneath it and fire it vertically upward in an attempt to destroy the intruder. With what minimum velocity must the missile leave the surface in order to suc-ceed in its mission? Mars is a sphere of radius 3400 km and of mean density 4120 kg\\\\bulletm^-3.\",\"src\":\"stemez-Physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}