{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"dee7e5da-8290-5e48-bb12-5b7318473d03","task_key":"default--test--9907","task_revision_id":"1","upstream_id":"9907","short_description":"Starting from rest, an engine at full throttle pulls a freight train of mass…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['2400 slugs', '2800 slugs', '1200 slugs', '1400 slugs', '2000 slugs', '2600 slugs', '1000 slugs', '2200 slugs', '1800 slugs', '1600 slugs']\",\"question\":\"Starting from rest, an engine at full throttle pulls a freight train of mass 4200 slugs along a level track. After 5 min, the train reaches a speed of 5 mph. After it has picked up more freight cars, it takes 10 min to ac-quire a speed of 7 mph. What was the mass of the added freight cars? Assume that no slipping occurs and that frictional forces are the same in both cases.\",\"src\":\"stemez-Physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}