{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fb34006b-5a40-5285-8ac3-9b72890ec30a","task_key":"default--test--10325","task_revision_id":"1","upstream_id":"10325","short_description":"Two people, one of mass 100 kg and the other of mass 50 kg, stand facing each…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['the force felt by the heavier person is 2F', 'the magnitude of the acceleration of the heavier person will be half of the magnitude of the acceleration of the lighter person', 'the force felt by the lighter person is -2F', 'the magnitude of the acceleration of the lighter person will be half of the magnitude of the acceleration of the heavier person', 'the force felt by the person is -2F', 'the magnitude of the acceleration of the heavier person will be twice the magnitude of the acceleration of the lighter person', 'the force felt by the lighter person is (-1/2)F', 'the magnitude of the acceleration of the lighter person will be twice the magnitude of the acceleration of the heavier person', 'the force felt by the heavier person is (-1/2)F']\",\"question\":\"Two people, one of mass 100 kg and the other of mass 50 kg, stand facing each other on an ice-covered (essentially frictionless) pond. If the heavier person pushes on the lighter one with a force F, then\",\"src\":\"ori_mmlu-high_school_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}