{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fbdf7515-797e-5a06-aa11-057161d825de","task_key":"default--test--9819","task_revision_id":"1","upstream_id":"9819","short_description":"A proton enters a solenoid. Upon entry, the proton is moving in a straight line…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['The proton will be bent in a circular path.', 'The proton will be bent in a parabolic path.', 'The proton will continue in its straight path and slow down.', 'The proton will be bent in a spiral path.', 'The proton will move in a zigzag pattern.', 'The proton will move in a straight path and speed up.', 'The proton will accelerate in its straight path.', 'The proton will oscillate back and forth along the axis of the solenoid.', 'The proton will continue in its straight path at constant velocity.', 'The proton will move in an elliptical path.']\",\"question\":\"A proton enters a solenoid. Upon entry, the proton is moving in a straight line along the axis of the solenoid. Which of the following is a correct description of the proton’s motion within the solenoid?\",\"src\":\"ori_mmlu-high_school_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}