{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8e99fadf-48b1-5039-96a4-dd1690fc406c","task_key":"default--test--9933","task_revision_id":"1","upstream_id":"9933","short_description":"A man stands in an elevator as it begins to ascend. Does the normal force from…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['Yes, and the work done will be positive.', 'No, because the normal force is a conservative force.', \\\"No, because the normal force and the man's displacement are perpendicular.\\\", 'No, because the man is not moving relative to the elevator.', 'No.', 'Yes, but only if the elevator is accelerating.', \\\"Yes, but the amount of work done can't be determined.\\\", 'Yes, and the work done will be negative.', 'Yes, but the sign can’t be determined.', 'Yes, and the work done is zero.']\",\"question\":\"A man stands in an elevator as it begins to ascend. Does the normal force from the floor do work on the man?\",\"src\":\"ori_mmlu-high_school_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}