{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e054596a-0186-550d-b29e-b3299a79f7ef","task_key":"default--test--9532","task_revision_id":"1","upstream_id":"9532","short_description":"Which of the following correctly describes the motion of a real object in free…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['It will fall with constant speed and constant acceleration.', 'It will fall with constant speed and decreasing acceleration.', 'It will fall with decreasing speed and constant acceleration.', 'It will fall with constant speed and increasing acceleration.', 'It will fall with increasing speed and decreasing acceleration.', 'It will fall with decreasing speed and decreasing acceleration.', 'It will fall with increasing speed and increasing acceleration.', 'It will fall with no speed and no acceleration.', 'It will fall with decreasing speed and increasing acceleration.', 'It will fall with increasing speed and constant acceleration.']\",\"question\":\"Which of the following correctly describes the motion of a real object in free fall? Assume that the object experiences drag force proportional to speed and that it strikes the ground before reaching terminal sped.\",\"src\":\"ori_mmlu-high_school_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}