{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1ea0a3be-2040-54eb-9131-fbb34cbbde63","task_key":"default--test--9533","task_revision_id":"1","upstream_id":"9533","short_description":"If two non-zero vectors are added together, and the resultant vector is zero,…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['They have different magnitudes and are pointed in opposite directions.', 'They have zero magnitude.', 'It is not possible for the sum of two non-zero vectors to be zero.', 'The angle between them is 90 degrees.', 'They have equal magnitude and are pointed in opposite directions.', 'They have equal magnitude and are pointed in the same direction.', 'They are perpendicular to each other.', 'They have different magnitudes and are pointed in the same direction.', 'They are parallel to each other.']\",\"question\":\"If two non-zero vectors are added together, and the resultant vector is zero, what must be true of the two vectors?\",\"src\":\"ori_mmlu-high_school_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}