{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"74cc025f-824d-55b4-89b1-6c2e92d1b31b","task_key":"default--test--9566","task_revision_id":"1","upstream_id":"9566","short_description":"Two lenses, A and B, have equal focal lengths but A has twice the diameter of B.…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['1:4, 4:1, 1:16', '1:1, 1:1, 1:1', '2:1, 4:1, 1:1', '4:1, 2:1, 1:16', '4:1, 1:4, 16:1', '2:1, 1:2, 1:2', '2:1, 1:2, 4:1', '4:1, 4:1, 16:1', '1:2, 2:1, 1:4', '8:1, 1:8, 64:1']\",\"question\":\"Two lenses, A and B, have equal focal lengths but A has twice the diameter of B. (a) What is the ratio of energy collected by lens A to that collected by lens B? (b) What is the corresponding ratio of the areas of the diffraction patterns? (c) What is the ratio of the energy per unit area in the two patterns?\",\"src\":\"stemez-Optics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}