{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7d38e684-2ea6-5f65-85c3-bf798f8883b7","task_key":"default--test--10203","task_revision_id":"1","upstream_id":"10203","short_description":"A number (2N + 1) of sheets of material are stacked together, each with axis…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['I_f = (I_o * sqrt(2))', 'I_f = 2I_o', 'I_f = I_o', 'I_f = 4I_o', 'I_f = (I_o / sqrt(2))', 'I_f = (I_o * sqrt(3) / 2)', 'I_f = (3I_o / 4)', 'I_f = (I_o / 4)', 'I_f = (I_o / 8)', 'I_f = (I_o / 2)']\",\"question\":\"A number (2N + 1) of sheets of material are stacked together, each with axis rotated by (\\\\pi / 4N) with respect to the previous one. The first and last arepolarizers, the remainder are half-wave plates. Find the ratio of final to incident light intensity.\",\"src\":\"stemez-Optics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}