{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fce9c965-efe3-535d-a167-b061492a2c57","task_key":"default--test--7980","task_revision_id":"1","upstream_id":"7980","short_description":"Suppose V is a finite-dimensional vector space on F. $M1={a_1,a_2,a_3}$ is a…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['[2, 3, 4]', '[3, 4, 5]', '[3, 1, 4]', '[1, 3, 5]', '[0, 1, 2]', '[1, 2, 3]', '[6, 7, 8]', '[5, 6, 7]', '[4, 5, 6]', '[2, 4, 6]']\",\"question\":\"Suppose V is a finite-dimensional vector space on F. $M1={a_1,a_2,a_3}$ is a basis of V, $M2={b_1,b_2,b_3}$ is another basis of V. Suppose the coordinates of b_1,b_2,b_3 under M1 are $c_1=(1,1,-1),c_2=(1,-1,1),c_3=(-1,1,1)$. Suppose the coordinate of $d\\\\in V$ under M1 is (1,3,5). What is the coordinate of d under M2? Return the three coordinate values as a list.\",\"src\":\"theoremQA-Math\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}