{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"72cbad13-0b31-5631-904e-fc4cbfb9c852","task_key":"default--test--10360","task_revision_id":"1","upstream_id":"10360","short_description":"Two expressions E and F are said to be unifiable if there are substitutions for…","config":"default","split":"test","body":"{\"category\":\"computer science\",\"options\":\"['(I, III) and (II, III) only', '(II, III) only', '(I, II) and (I, III) only', '(I, II) only', 'Only single expressions are unifiable, not pairs', 'None of the pairs are unifiable', '(I, II), (I, III), and (II, III)', 'All three expressions are unifiable', '(I, II) and (II, III) only', '(I, III) only']\",\"question\":\"Two expressions E and F are said to be unifiable if there are substitutions for the variables of E and F that make the expressions lexically identical. In the following three expressions, only w, x, y, and z are variables.\\nI. f(w,w)\\nII. f(x,1)\\nIII. f(y,g(z))\\nWhich pairs of these expressions is (are) pairs of unifiable expressions?\",\"src\":\"ori_mmlu-college_computer_science\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}