{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4be8dad6-5453-5754-acbd-c16e9e000eea","task_key":"default--test--8573","task_revision_id":"1","upstream_id":"8573","short_description":"Suppose V is a real vector space of finite dimension n. Call the set of matrices…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['I and III only', 'II, III and not I', 'III only', 'I, II and not III', 'I and II only', 'All are true (I, II, and III)', 'I only', 'II only', 'II and III only', 'None are true']\",\"question\":\"Suppose V is a real vector space of finite dimension n. Call the set of matrices from V into itself M(V). Let T be in M(V). Consider the two subspaces U = {X ∈ M(V) : TX = XT} and W = {TX − XT : X ∈ M(V)}. Which of the following must be TRUE?\\nI. If V has a basis containing only eigenvectors of T then U = M(V).\\nII. dim (U) + dim (W) = n^2\\nIII. dim (U) < n\",\"src\":\"ori_mmlu-college_mathematics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}