{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1e1328ea-68f0-55a8-af6b-a0420a064539","task_key":"default--test--3807","task_revision_id":"1","upstream_id":"3807","short_description":"Nitrogen's electronegativity value is between those of phosphorus and oxygen.…","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['The value for nitrogen is less than that of phosphorus because nitrogen has a smaller atomic mass, but greater than that of oxygen because nitrogen has a larger atomic number.', 'The value for nitrogen is greater than that of phosphorus because nitrogen is smaller, but less than that of oxygen because nitrogen has a smaller effective nuclear charge.', 'The value for nitrogen is greater than that of phosphorus because nitrogen has a greater atomic number, but less than that of oxygen because nitrogen has fewer valence electrons.', 'The value for nitrogen is less than that of phosphorus because nitrogen is larger, but greater than that of oxygen because nitrogen has a greater effective nuclear charge.', 'The value for nitrogen is greater than that of phosphorus because nitrogen has less electrons, but less than that of oxygen because nitrogen is smaller.', 'The value for nitrogen is less than that of phosphorus because nitrogen has a smaller atomic radius, but greater than that of oxygen because nitrogen has a greater atomic number.', 'The value for nitrogen is greater than that of phosphorus because nitrogen has more valence electrons, but less than that of oxygen because nitrogen is larger.', 'The value for nitrogen is less than that of phosphorus because nitrogen has fewer protons but greater than that of oxygen because nitrogen has less valence electrons.', 'The value for nitrogen is less than that of phosphorus because nitrogen has fewer neutrons, but greater than that of oxygen because nitrogen has more protons.', 'The value for nitrogen is greater than that of phosphorus because nitrogen has a smaller atomic mass, but less than that of oxygen because nitrogen has fewer protons.']\",\"question\":\"Nitrogen's electronegativity value is between those of phosphorus and oxygen. Which of the following correctly describes the relationship between the three values?\",\"src\":\"ori_mmlu-high_school_chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}