{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f2fbb328-1447-5444-8246-a8a17b8b2675","task_key":"default--test--10133","task_revision_id":"1","upstream_id":"10133","short_description":"A barge of mass 15,000 kg, made from metal of density 7500 kg\\bulletm^-3, is…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['3.5 cm', '2 cm', '0.2 cm', '2.5 cm', '0.8 cm', '1.5 cm', '3 cm', '1 cm', '0.5 cm', '4 cm']\",\"question\":\"A barge of mass 15,000 kg, made from metal of density 7500 kg\\\\bulletm^-3, is being loaded in a closed dock of sur-face area 6633 m^2 with ore of density 3 g\\\\bulletcm^-3. When 80,000 kg of ore are aboard, the barge sinks. How does the water level in the dock alter? The area of the barge is assumed negligible in comparison with the area of the dock.\",\"src\":\"stemez-Physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}