{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c3ed9a8e-0401-5bfb-a858-2216aa934b0a","task_key":"default--test--11582","task_revision_id":"1","upstream_id":"11582","short_description":"An automobile battery is able to deliver 5.2 MJ of electrical energy to start a…","config":"default","split":"test","body":"{\"category\":\"engineering\",\"options\":\"['0.2276 m^3', '0.1500 m³', '0.3500 m³', '0.2567 m^3', '0.1750 m³', '0.4000 m³', '0.1892 m^3', '0.2100 m³', '0.2750 m³', '0.3124 m^3']\",\"question\":\"An automobile battery is able to deliver 5.2 MJ of electrical energy to start a car. Suppose compressed air at 7MPa, 25°C were to be used for doing the same work. What will be the volume of the tank required in order for the compressed air to have an availability of 5.2 MJ?\",\"src\":\"stemez-Thermodynamics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}