{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7aa5e91d-4994-52a6-bde3-dadd10b2f0d7","task_key":"default--test--3681","task_revision_id":"1","upstream_id":"3681","short_description":"A chemist expands an ideal gas against a constant ex-ternal pressure of 700…","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['-1.25 Kcal', '1.55 Kcal', '3.00 Kcal', '-2.10 Kcal', '-1.55 Kcal', '0.75 Kcal', '1.00 Kcal', '2.23 Kcal', '- .68 Kcal', '-0.50 Kcal']\",\"question\":\"A chemist expands an ideal gas against a constant ex-ternal pressure of 700 mmHg, and finds its volume changes from 50 to 150 liters. He finds that 1.55 Kcal of heat have been absorbed in the process. Determine the internal energy change that took place. 24.217 cal = 1 liter-atm.\",\"src\":\"stemez-Chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}