{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5b3fa22c-9e3b-5557-a4df-53a9d8197b45","task_key":"default--test--3800","task_revision_id":"1","upstream_id":"3800","short_description":"Calculate the work done a) at 25°C, b) at 100°C, when 100 g of nitrogen at a…","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['a) 3500 cal, b) 4500 cal', 'a) 4000 cal, b) 5366 cal', 'a) 5000 cal, b) 5366 cal', 'a) 3500 cal, b) 6000 cal', 'a) 4287 cal, b) 6000 cal', 'a) 4500 cal, b) 5366 cal', 'a) 4000 cal, b) 5000 cal', 'a) 4287 cal, b) 5366 cal', 'a) 5000 cal, b) 6000 cal', 'a) 4287 cal, b) 5000 cal']\",\"question\":\"Calculate the work done a) at 25°C, b) at 100°C, when 100 g of nitrogen at a pressure of 760 torr is expanded reversibly and isothermally to a pressure of 100 torr.\",\"src\":\"stemez-PhysicalChemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}