{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"09674c6a-b4aa-5544-af89-97188a1e0fcd","task_key":"default--test--11597","task_revision_id":"1","upstream_id":"11597","short_description":"A steam pipe containing saturated steam at 100 psia is connected to an empty…","config":"default","split":"test","body":"{\"category\":\"engineering\",\"options\":\"['The steam in the container is condensed', 'The steam in the container is saturated', 'The steam in the container is at the same temperature as the reservoir', 'The steam in the container is subcooled', 'The steam in the container is in a vacuum state', 'The steam in the container is partially condensed with a liquid phase present', 'The steam in the container is slightly superheated', 'The steam in the container is at a higher temperature than the pipe', 'The steam in the container is at a lower pressure than the pipe', 'The steam in the container is highly superheated']\",\"question\":\"A steam pipe containing saturated steam at 100 psia is connected to an empty cylinder. What will be the final state of the steam if the cylinder is filled with steam at the same pressure as the pipe?\",\"src\":\"stemez-Thermodynamics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}