{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3b91c90c-18d1-5a76-a023-c5ce92f49ad1","task_key":"default--test--10042","task_revision_id":"1","upstream_id":"10042","short_description":"A person can stand outside on a cold day for hours without ill effect, but…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['Water has more molecules per volume than air, increasing molecular contact with the person.', 'Thermal energy moves from high concentration areas (hot) to low concentration areas (cold).', 'The molecules of the person are, on average, moving faster than those of the surroundings.', \\\"The person's body is more resistant to cold in air than in water.\\\", 'Water has less heat capacity than air, causing a quicker heat loss.', 'As heat flows out of the person and warms the fluid surrounding the person, the warmer fluid rises, allowing fresh cool fluid to come in contact with the person and increasing the rate of heat transfer.', 'The person is more buoyant in water, causing a faster heat transfer.', \\\"The person's clothing provides more insulation against cold air than against cold water.\\\", 'Water molecules are polar, increasing heat transfer when in contact with the person.', 'Air is less dense than water, so heat transfer is slower in air.']\",\"question\":\"A person can stand outside on a cold day for hours without ill effect, but falling into a cold lake can kill a person in a matter of minutes. Which of the following is the primary reason for this phenomenon?\",\"src\":\"ori_mmlu-high_school_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}