{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"77b9d583-6e62-5657-8690-fe3e79f07a96","task_key":"default--test--2528","task_revision_id":"1","upstream_id":"2528","short_description":"The driver of a car that has pulled up next to you at a red light turns up the…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['increased secretions by the pituitary gland', 'changes in the atmospheric pressure', 'changes in the temperature of the air', 'neurons firing more intensely', 'a decrease in the distance between the sound source and your ear', 'increased frequency of the radio waves', 'increase in the speed of sound', 'changing frequency of sound waves', 'vibrations of the car windows', 'more neurons firing more frequently']\",\"question\":\"The driver of a car that has pulled up next to you at a red light turns up the volume on her radio. The increase in perceived sound that you experience is primarily due to\",\"src\":\"ori_mmlu-high_school_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}