{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d47b9b2f-9a59-5029-9e12-9f551dc8c943","task_key":"default--test--9431","task_revision_id":"1","upstream_id":"9431","short_description":"When the temperature of a strip of iron is increased, the length of the strip","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['increases at first, then decreases.', 'decreases in width as it gets longer.', 'increases exponentially.', 'remains the same.', 'may increase and may decrease.', 'decreases in thickness as it gets longer.', 'also increases.', 'becomes unpredictable.', 'oscillates between increasing and decreasing.', 'actually decreases.']\",\"question\":\"When the temperature of a strip of iron is increased, the length of the strip\",\"src\":\"ori_mmlu-conceptual_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}