{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d5fd8520-0bf9-5b08-a6b2-905587f57956","task_key":"default--test--2015","task_revision_id":"1","upstream_id":"2015","short_description":"Certain cross-cultural studies have suggested that six facial expressions are…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Happiness, sadness, desire, repulsion, fear, and surprise', 'Joy, sorrow, fear, rage, astonishment, and repugnance', 'Happiness, sadness, anger, loathing, lust, and surprise', 'Elation, melancholy, panic, ire, stupefaction, and detestation', 'Glee, gloom, terror, outrage, amazement, and abhorrence', 'Happiness, sadness, fright, anger, shock, and revulsion', 'Happiness, sadness, indifference, fright, surprise, and dislike', 'Delight, despair, dread, fury, wonder, and aversion', 'Happiness, sadness, fear, wrath, startle, and repulse', 'Happiness, sadness, fear, anger, surprise, and disgust']\",\"question\":\"Certain cross-cultural studies have suggested that six facial expressions are recognized by people of virtually every culture. Which of the following correctly lists these expressions?\",\"src\":\"ori_mmlu-high_school_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}