{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c2548b44-65f2-5663-b9b2-1ce566fed189","task_key":"default--test--2063","task_revision_id":"1","upstream_id":"2063","short_description":"After World War II there was much speculation about the personalities of those…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Only those with a high level of education and indoctrination committed the atrocities', 'Only sociopaths would obey such harmful orders', 'The actions were a result of a widespread cultural acceptance of violence at the time', 'People with a predisposition to violence committed the atrocities', 'Atrocities were committed by those who misinterpreted orders due to language barriers', 'The atrocities were committed independently of any orders, based on personal beliefs', 'Ordinary people would comply with the orders of a legitimate authority even to the point of committing cruel and harmful actions', 'Only those with aggressive personalities committed atrocities', 'Individuals who committed atrocities did so only when under direct threat to their own lives', 'Only those with a history of criminal behavior were capable of committing the atrocities']\",\"question\":\"After World War II there was much speculation about the personalities of those who had committed atrocities. What do the findings of Milgram's studies imply about the personalities of Nazi war criminals?\",\"src\":\"stemez-Psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}