{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6d651ee0-5460-53d0-ac24-87213e36e385","task_key":"default--test--5304","task_revision_id":"1","upstream_id":"5304","short_description":"From 1988 to 1990 among heterosexuals in the US, the number of unmarried adults…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['declined for men, and remained the same for women', 'increased for men, but not for women', 'declined for both men and women', 'increased for men, and remained the same for women', 'increased for women, but not for men', 'remained the same for men, but declined for women', 'declined for men, but not for women', 'remained the same for both men and women', 'increased for both men and women', 'declined for women, but not for men']\",\"question\":\"From 1988 to 1990 among heterosexuals in the US, the number of unmarried adults aged 20 to 45 who report having multiple partners has:\",\"src\":\"ori_mmlu-human_sexuality\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}