{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6d174283-8f0e-5c3c-b843-e3490f1699d2","task_key":"default--test--4785","task_revision_id":"1","upstream_id":"4785","short_description":"In periods of great difficulty and stress, it appears that the Minoans…","config":"default","split":"test","body":"{\"category\":\"history\",\"options\":\"['stopped all forms of art and music', 'fasted for long periods', 'built monumental structures', 'sacrificed adults and occasionally children', 'burnt their own houses to the ground', 'abstained from sexual relations', 'enacted strict laws and punishments', 'buried their possessions', 'performed complex dance rituals', 'migrated to different regions']\",\"question\":\"In periods of great difficulty and stress, it appears that the Minoans __________ in order to appease the gods.\",\"src\":\"ori_mmlu-prehistory\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}