{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a5c07b74-6f8a-5cf0-b0f2-0689a71af7cd","task_key":"all--test--a5c07b74-6f8a-5cf0-b0f2-0689a71af7cd","task_revision_id":"1","upstream_id":"","short_description":"A major advantage of direct mapping of a cache is its simplicity. The main…","config":"all","split":"test","body":"{\"choices\":\"['it does not allow simultaneous access to the intended data and its tag', 'it is more expensive than other types of cache organizations', 'the cache hit ratio is degraded if two or more blocks used alternately map onto the same block frame in the cache', 'its access time is greater than that of other cache organizations']\",\"question\":\"A major advantage of direct mapping of a cache is its simplicity. The main disadvantage of this organization is that\",\"subject\":\"college_computer_science\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}