{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a03f50cf-b789-5fe8-8f55-f8e8f0e7fce7","task_key":"default--test--10449","task_revision_id":"1","upstream_id":"10449","short_description":"Hash tables can contribute to an efficient average-case solution for all of the…","config":"default","split":"test","body":"{\"category\":\"computer science\",\"options\":\"['Key comparison: Given two keys, determine if they are identical or different.', 'Counting distinct values: Given a set of n keys, determine the number of distinct key values.', 'Dynamic dictionary: Support the operations of insert, delete, and search in a dictionary.', 'Range search: Given values a and b, find all the records whose key value is in the range a, b.', 'Symbol table lookup: Given a program identifier, find its type and address.', 'Frequency count: Given a set of n keys, determine how many times each key appears.', 'Unique key identification: Given a key, find if it is unique in the set.', 'Sequential search: Given a key value, find the first occurrence of the key in the set.']\",\"question\":\"Hash tables can contribute to an efficient average-case solution for all of the problems described below EXCEPT:\",\"src\":\"ori_mmlu-college_computer_science\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}