{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"808a453e-37b2-5ffa-b1cf-82222fc185fb","task_key":"default--test--2315","task_revision_id":"1","upstream_id":"2315","short_description":"What have been the results of studies which investigated the…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['People in TOT state cannot recall any characteristics of the target word.', 'The TOT state is exclusively linked to the recall of proper nouns.', 'Age is an irrelevant factor in the frequency of TOT occurrences.', 'People use semantic and phonetic characteristics associated with a word for the retrieval process in memory.', 'The TOT state typically resolves without the individual recalling the target word.', 'The TOT state is an indication of memory loss.', 'Individuals can recall only the first letter of the target word during TOT.', 'TOT is more common in individuals with a limited vocabulary.', 'Only phonetic characteristics help in memory retrieval.']\",\"question\":\"What have been the results of studies which investigated the tip-of-the-tongue(TOT) state?\",\"src\":\"stemez-Psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}