{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"566050c7-bfca-5cda-8472-ce61563fc9f5","task_key":"default--test--10460","task_revision_id":"1","upstream_id":"10460","short_description":"Let I = (S,V)I=(S,V) be a MAC. Suppose S(k,m)S(k,m) is always 5 bits long. Can…","config":"default","split":"test","body":"{\"category\":\"computer science\",\"options\":\"['Yes, the attacker cannot generate a valid tag for any message', 'Yes, the PRG is pseudorandom', 'No, an attacker can simply guess the tag for messages', 'No, the attacker can simply guess the key', 'Yes, the key length is sufficiently long', 'No, the key length is not long enough', 'Yes, the tag length is sufficiently long', 'It depends on the complexity of the PRG', 'It depends on the details of the MAC', 'It depends on the encryption algorithm used in the MAC']\",\"question\":\"Let I = (S,V)I=(S,V) be a MAC. Suppose S(k,m)S(k,m) is always 5 bits long. Can this MAC be secure?\",\"src\":\"ori_mmlu-computer_security\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}