{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"60e7f2f4-0a5c-5b04-8d79-48855631d8c2","task_key":"default--test--10648","task_revision_id":"1","upstream_id":"10648","short_description":"Briefly explain thesignificance of the ASSEMBLER language.","config":"default","split":"test","body":"{\"category\":\"computer science\",\"options\":\"['ASSEMBLER language allows programmers to write code using only English phrases and sentences.', 'ASSEMBLER language is a scripting language used for automating web page interactions.', 'ASSEMBLER language directly executes instructions without translation.', 'ASSEMBLER language is used to increase the size of the code, making it more readable.', 'ASSEMBLER language is a high-level programming language.', 'ASSEMBLER language is a tool for creating graphical user interfaces in software development.', 'ASSEMBLER language eliminates the need for binary code.', 'ASSEMBLER language is primarily used for database management.', 'ASSEMBLER language translates binary code into high-level programming languages.', 'ASSEMBLER language is an intermediate language that translates high-level programming languages into binary code, saving execution time.']\",\"question\":\"Briefly explain thesignificance of the ASSEMBLER language.\",\"src\":\"stemez-ComputerScience\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}