{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d5860304-9941-5756-9389-f2ec54709d90","task_key":"default--test--10359","task_revision_id":"1","upstream_id":"10359","short_description":"Which of the following is NOT a property of bitmap graphics?","config":"default","split":"test","body":"{\"category\":\"computer science\",\"options\":\"['They can support millions of colors', 'Realistic lighting and shading can be done.', 'Bitmaps can be made transparent', 'Fast hardware exists to move blocks of pixels efficiently.', 'Bitmap graphics can be created in multiple layers', 'All line segments can be displayed as straight.', 'Polygons can be filled with solid colors and textures.']\",\"question\":\"Which of the following is NOT a property of bitmap graphics?\",\"src\":\"ori_mmlu-college_computer_science\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}