{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0f0e103a-bb6a-5066-aa0d-b52c48e8fff4","task_key":"default--test--11608","task_revision_id":"1","upstream_id":"11608","short_description":"Let A =\\vert1- 12\\vertandB =\\vert283\\vert \\vert323\\vert\\vert151\\vert, find A + B…","config":"default","split":"test","body":"{\"category\":\"engineering\",\"options\":\"['A + B = |473| |375|, A - B = |-1 -9 -1| |3 -23|', 'A + B = |375| |475|, A - B = |-2 -9 -1| |2 -32|', 'A + B = |473| |475|, A - B = |-1 -10 -1| |2 -31|', 'A + B = |375| |475|, A - B = |-2 -9 -2| |1 -31|', 'A + B = |375| |474|, A - B = |-1 -8 -1| |2 -32|', 'A + B = |376| |475|, A - B = |0 -8 -1| |2 -33|', 'A + B = |373| |474|, A - B = |0 -9 -2| |3 -33|', 'A + B = |375| |474|, A - B = |-1 -9 -1| |2 -32|', 'A + B = |474| |375|, A - B = |-2 -8 -1| |1 -32|', 'A + B = |374| |473|, A - B = |-1 -10 -2| |3 -32|']\",\"question\":\"Let A =\\\\vert1- 12\\\\vertandB =\\\\vert283\\\\vert \\\\vert323\\\\vert\\\\vert151\\\\vert, find A + B and A - B.\",\"src\":\"stemez-ElectricCircuits\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}