{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9aace8de-a1df-554d-84d3-a5f0e5e22d7c","task_key":"default--test--846","task_revision_id":"1","upstream_id":"846","short_description":"A super market has two cans of the same brand of tuna fish. The 7(1 / 2) ounce…","config":"default","split":"test","body":"{\"category\":\"business\",\"options\":\"['None of the cans', 'The first can', 'The second can is better by 7 cents per ounce', 'The second can is better by 5 cents per ounce', 'The second can is better only when buying in bulk', 'The first can is better by 10 cents per ounce', 'The second can', 'Both cans are better buys on alternate days', 'Both cans are the same', 'The first can is better when considering nutritional value']\",\"question\":\"A super market has two cans of the same brand of tuna fish. The 7(1 / 2) ounce can sells for 57 cents, and the 8(1 / 4) ounce can sells for 64 cents. Which can is the better buy?\",\"src\":\"stemez-Business\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}