{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"57723870-16c5-5c6b-ad6b-f20a9ee70d2e","task_key":"default--test--745","task_revision_id":"1","upstream_id":"745","short_description":"A television manufacturing firm lowered the price of its deluxe portable…","config":"default","split":"test","body":"{\"category\":\"business\",\"options\":\"['The decision was a poor one', 'The decision had no effect', 'The decision was a good one', 'The decision increased profits', 'The decision increased profits but not revenues', 'The decision increased revenues but decreased profits', 'The decision had no effect on revenues but increased profits', 'The decision decreased both revenues and profits', 'The decision had no effect on profits but increased revenues', 'The decision increased revenues but not profits']\",\"question\":\"A television manufacturing firm lowered the price of its deluxe portable television set from $350 to $320 per unit, in order to increasetheir revenues and profits. This price change resulted in an increase in sales from 1,000 units per month to 1,080 units per month. Assuming the unit production costs did not change, was their decision to lower prices a good one?\",\"src\":\"stemez-Business\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}