{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6c50844f-ea92-5dc8-868a-b605a38fdf55","task_key":"default--test--7435","task_revision_id":"1","upstream_id":"7435","short_description":"Which of the following is most likely to result in a shift to the right in the…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['A decrease in the price of orange juice', 'A decrease in the price of Tang', 'An increase in the price of grapefruit juice', 'A decrease in the income of juice drinkers', 'A bumper crop of oranges in Florida', 'Expectations of higher future income among juice drinkers', 'A decrease in the popularity of healthy drinks', 'A significant increase in the price of orange juice', 'A decrease in the price of apples', 'Expectations of lower future prices for orange juice']\",\"question\":\"Which of the following is most likely to result in a shift to the right in the demand curve for orange juice?\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}