{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"99f3aa34-290e-57cb-a4b8-1a30b0f2455b","task_key":"default--test--6993","task_revision_id":"1","upstream_id":"6993","short_description":"Briefly enumerate two reasons why the supply curves of the farmersare relatively…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['(a) High price of raw materials. (b) Demand is always high.', '(a) Farmers have complete control over price. (b) Costs decrease with increased production.', '(a) Farmers are often subsidized by the government. (b) Costs increase with decreased production.', '(a) Intensified effort to increase output when price is low. (b) Many costs occur regardless of production quantity.']\",\"question\":\"Briefly enumerate two reasons why the supply curves of the farmersare relatively inelastic.\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}