{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"692ddd1a-561f-5e81-84c7-b0c2e78039ce","task_key":"default--test--7093","task_revision_id":"1","upstream_id":"7093","short_description":"Mr.Caponiis a butcher who recently raised the price of steak at his market from…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['The demand for steak is constant', 'The demand for steak is elastic', 'The demand for steak is indirectly proportional to price', 'The demand for steak is not affected by price', 'The demand for steak is perfectly elastic', 'The demand for steak is perfectly inelastic', 'The demand for steak is inelastic', 'The demand for steak is determined by the quality', 'The demand for steak is dependent on consumer income levels', 'The demand for steak fluctuates seasonally']\",\"question\":\"Mr.Caponiis a butcher who recently raised the price of steak at his market from $1.50 to $2.00 a pound. Correspondingly, his sales dropped from 200 pounds per day to 100 pounds per day. Is the demand for steak?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}