{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"439b7ca7-5909-5cdb-a7c8-c6772c758a50","task_key":"default--test--7520","task_revision_id":"1","upstream_id":"7520","short_description":"The ABC Pencil Co. was considering a price increase and wished to determine the…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['The calculated elasticity will appear much higher, misleading the company', 'The change in base will create an illusion of inelastic demand', 'The elasticity measure will decrease', 'The price point for maximum revenue will appear to change dramatically', 'Elasticity is independent of the unit of measurement employed.', 'The price-demand curve will shift to the right', 'The elasticity measure will increase', 'The calculated elasticity will seem artificially low, causing confusion in pricing strategy', 'The sabotage will significantly affect the results', 'The demand will seem more elastic than it actually is']\",\"question\":\"The ABC Pencil Co. was considering a price increase and wished to determine the elasticity of demand. An economist and a market researcher, Key and Worce, were hired to study demand. In a controlled experiment, it was determined that at 8\\\\textcent, 100 pencils were sold yielding an elasticity of 2.25. However, key and worce were industrial spies, employed by the EF Pencil Co. And sent to ABC to cause as much trouble as possible. So key and worce decided to change the base for their elasticity figure, measuring price in terms of dollars instead of pennies ( i.e., $.08 for 8\\\\textcent and $.10 for 10\\\\textcent ). How will this sabotage affect the results?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}