{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ef7682d5-410a-54b6-9852-26eec257a501","task_key":"default--test--7672","task_revision_id":"1","upstream_id":"7672","short_description":"What is bilateral monopoly?","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['A market with many buyers and one seller', 'A market with one buyer and one seller, but where the government regulates all transactions', 'A market with one buyer and many sellers', 'A market where multiple buyers and sellers exist, but only two dominate the market', 'A market with two buyers and two sellers, all of whom have equal market power', 'A market with many sellers, each selling a unique product to one buyer', 'A market structure where two sellers compete to sell to many buyers', 'A market situation in which a single seller faces a single buyer', 'A market with many sellers and many buyers', 'A market where a single seller faces many buyers, but all buyers coordinate their purchases']\",\"question\":\"What is bilateral monopoly?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}