{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"04755d7e-804d-5464-a2bc-de4abdc7fb87","task_key":"default--test--6901","task_revision_id":"1","upstream_id":"6901","short_description":"Which of the following is a characteristic of perfect competition?","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Only one firm controls the entire market.', 'Firms produce a homogeneous product.', 'The government sets the price for all products.', 'Firms produce a diverse range of products.', 'Firms are price-setting profit maximizers.', 'Firms can differentiate their products.', 'Barriers to entry exist.', 'Firms can freely enter and exit the market.', 'The government regulates the price so that deadweight loss is eliminated.', 'Only a few firms dominate the market.']\",\"question\":\"Which of the following is a characteristic of perfect competition?\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}