{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"472a90ba-3117-5d21-ac8e-036138fb9cec","task_key":"default--test--7043","task_revision_id":"1","upstream_id":"7043","short_description":"Which of the following conditions is characteristic of oligopoly?","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['(I)) Rival firms act independently of each other.', \\\"Rival firms never interact or influence each other's prices.\\\", 'A rival firm matches price decreases of rivals, but fails to match any price increases of rivals.', 'A rival firm always undercut the price of rivals regardless of their price changes.', 'A rival firm matches price increases of rivals, but fails to match any price decrease of rivals.', 'A rival firm matches both price increases and decreases of rivals.', 'A rival firm fails to match both price increases or price decreases of rivals.', 'Rival firms always collude to set prices.', 'A rival firm matches price decreases of rivals, but exceeds any price increases of rivals.', 'Rival firms always have the same production costs.']\",\"question\":\"Which of the following conditions is characteristic of oligopoly?\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}