{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"24bc0625-f468-5b4e-ad8e-7eeb8caeae80","task_key":"default--test--7057","task_revision_id":"1","upstream_id":"7057","short_description":"For the perfectly competitive firm, the profit-maximizing decision to shut down…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['exceeds maximum average total cost.', 'is less than maximum average variable cost.', 'is equal to minimum marginal cost.', 'falls below minimum average total cost.', 'is equal to average revenue.', 'is equal to maximum average total cost.', 'falls below minimum average variable cost.', 'is equal to minimum average total cost.', 'exceeds average total cost but less than average variable cost.', 'is greater than minimum average variable cost, but lower than minimum average total cost.']\",\"question\":\"For the perfectly competitive firm, the profit-maximizing decision to shut down is made when the price\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}