{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c3759c2b-7987-570d-9fa5-a0236fafbd9f","task_key":"default--test--7127","task_revision_id":"1","upstream_id":"7127","short_description":"Why is it realistic to expect relatively free entry and exit in a…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Long-term contractual obligations for suppliers', 'Intense competition from foreign markets', 'Patents and copyrights that restrict competition', 'Perfectly homogeneous products', 'Significant economies of scale', 'Numerous, small firms with relatively little capital required', 'High barriers to entry', 'High customer loyalty to existing brands', 'Limited market demand', 'Government regulations']\",\"question\":\"Why is it realistic to expect relatively free entry and exit in a monopolistically competitive industry?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}