{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ea495249-b88f-58ff-9986-df4a19086e4f","task_key":"default--test--7173","task_revision_id":"1","upstream_id":"7173","short_description":"The Law of Diminishing Marginal Returns is responsible for","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['MC that first falls, but eventually rises, as output increases.', 'AP that first falls, but eventually rises, as output increases.', 'TC that first falls, but eventually rises, as output increases.', 'AVC that first rises, but eventually falls, as output increases.', 'MP that first falls, but eventually rises, as output increases.', 'AFC that first rises, but eventually falls, as output increases.', 'TP that first rises, but eventually falls, as output increases.', 'AFC that first falls, but eventually rises, as output increases.']\",\"question\":\"The Law of Diminishing Marginal Returns is responsible for\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}