{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6e1829e1-9dff-57d1-a014-afc722247629","task_key":"default--test--6874","task_revision_id":"1","upstream_id":"6874","short_description":"Suppose that the wage rate is $30 per day and a factory owner is considering how…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Eight', 'Nine', 'Four', 'Eleven', 'Twelve', 'Ten', 'Three', 'Five', 'Seven', 'Six']\",\"question\":\"Suppose that the wage rate is $30 per day and a factory owner is considering how many workers to hire. Using the table below, determine how many workers should be hired in order to maximize profits (total revenues less total costs). Input (Labor) Value of Marginal Product per day 6 7 8 9 10 $50 70 35 20 10\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}