{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"89e576b9-a2c8-5beb-af67-05f026eb64d3","task_key":"default--test--7444","task_revision_id":"1","upstream_id":"7444","short_description":"A competitive firm's demand for labor is determined directly by","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['the marginal (physical) product of labor and the output price', 'the opportunity cost of workers’ time', 'the marginal (physical) product of labor and the wage', 'the number of employees', 'profits', 'the average (physical) product of labor and the output price', 'the wage and the average (physical) product of labor', 'the cost of raw materials', 'the cost of production and the sales revenue', 'the market share of the firm']\",\"question\":\"A competitive firm's demand for labor is determined directly by\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}