{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"85a8f2e3-18b0-5c10-b732-c28cae0394ed","task_key":"default--test--7246","task_revision_id":"1","upstream_id":"7246","short_description":"In order to find the market supply curve for a particular good, one would","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['aggregate the firm revenue curves vertically', 'aggregate the firm supply curves horizontally', 'aggregate the firm demand curves vertically', 'aggregate the firm marginal revenue curves horizontally', 'aggregate the firm marginal cost curves vertically', 'aggregate the firm revenue curves horizontally', 'aggregate the firm demand curves horizontally', 'aggregate the firm marginal cost curves horizontally', 'aggregate the firm marginal revenue curves vertically', 'aggregate the firm supply curves vertically']\",\"question\":\"In order to find the market supply curve for a particular good, one would\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}