{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3388d333-60cb-523f-a083-4302de7b7e07","task_key":"default--test--7094","task_revision_id":"1","upstream_id":"7094","short_description":"Why is it that in the United States, labor constitutes the singlemost important…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Labor has a fixed supply unlike land or capital', 'Labor is the most flexible and easily transportable factor of production', \\\"Labor is not directly linked to an individual's income\\\", 'Labor is the most abundant resource in the United States', 'Labor is the only factor that can be improved through education', 'Labor is not valued highly in the United States', 'Labor is the only factor of production that is not subject to taxation', 'Labor is less important than land or capital', \\\"Labor constitutes most of an individual's lifetime income\\\", 'Labor is the least costly factor of production']\",\"question\":\"Why is it that in the United States, labor constitutes the singlemost important factor of production?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}