{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"db16a077-afaa-54ef-a665-b58b5eb333ce","task_key":"default--test--5773","task_revision_id":"1","upstream_id":"5773","short_description":"Long was a purchasing agent for Frost, a sole proprietor. Long had express…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"[\\\"No, because Grant should have verified Frost's competency.\\\", 'No, because the law protects those declared incompetent from contractual obligations.', 'Yes, because the contract was conducted online.', 'No, because Grant dealt only with Long.', \\\"Yes, because the order was valid regardless of Frost's incompetence.\\\", \\\"No, because Frost's incompetence voided the agency relationship.\\\", 'No, because Long did not have authority to enter into the contract.', 'Yes, because Long had express authority to place orders.', \\\"Yes, because Long's agency survived Frost's incompetence.\\\", 'Yes, because the order was placed before Frost was declared incompetent.']\",\"question\":\"Long was a purchasing agent for Frost, a sole proprietor. Long had express authority to place purchase orders with Frost's suppliers. Long conducted business on the Internet and had little contact with Frost. After Frost was declared incompetent in a judicial proceeding, Long placed an order with Grant Corp. on Frost's behalf. Both Long and Grant were unaware of Frost's incompetence. Will Frost or Frost's legal representative be liable to Grant?\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}