{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4305e32e-0b74-501e-b520-db4b2433663d","task_key":"default--test--5712","task_revision_id":"1","upstream_id":"5712","short_description":"Which of the following methods should be used to account for research and…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['Charging half of the costs to expense when incurred and capitalizing the other half.', 'Charging all costs to a research and development reserve account.', \\\"Accumulating all costs in a separate component of stockholders' equity until the existence of future benefits can be determined.\\\", 'Capitalizing all costs when incurred.', 'Charging all costs to expense when incurred.', 'Capitalizing all costs until future benefits can be ascertained, then shifting to expensing.', 'Capitalizing costs specified by management and charging all other costs to expense.', 'Capitalizing only those costs associated with successful projects and expensing all others.', 'Deferring all costs until the project is complete or abandoned.', 'Allocating costs proportionally to current and future periods based on expected benefits.']\",\"question\":\"Which of the following methods should be used to account for research and development costs with no alternative future use?\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}