{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9486f9df-3dc2-5db5-a76f-6451f73a2aca","task_key":"default--test--808","task_revision_id":"1","upstream_id":"808","short_description":"A gas main carrying natural gas and laid in a corrosive soil ultimately reaches…","config":"default","split":"test","body":"{\"category\":\"business\",\"options\":\"['The total annual cost of the 25-year line is substantially lower than that of the 20-year line due to reduced leak repair costs', 'The total annual cost of the 20-year line is equal to that of the 25-year line when including the cost of gas loss', 'The total annual cost of the 25-year line is equal to that of the 20-year line', 'The total annual cost of the 20-year line is lower than that of the 25-year line when factoring in technological improvements in leak detection', 'The total annual cost of the 20-year line is lower than that of the 25-year line', 'The total annual cost of the 25-year line is higher than that of the 20-year line due to higher cumulative gas loss', 'The total annual cost of the 25-year line is lower than that of the 20-year line', 'The total annual cost of the 20-year line is higher than that of the 25-year line due to increased maintenance costs', 'The total annual cost of the 25-year line is higher than that of the 20-year line', 'The total annual cost of the 25-year line is slightly higher than that of the 20-year line due to escalating gas prices']\",\"question\":\"A gas main carrying natural gas and laid in a corrosive soil ultimately reaches the point where it develops small leaks that increase as time goes on. Assume the cost per mile of main is $8,000, that no gas is lost for the first 15 years of life, and that a main has zero net salvage value whenever retired. Assume also that the mile of main loses $60 worth of gas in the 16th year of life and that the cost of lost gas increases $60 every year thereafter. Interest or minimum attractive return is at 7%. Compare the equivalent uniform annual cost over the life of a main retired after 20 years with the equivalent uniform annual cost over the life of the main retired after 25 years.\",\"src\":\"stemez-Business\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}