{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"879c3bf3-ed99-5266-8fe8-3596ae420c35","task_key":"all--test--879c3bf3-ed99-5266-8fe8-3596ae420c35","task_revision_id":"1","upstream_id":"","short_description":"A senator, had been a state senator for three years. During that period of time,…","config":"all","split":"test","body":"{\"choices\":\"['None.', 'Four.', 'Seven.', 'Eleven.']\",\"question\":\"A senator, had been a state senator for three years. During that period of time, it had been his regular practice to stay at a hotel in the state's capital while the legislature was in session (namely from September through March). As a general rule, the senator stayed at the hotel each and every night during that seven-month period. When the senator checked into the hotel on September 1, 2007 for the 2007 €”2008 term, the hotel manager quoted the senator his usual senatorial discount rate of $50 per night. Although nothing was said regarding payment terms, the senator customarily paid the bill in weekly installments. During the months of September and October, he paid the hotel at the end of each week $350 plus tax and other accumulated expenses. On the last day of October, the senator made his usual weekly payment covering the rental period from October 25 to October 31. Then, on the morning of November 4 (before checkout time for that day), the senator decided to suddenly move out of the hotel and register at a new motel located two blocks away. He notified the manager at the hotel of his intent to check out and tendered a check for $150 plus tax and other incidental charges incurred from November 1 through November 3. When the manager noticed that the senator was paying only for three days, she remarked, \\\"You still owe us $200 plus tax for the rest of the week and $350 plus taxes for an additional week because you didn't give us a week's notice that you planned to vacate. \\\" The senator refused to pay the additional charges. If the hotel brings suit against the senator to recover for the unpaid balance, how many additional days lodging will he be required to pay for?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}