{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"26f9305e-cba2-50dc-aaff-ba5ddbf7162e","task_key":"default--test--1367","task_revision_id":"1","upstream_id":"1367","short_description":"A landlord rented an apartment to a tenant for $250 per month in accordance with…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['No, because the landlord can retract the offer at any time before the tenant accepts.', 'No, unless the tenant paid or tendered to the landlord the one-dollar consideration.', \\\"Yes, because it manifested a willingness to enter into a specific bargain that could be concluded by the tenant's assent.\\\", 'No, because the tenant did not immediately respond to the letter.', 'Yes, because the landlord cannot change their mind once an offer is made.', 'Yes, because consideration for the option can be infeffed from the previous month-to-month lease.', \\\"No, because the tenant did not pay the two months' rent in advance.\\\", 'Yes, because the tenant expressed the intent to sign the lease within the specified timeframe.', 'No, because it contained a condition precedent to execution of the proposed lease.', \\\"Yes, because the tenant has the right to accept the offer within the 20 days, regardless of the landlord's change of mind.\\\"]\",\"question\":\"A landlord rented an apartment to a tenant for $250 per month in accordance with a month-to-month agreement. On September 3, the landlord mailed the following letter to the tenant:\\\"September 2 In consideration of one dollar, receipt of which is acknowledged, I hereby give you an option for 20 days from the above date to sign a two-year lease at $225 per month, provided you pay two months' rent in advance. \\\"The tenant received the letter on September 4, but did not read it until September 5. On September 23, the tenant telephoned the landlord and said, \\\"I want to give you the $450 and sign the two-year lease as soon as possible. \\\" The landlord replied, \\\"I've change my mind. I do not want to sign a lease with you. Moreover, I want you to vacate the apartment immediately. \\\" At no time after receiving the landlord's letter on September 4 did the tenant pay him the one-dollar consideration. Did the landlord's letter constitute an effective offer for a two-year lease?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}