{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"334c3ba3-6afb-559d-aaa2-f19c34b0fd39","task_key":"default--test--1062","task_revision_id":"1","upstream_id":"1062","short_description":"In 1985, a landowner, the undisputed owner of an apartment building, leased it…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['if the tenant believed that his lease with the landowner was still in effect after the latter died.', 'because the Girl Scouts of America did not know about their ownership until 2009.', \\\"because the tenant discontinued paying rent following the landowner's death.\\\", \\\"if the tenant was never formally notified about the landowner's will.\\\", 'if the tenant leased the apartment building to a teacher for three years while he (the tenant) was called into military service.', 'because the statutory period for adverse possession had not yet been reached.', 'if the tenant claimed to have bought the apartment building from the landowner before her death.', 'because the tenant never paid taxes on the apartment building.', 'if the tenant made significant improvements to the apartment building.']\",\"question\":\"In 1985, a landowner, the undisputed owner of an apartment building, leased it to a tenant for a term of seven years. Rent was to be paid in installments due on the first day of each month. One year later, in 1986, the landowner died leaving no heirs. In her will, the landowner left her entire estate to the Girl Scouts of America. The apartment building was not specifically mentioned in the will. One month after the landowner died, the tenant, having learned of the landowner's death, decided to remain on the property, but stopped paying rent. The Girl Scouts of America organization was unaware of the landowner's gift to the organization until 2009, when attorneys for the organization conducted an inventory of testamentary gifts to the Girl Scouts of America, which revealed that they had a claim to ownership of the apartment building. The statutory period for adverse possession in this jurisdiction is 20 years. In an ejection action by the Girl Scouts of America against the tenant, the organization will prevail\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}