{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9d61db5f-077e-5df3-8247-ebef15825eba","task_key":"default--test--1442","task_revision_id":"1","upstream_id":"1442","short_description":"A farmer conveyed 100 acres of his farm to a landscaper. The deed contained the…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['the buyer, but only for the covenants of seisinand right to convey.', 'the buyer, because the covenants of seisin and right to convey were breached by the farmer.', 'the farmer, because the buyer should have conducted a thorough title search before purchasing the property.', 'the buyer, because the covenant against encumbrances was breached by the farmer.', 'the farmer, because the buyer did not directly purchase the land from him.', 'the farmer, because no privity of estate existsbetween the buyer and the farmer.', 'the farmer, because the buyer was evicted due to a problem not related to the covenants in the deed.', 'the farmer, because the covenants are personalin nature and do not run with the land.', 'the buyer, because the warranty deed guarantees clear title.']\",\"question\":\"A farmer conveyed 100 acres of his farm to a landscaper. The deed contained the following covenants: (1) seisin, (2) right to convey, and (3) against encumbrances. Subsequently, the landscaper conveyed the property to a buyer by warranty deed. However, the buyer is later evicted by a woman because of paramount title. The buyer now brings suit against the farmer for breach of covenants in the deed. Judgment should be for\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}