{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"725fa328-5579-549a-b946-856576e28620","task_key":"all--test--725fa328-5579-549a-b946-856576e28620","task_revision_id":"1","upstream_id":"","short_description":"A landowner conveyed 50 acres of his orchard to a fanner. The deed contained the…","config":"all","split":"test","body":"{\"choices\":\"['recovery for the amount in principal and ininterest thereon from the time of the mortgagepayment.', 'recovery for only the principal that the farmerpaid on the mortgage.', 'recovery for the measure of value betweenthe value of the land with and without such anencumbrance.', 'no recovery.']\",\"question\":\"A landowner conveyed 50 acres of his orchard to a fanner. The deed contained the following covenants:(1) seisin, (2) right to convey, and (3) against encumbrances. When the landowner conveyed the property to the farmer, there was an outstanding mortgage. The farmer took possession, and shortly thereafter, threatened with foreclosure, he paid off the mortgage with interest. The farmer now sues for breach of covenant against encumbrances. The court will most likely allow\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}