{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"13f24746-43ab-5519-b9f2-ef532fc4f3c5","task_key":"all--test--13f24746-43ab-5519-b9f2-ef532fc4f3c5","task_revision_id":"1","upstream_id":"","short_description":"A farmer has a large field where he grows corn. He hires a crop-duster to spray…","config":"all","split":"test","body":"{\"choices\":\"[\\\"only the crop-duster is liable, because its negligence occurred before the construction company's negligence.\\\", \\\"only the construction company is liable, because its negligence would have destroyed the farmer's corn crop no matter what the crop- duster did.\\\", \\\"both the crop-duster and the construction company are liable, because each engaged in a negligent action that was sufficient to destroy the farmer's corn crop.\\\", 'neither the crop-duster nor the construction company is liable, because neither is a \\\"but for\\\" cause of the harm.']\",\"question\":\"A farmer has a large field where he grows corn. He hires a crop-duster to spray the field, but the crop- duster mistakenly sprays a strong herbicide, rather than a pesticide, and kills all the corn plants in the field. Several days later, a construction company building a house near the farmer's field negligently hits a gas line and starts a fire that burns everything in the area, including the farmer's field. If the farmer brings negligence claims against the crop-duster and the construction company in order to recover damages for the destruction of his corn crop, the most likely result is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}