{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2b994812-f9f3-529a-a027-88a60f60c132","task_key":"all--test--2b994812-f9f3-529a-a027-88a60f60c132","task_revision_id":"1","upstream_id":"","short_description":"A manufacturer of snowmobiles produced one model, Model IA, which was a very…","config":"all","split":"test","body":"{\"choices\":\"['the manufacturer only.', 'the manufacturer, even though it was unaware of the removal of the speed regulator.', 'the operator only.', 'both the manufacturer and the operator.']\",\"question\":\"A manufacturer of snowmobiles produced one model, Model IA, which was a very popular snowmobile. On each of the Model 1 A snowmobiles produced, the manufacturer installed a speed-regulating device that prevented the snowmobiles from exceeding 50 m. p. h. The manufacturer distributed the snowmobile through wholesalers and retailers across the country. In its sales brochures and advertising materials, the manufacturer advises all of its distributors against making any modifications on snowmobiles without first conferring with the manufacturer's staff of engineers. The operator of a shop in a mountain community had the exclusive right to distribute Model 1A snowmobiles in the area. Amid complaints from previous buyers regarding Model lA's speed limitations, the operator removed the speed regulators from all the Model 1A snowmobiles in stock. A buyer purchased from the operator one of the Model 1A snowmobiles that had the speed regulator removed. One day, the buyer was driving the Model 1A snowmobile through a snow-covered field, at speeds in excess of 70 m. p. h. The snowmobile hit a snow-covered rock, causing the buyer to lose control of the snowmobile, crash into a tree, and suffer serious injuries. If the buyer institutes a strict liability in tort action, he will most likely recover against\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}