{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"81395314-e151-5c9b-81be-2bc623ac7c4e","task_key":"all--test--81395314-e151-5c9b-81be-2bc623ac7c4e","task_revision_id":"1","upstream_id":"","short_description":"A city was growing rapidly and becoming a major tourist area because of its warm…","config":"all","split":"test","body":"{\"choices\":\"['recover, because the businessman is liable for permitting a dangerous condition to exist along an area of public use.', 'recover, under the doctrine of respondeat superior.', 'not recover, because the builder was an independent contractor.', 'not recover, because the jogger caused the pedestrian to fall into the wall.']\",\"question\":\"A city was growing rapidly and becoming a major tourist area because of its warm climate and proximity to the ocean. A businessman, anticipating a continued financial boom, purchased an expensive corner lot, which he planned to convert into an exclusive resort hotel. The businessman hired a builder to perform the construction work according to architectural guidelines provided by the businessman. It was anticipated that construction would take approximately six months. During this time, the builder dug a large hole and worked with heavy equipment inside the excavation site, leveling the terrain and building the foundation. The builder also erected a large wall along the sidewalk, which encircled the entire corner. The wall, which was negligently constructed, was made of thinly cut wood nailed together in sections and affixed just under the base of the sidewalk next to the excavation. Early one morning, a pedestrian was walking past the construction site on his way to work. As he was rounding the corner, the pedestrian saw a jogger approaching. When they drew even, the jogger accidentally brushed against the pedestrian with the swing of her arms. The pedesthan fell into the wall, which gave way, and he fell into the hole below. The pedestrian suffered serious injuries from the fall. If the pedestrian sues the businessman for the personal injuries he suffered in his fall, the most likely result is that the pedestrian will\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}