{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c37538af-b577-5d02-b6da-28af5dcf8651","task_key":"all--test--c37538af-b577-5d02-b6da-28af5dcf8651","task_revision_id":"1","upstream_id":"","short_description":"A man attended a convention at a hotel, but did not get a room. One evening…","config":"all","split":"test","body":"{\"choices\":\"['No, there was probable cause for an arrest and there can therefore be no false imprisonment.', 'No, the failure to immediately provide receipts gave the security personnel justification to confine him.', 'Yes, when one is told to sit in one place until further notice that is always false imprisonment.', 'Yes, the elements of false imprisonment are sufficiently established \\\\x80\\\\x93 there was an intentional illegal confinement that caused harm.']\",\"question\":\"A man attended a convention at a hotel, but did not get a room. One evening armed security officers detained him and accused him of stealing. According to the hotel, the security camera displayed him \\\"wandering aimlessly\\\", and at times \\\"suspiciously\\\" handling various items offered for sale. They seized the man's paper bags which contained various items sold by the convention vendors; due to being surprised and disoriented, he could not immediately locate his receipts. The officers made him sit on a chair in the hallway until the police arrived 20 minutes later. The man later produced receipts for the items and was released. He sued the hotel for the tort of false imprisonment. Will the court allow the case to go to the jury?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}