{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c26c67da-4ab6-532b-b629-ece1fddc3ede","task_key":"default--test--1654","task_revision_id":"1","upstream_id":"1654","short_description":"Police were conducting a house-to-house search for an escaped convict. During…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['denied, because the teenager gave consent for the police officer to enter and search the home.', 'denied, because the police officer reasonably believed that he had permission to enter the dwelling and reasonably believed that the teenager was competent to give consent.', 'granted, because the teenager did not have an ownership or shared occupancy interest in the premises.', 'denied, because the police officer reasonably believed that he had permission to enter the dwelling.', 'denied, because the police officer was invited into the home by the teenager.', 'granted, because the police officer did not have a warrant to search the house.', 'granted, because the police officer did not have probable cause to search the house.', 'denied, because the police officer suspected that the bag contained narcotics.', 'granted, because the homeowner was not present during the search.', 'granted, because the teenager did not have the authority to consent to a search of the house.']\",\"question\":\"Police were conducting a house-to-house search for an escaped convict. During the search, a police officer went to a home and rang the doorbell. When a teenage male opened the door, the police officer told him that a convict had escaped and asked if he had seen anyone suspicious in the neighborhood. The teenager said he hadn't but then asked the police officer to come inside to ensure the house was safe. The police officer entered the home and proceeded to look around. When he went into the kitchen, the police officer noticed a bag containing white powder on the counter. Suspecting that the bag contained narcotics, the police officer questioned the teenager as to its contents. The teenager said he didn't live in the house and had never seen it before. The police officer confiscated the bag and took the teenager to the station house for questioning, at which time it was determined that the teenager had escaped from a psychiatric hospital and had broken into the home earlier in the day. Furthermore, the results of a police lab test found that the bag contained heroin. As a result, the police issued an arrest warrant for the owner of the home. He was apprehended and charged with possession of heroin. The homeowner's attorney has filed a motion to suppress, claiming that the heroin confiscated by the police was unlawfully obtained. The motion should be\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}