{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ef042ea3-8960-5244-b8a4-21d16ec5da66","task_key":"all--test--ef042ea3-8960-5244-b8a4-21d16ec5da66","task_revision_id":"1","upstream_id":"","short_description":"A man and a woman who were friends decided to buy a car together. The man put…","config":"all","split":"test","body":"{\"choices\":\"['He can be convicted of larceny but he cannot be convicted of criminal mischief.', 'He cannot be convicted of any crime.', 'He can be convicted of criminal mischief only.', 'He can be convicted of both larceny and criminal mischief.']\",\"question\":\"A man and a woman who were friends decided to buy a car together. The man put $5,000 down and the woman's father co-signed on the loan of $10,000. The title was put in the woman's name and the man made the loan payments. They orally agreed that it was a shared ownership, with equal rights. After several months, their relationship deteriorated. They argued over ownership and use of the car. One day, the man attempted to leave with the car, but the woman climbed in and kept arguing as he was driving away. He then crashed into a pole and destroyed the car just to spite her, saying that if he couldn't have it no one would have it. The authorities arrested the man for auto larceny (stealing a car from the owner) and criminal mischief (damaging property of another or property shared with another). Assuming the accuracy of the foregoing facts beyond a reasonable doubt, which crime or crimes alleged can likely result in a sustainable conviction?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}