{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"66bf22b7-0b58-5368-ab3a-0b9a70d05a9d","task_key":"default--validation--22357","task_revision_id":"1","upstream_id":"22357","short_description":"HellaSwag validation 22357","config":"default","split":"validation","body":"{\"activity_label\":\"Cars & Other Vehicles\",\"ctx\":\"[header] How to get car title in your name [title] Buy a car from a dealership. [step] A transfer of ownership takes place whenever a car changes registered owners. One of the most common ways this takes place is when you buy a car from a dealership.\",\"endings\":\"['The dealer will then transfer the title to you. Similar to buying a foreign car, you cannot transfer your title.', 'Different dealerships offer lower interest rates during escrow, and state exchanges schedule the transfer. You are responsible for contacting a dealer to have your title transferred into your name.', 'However, auto dealers do not provide the same, exact contents as dealers so going into that new automotive junkyard may come with some confusion. [substeps] A dealership will not hold an item a back-lease dealer holds.', 'Before you buy the car, it is owned by the dealership or the manufacturer. Once you purchase the vehicle, title needs to be transferred into your name.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}