{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3c2286d2-86c8-505e-88b2-38d268735a68","task_key":"default--validation--41692","task_revision_id":"1","upstream_id":"41692","short_description":"HellaSwag validation 41692","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to transfer your number to a new iphone [title] Power off your old phone. [step] Before removing the sim card from your phone, make sure the phone is powered off. [title] Locate the sim tray on your old phone.\",\"endings\":\"[\\\"[step] This will vary by phone model, so it will typically be located on the front of the passenger side door or in the driver's side door. The sim tray may be green, or it may also have a name or symbol instead.\\\", \\\"[step] You can take your phone's sim tray to a nearby receptacle. Use the soft-touch touch provided by the screen to flip the tray onto the soft side.\\\", '[step] For all phones, a sim tray is built into the inside of the phone so that you can store older phones and other hard drives. It is usually located at the top of the phone, near the tray.', '[step] The sim tray is a oval shaped compartment with pin hole located on the side of your phone. On samsung phones, the sim tray is usually located on the top of the phone.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}