{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2f327995-43fd-5576-acdf-3cc90c9be6a0","task_key":"default--validation--43540","task_revision_id":"1","upstream_id":"43540","short_description":"HellaSwag validation 43540","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to transfer pictures from a razr maxx to a computer [title] Make sure that your phone is powered on and is unlocked. [title] Connect the razr maxx to the computer. [step] Grab a usb cable and insert the smaller end to the charging port on your phone.\",\"endings\":\"['Take the larger usb 2.0 end of the cable and insert it into a vacant usb 2.0 port on your computer. [substeps] Use the usb cable that was included in the box when you purchased your razr maxx.', 'Next, plug the long end into the power source of the computer. If there is a power outlet, it may also work.', '[title] Check the shutter speed in front of the screen. [step] You can usually tell the shutter speed by the number on the screen.', '[title] Enable the \\\" jailbreak \\\" feature. [step] This mode allows you to erase any data you have stored from the phone, including your phone\\\\'s file.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}