{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9c1d273d-8d36-5f39-91e1-aace8416c317","task_key":"default--validation--20606","task_revision_id":"1","upstream_id":"20606","short_description":"HellaSwag validation 20606","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to recover a gmail password [title] Go to http: // www.gmail.com. [step] Use the link or type the url in a web browser. [substeps] If your email or phone number isn't filled in automatically, type it into the labeled field and click on next.\",\"endings\":\"['If you accidentally enter your email address in the field, click on next. A confirmation message will appear telling you the password has been successfully deleted.', 'All of your email addresses will be listed here. [title] Enter the email address of the sender and password associated with that email address.', '[title] Click on forgot password? [step] Below the password field. [title] Enter the last password you remember and click on next.', 'You can also click on \\\" change password \\\" to see if any changes are currently required. If you signed in directly from gmail.com, you\\\\'ll be prompted to enter the email address and password associated with your gmail account.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}