{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c861e4a2-dc1f-532a-a8ac-ff4fbc14e5a7","task_key":"default--validation--40938","task_revision_id":"1","upstream_id":"40938","short_description":"HellaSwag validation 40938","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to create a google forum [title] Visit the google groups website. [step] Open a new tab on your browser and go to the. [title] Sign into google groups with your google account.\",\"endings\":\"['[step] You can sign up either by clicking \\\" create a google group \\\" or by logging in with a google username and password. You can also click \\\" sign up \\\" and you will be redirected to the group\\\\'s login page where you will see a lot of people who have joined each group.', '[step] When you log into your account, you will get to the page where you can create a google account. Go to your google groups website and click the blue link in the bottom right corner.', '[step] Google has made it possible to use one google account across all google products. Enter your google email address on the first text box and your password on the second text box.', '[step] Use the email and password that you log in with so you can create a google group. Click on the arrow next to a link to the group page.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}