{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4a8537a4-00b4-55fb-b01f-0fd180c0363a","task_key":"default--validation--7433--variant-04546b008c8d","task_revision_id":"1","upstream_id":"7433--variant-04546b008c8d","short_description":"HellaSwag validation 7433","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to manage a google group [title] Create a google group. [step] The first thing to managing a google groups is to create one. If you don't have a google account, you must register for a.\",\"endings\":\"['Do this by creating one of the following methods : [substeps] Create your own google group templates by clicking on the \\\" create \\\" button, then choosing the \\\" create agreement \\\" option that appears, and enter your name and password. Follow the prompts to create a sign-in page that establishes your google group and your personal google group.', 'In order to create one, you have to do the following : [substeps] Open a new web browser from the start menu and go to google groups. Click the \\\" create new group \\\" button.', 'A google group is also created for personal use. You can choose to join a google group by clicking the \\\" sign up with google \\\" button on the website.', 'Once you have your account, go to groups.google.com (it will automatically go to the sign in page) and sign in, unless you already have. You can take a tour, but if you choose to skip it, click on \\\" create a group \\\" which is underneath the three steps to create one.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}