{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cd7f8e7b-75c3-55f1-885a-a254525aa469","task_key":"default--validation--18110","task_revision_id":"1","upstream_id":"18110","short_description":"HellaSwag validation 18110","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to put a twitter badge on a website [title] Open 2 internet browser tabs on your computer. [step] In the first tab, login to your website's admin area. If you do not know html code, or you do not have access to the website's admin area, seek the help of your website's web developer.\",\"endings\":\"['[title] Check for a link to an existing website. [step] The link will be located in the banner or text field on your admin area.', '[title] Type \\\" twitter \\\" in the top right corner of your web browser. [step] While this is probably not the best line of defense, it is still a good option for the website\\\\'s layout.', \\\"[title] Access twitter's admin page. [step] At the top part of the page are three vertical links.\\\", '[title] Go to twitter.com in the second tab. [step] Sign into the twitter account that you want to post to your website.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}