{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e91442b0-edd2-5c42-9ee2-1f778f36fe9b","task_key":"default--validation--36525","task_revision_id":"1","upstream_id":"36525","short_description":"HellaSwag validation 36525","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to get free lives in pet rescue saga [title] Connect to facebook. [step] Once you've installed the game on your mobile device, connect it to your facebook account. Doing so will automatically back up and sync your progress since it's now stored within your facebook account.\",\"endings\":\"[\\\"Even if you've lost your mobile device or accidentally deleted the game, you're assured that your progress hasn't been lost. [substeps] If you're playing directly from facebook with a web browser, you may skip this step.\\\", 'You can choose your privacy settings by clicking your profile picture in the upper left corner of the screen. On the link to this tab, select \\\" settings, \\\" and then tap \\\" security.', 'To access the messages option, enter the appropriate email address for the app and tap on the \\\" log in \\\" button at the top. [substeps] You can also sign in via facebook by tapping the facebook icon in the top right corner of the screen.', 'Log in with facebook and click \\\" log in \\\" on the upper right-hand corner to access the facebook accounts page. [title] Open the game world menu and select \\\" game world.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}