{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4d720c14-1292-5693-ba21-fa958f603c6d","task_key":"default--validation--35162","task_revision_id":"1","upstream_id":"35162","short_description":"HellaSwag validation 35162","config":"default","split":"validation","body":"{\"activity_label\":\"Work World\",\"ctx\":\"[header] How to ten key [title] On a computer, make sure number lock is on. [step] Press the num lock button until the corresponding light is on. If the number lock is not on, pressing keys on the number pad will generally do the other thing that is written on them, typically some combination of arrow and page up/page down keys.\",\"endings\":\"['[title] Now press \\\" open/home/menu \\\" if you are using windows. [title] Deactivate google drive by pressing the shift key or ctrl + alt + alt (with a sausage key).', '[title] On the keypad, hit the num trap button, which is usually located at the top-right corner of the keypad. [title] Decide if you want to set the code or not.', '[title] Place your middle finger on the 5 key. [step] Often, there will be a small dimple or ridge on this key so that you can easily find it by feel.', \\\"If your number lock button is on the top screen, press the-button to set it on. If it's not on the top screen, press the 3/4 key.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}