{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"93002ac9-74ed-56f6-b4da-b618aef94799","task_key":"default--validation--47683","task_revision_id":"1","upstream_id":"47683","short_description":"HellaSwag validation 47683","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to tell if your neighbors are stealing your wi fi [title] Turn on your computer/laptop, and log in. [step] In the dialogue, type in \\\" explorer. Exe \\\" [title] Press the windows key and r. [step] Click on \\\" network \\\" on the side bar. [title] Wait for a window to open up displaying hard drives and stuff.\",\"endings\":\"[\\\"[step] If you see they're overflowing with stuff, move along to your next step. [title] Reenable the printer and wi-fi as soon as you reconnect them.\\\", '[step] Within the window, go to your external hard drive: \\\" motherboard \\\". This should give you the vitiligo storage size or how much memory it will need depending on the size and speed of the disk.', \\\"[step] If you see any fishy devices, like a weird phone, or computer, etc read on. If you don't, navigate to 192.168.1.1 or your router's address.\\\", '[step] You can also do this on the windows taskbar. [title] Head to your desktop [title] Hover your mouse over burn and select go or delete.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}