{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"76217103-db6e-5a36-8943-69da277cbaed","task_key":"default--validation--25654","task_revision_id":"1","upstream_id":"25654","short_description":"HellaSwag validation 25654","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to play online games safely [title] Download an anti-virus system onto the computer. [substeps] These systems need to be downloaded to ensure that if you do find the wrong website to play games on, you don't have to worry about viruses or hackers seeping into your computer. Mac computers are much less vulnerable to viruses than pc systems.\",\"endings\":\"['If you have a mac, you might be able to get by without antivirus software. [title] Have a strong password for your gaming account.', \\\"[title] Always change your passwords when you're playing online games. [substeps] A common mistake is not to know your password at all.\\\", '[title] Note the username and password of the computer you want to install the game on. [step] Where are these passwords? [substeps] If you cannot find them, just type the password into the box below the \\\" what is this app/game? \\\" field.', 'Apple will also likely have anti-virus systems that you can download for free. [title] Navigate to the category download page and click \\\" free download.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}