{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d6e38399-20c9-5f3d-be8a-1274e1bb0dba","task_key":"default--validation--48146","task_revision_id":"1","upstream_id":"48146","short_description":"HellaSwag validation 48146","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to make a video call on skype [title] Check for a webcam. [step] Be sure your computer has a webcam. On most laptops, the webcam will look like a small hole in the top border of the screen.\",\"endings\":\"['It may also have some shaking and vibrations. The webcam may also be gear shaped.', \\\"[substeps] If you don't have any webcam at your home, take a photo of yourself without a webcam-people will be asking for one later. This will help prevent people from knowing where you have your webcam.\\\", 'The majority of laptops made in the past five years have webcams. [substeps] If you do not see a webcam, you will have to purchase an external webcam.', 'Call the webcam manufacturer to request it. [title] If your laptop has a webcam-either debug or try skype.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}