{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"6dacfd97-4922-529d-ba16-07999337504c","task_key":"default--validation--876","task_revision_id":"2","upstream_id":"876","short_description":"HellaSwag validation 876","config":"default","split":"validation","body":"{\"activity_label\":\"Kayaking\",\"ctx\":\"A man in a small blue boat discusses the topic of hand surfing in a hole and gives hints on how to perform this activity. the man\",\"endings\":\"['demonstrates how to properly back up a hand surf by requiring an anchor.', 'demonstrates tricks that can be done while performing this activity.', 'in the small blue boat holds a fish in his hands and then holds the fish while talking about the technique needed to perform this activity.', \\\"is omic on the water as he's riding the speedo on the aerobic machine still pointing down at the water.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}