{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f139cbda-da24-5f3f-b118-23c04717e1d2","task_key":"default--validation--12084","task_revision_id":"1","upstream_id":"12084","short_description":"HellaSwag validation 12084","config":"default","split":"validation","body":"{\"activity_label\":\"Swimming\",\"ctx\":\"A small group of people are seen swimming in a pool and leads into various clips of children learning how to swim. A man helps them practice in the water as well as jumping off a diving board. more shots\",\"endings\":\"['are shown of kids swimming and leads into a girl grabbing objects under water and swimming with another.', 'of kids play as well as holding up silver objects and getting worn down.', 'are shown of the crowd cheering and showing the girls swimming successfully.', 'are shown of kids swimming and lie down afterwards while the boy keeps in time with everyone inside the pool.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}