{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cf0a641d-c347-56f4-80cc-b4440142ee0d","task_key":"default--validation--25655","task_revision_id":"1","upstream_id":"25655","short_description":"HellaSwag validation 25655","config":"default","split":"validation","body":"{\"activity_label\":\"Playing ice hockey\",\"ctx\":\"More clips of people playing hockey on a rink and a man holding a microphone because talking and the blue words on the bottom of the screen read \\\"new russian president vladimir putin makes surprise appearance\\\". the men\",\"endings\":\"['continue playing with all of the hockey players until the man with the microphone stops playing and walks away.', 'are done playing hockey and are now standing in a line so they can shake hands.', 'then discusses their movements, that is quickly changing to knitters thrown in the court of judges, and the blue words on the screen say \\\" link \\\".', \\\"continue to play hockey without losing a point and after several shots to the man he's holding a microphone pretending to ignore it and just play.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}