{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7af8e0dc-1d09-5acd-9c21-297c4e39eb83","task_key":"default--validation--5405","task_revision_id":"1","upstream_id":"5405","short_description":"HellaSwag validation 5405","config":"default","split":"validation","body":"{\"activity_label\":\"Doing karate\",\"ctx\":\"He begins to show several different clips from the show. He also comments about different aspects of the show while being interviewed. he\",\"endings\":\"['closes the video with a quick goodbye and the closing credits are shown on the screen.', 'is also seen sitting on a bench speaking to a woman who begins to pass the pitcher back and fourth.', 'seems to know the routines as he talks about the ropes that he uses.', \\\"alternate health issues and hairstyle on the show, providing some very detailed advice on what's to come.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}