{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2b0bd810-fe58-52f5-848a-bead66c79be0","task_key":"default--validation--42445","task_revision_id":"1","upstream_id":"42445","short_description":"HellaSwag validation 42445","config":"default","split":"validation","body":"{\"activity_label\":\"Layup drill in basketball\",\"ctx\":\"A man is seen speaking to the camera while people practice basketball drills intermittently. The man ties a rope around a ball and continues having him perform layups on the court. more clips\",\"endings\":\"['are shown of people receiving various awards and the word \\\" winner \\\" is shown on the screen.', 'are shown of them practicing while the man speaks to the camera.', 'are shown of the team performing professional basketball drills and announcers speaking to the camera.', 'are shown of the man speaking to the camera.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}