{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"eb74b41f-b0fa-58a4-87cf-0839adbb0e23","task_key":"default--validation--2893","task_revision_id":"1","upstream_id":"2893","short_description":"HellaSwag validation 2893","config":"default","split":"validation","body":"{\"activity_label\":\"Layup drill in basketball\",\"ctx\":\"A graphic shows on the screen for a website about basketball drills. Some young men on an indoor basketball court are shown doing some of the basketball drills. an older man\",\"endings\":\"['in a blue shirt watches him.', 'watch on the sidelines as the young men on the court make some best strikes with sticks.', 'does roanoke court drills with the young man before him.', 'begins explaining the drills while more footage is shown of the players demonstrating them.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}