{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"47e06777-ddd3-5b4d-ae21-dd1d757ca0f0","task_key":"default--validation--37998","task_revision_id":"1","upstream_id":"37998","short_description":"HellaSwag validation 37998","config":"default","split":"validation","body":"{\"activity_label\":\"Longboarding\",\"ctx\":\"Two people are seen kicking around a skateboard and lead into clips of them riding down a road. the camera\",\"endings\":\"['shows the men in several shots lifting a board and people pushing it.', 'captures several scenes of people riding through a garage and over a hurdle.', 'man watches the view and then zooms in on the board as well as computer equipment behind the boys and them riding down the road.', 'follows the boys from several angles skating down a street and performing various tricks.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}