{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0ad27e5f-cf84-50c2-a874-3ebaef9eaedf","task_key":"default--validation--40605","task_revision_id":"1","upstream_id":"40605","short_description":"HellaSwag validation 40605","config":"default","split":"validation","body":"{\"activity_label\":\"Gargling mouthwash\",\"ctx\":\"She takes a sip of listerine mouth was from her cup and holds it in her mouth for several seconds while her older brother video tapes her. another little girl\",\"endings\":\"['with a powder poodle watches in the background.', 'comes on screen to cue cues and she shows the shot from several angles that include the lap throws while the older girl watches her off her breasts.', 'takes a sip of her wile and starts to laugh in enjoyment.', 'walks behind her and watches her rinse.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}