{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"649cf4c1-130e-57ba-81a3-5802943bafbb","task_key":"default--validation--47190","task_revision_id":"1","upstream_id":"47190","short_description":"HellaSwag validation 47190","config":"default","split":"validation","body":"{\"activity_label\":\"Getting a piercing\",\"ctx\":\"When they are done talking, the man takes a pair of silver clamps and puts it around the girl's eyebrows. after\",\"endings\":\"[', he inserts a needle into her left eyebrow, followed by the jewelry.', ', the bump is put back in and the screen fades to a black screen with black numbers across it.', ', one of the girls kneels on the ground and the other shakes her shoulders and dances.', 'the guy has done the piercing, he finishes with a flourish action art jar and starts to hug the little girl celebrating.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}