{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1d284d8d-e096-5478-a198-f3f641bb6150","task_key":"default--validation--18148","task_revision_id":"1","upstream_id":"18148","short_description":"HellaSwag validation 18148","config":"default","split":"validation","body":"{\"activity_label\":\"Washing face\",\"ctx\":\"A young woman talks and two girls appear for few seconds splashing water on their face then she continues talking. the girl\",\"endings\":\"['is on front a sink and puts soap on her face, then she remembers the two girls splashing water on her face.', 'in black is acting inappropriately and fuelling an angry fire in her eyes which she then run away from as the other girls turn her in their direction to dodge the brush while dancing.', 'in front is now standing and mages chest is showing how to lift weights.', 'walks to the sink and rinses her face while holding her face.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}