{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4996ca27-7a05-5284-9a18-7ba12e6b1c48","task_key":"default--validation--40606","task_revision_id":"1","upstream_id":"40606","short_description":"HellaSwag validation 40606","config":"default","split":"validation","body":"{\"activity_label\":\"Gargling mouthwash\",\"ctx\":\"She takes a sip of listerine mouth was from her cup and holds it in her mouth for several seconds while her older brother video tapes her. Another little girl walks behind her and watches her rinse. she\",\"endings\":\"['takes another sip of her face wash and then holds it over her mouth and spits it out.', 'takes another sip of her listerine mouth and sets the cup down.', 'holds the mouth wash in her mouth for a long time and then spits it out in the sink.', 'mixes each dish then makes a salad.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}