{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f97ce7cc-b822-52e1-9491-275c87a621c7","task_key":"default--validation--47821","task_revision_id":"1","upstream_id":"47821","short_description":"HellaSwag validation 47821","config":"default","split":"validation","body":"{\"activity_label\":\"Putting on makeup\",\"ctx\":\"This woman is shown putting on her makeup while looking in the mirror. First she makes her eyes up and winks at the camera. then she\",\"endings\":\"['applies lip gloss and puts a bow around her face.', 'says something and turns back around to do what she was doing.', 'moves her hair out of her way so that she can continue her dancing.', 'puts mascara on her eyelashes and two small heart shaped lenses.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}