{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5e0c62c2-8548-5e11-a8f8-1bfef3904490","task_key":"default--validation--23586","task_revision_id":"1","upstream_id":"23586","short_description":"HellaSwag validation 23586","config":"default","split":"validation","body":"{\"activity_label\":\"Running a marathon\",\"ctx\":\"She talks about how different the marathons used to be back in the day where she shows clips of participants from a marathon at least 50 years ago. There are pictures of male participants shown dominating the scene where she was the only female participant. she\",\"endings\":\"['kayaks through the scenic body of water to a river where the ahead rock views are surrounded by tall trees.', \\\"continues to talk about her marathon skills that are so well known around the news and films she's shown during her own marathons.\\\", 'talks about her experience as she shows more pictures of her participation against all odds.', \\\"also talks again about the extraordinary waxes she's shown running marathons.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}