{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d6997ccd-8421-5f16-ac23-b775b06484fc","task_key":"default--validation--23585","task_revision_id":"1","upstream_id":"23585","short_description":"HellaSwag validation 23585","config":"default","split":"validation","body":"{\"activity_label\":\"Running a marathon\",\"ctx\":\"She talks about how different the marathons used to be back in the day where she shows clips of participants from a marathon at least 50 years ago. there\",\"endings\":\"['are also clips where she talks about the benefits of long distance running while taking deep breaths as she who smokes the cigarette repeatedly.', 'are pictures of male participants shown dominating the scene where she was the only female participant.', 'is a little pink skater girl who is wearing a white sweatsuit.', 'are more clips of marathoners and marathoners participating in everything from rafting down to a walk.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}