{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6841a576-e308-513c-8884-cd451868511e","task_key":"default--validation--23587","task_revision_id":"1","upstream_id":"23587","short_description":"HellaSwag validation 23587","config":"default","split":"validation","body":"{\"activity_label\":\"Running a marathon\",\"ctx\":\"There are pictures of male participants shown dominating the scene where she was the only female participant. She talks about her experience as she shows more pictures of her participation against all odds. she\",\"endings\":\"['eventually walks off the set.', 'also shares information on how things changed later when she was finally allowed to participate freely.', 'talks about her experience again in waking and reading.', 'challenges her opponents with her stance and high kick, doing incredible falls on the mat in the process.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}