{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e6646931-4f3b-51fb-9d9f-cfaf15d55e0b","task_key":"default--validation--39875","task_revision_id":"1","upstream_id":"39875","short_description":"HellaSwag validation 39875","config":"default","split":"validation","body":"{\"activity_label\":\"High jump\",\"ctx\":\"An african american female takes off running and begins to attempt a high jump but fails when the bar drops and walks off. Another female shows up and tries her third attempt and the same thing happens. after\",\"endings\":\"[', another female with a large assortment of tattoos tries and she clears the bar and comes in first place on her first attempt and a replay is shown.', ', the girl makes several unsuccessful attempts and finally has her third attempt.', 'over 400 attempts high jumps her young girl stops and begins to walk off the bar.', ', a second female throws six balls of green paint and heads towards shots as the males walk away.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}