{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5f17d892-883b-58b3-bbd9-0c75a4a34540","task_key":"default--validation--28574","task_revision_id":"1","upstream_id":"28574","short_description":"HellaSwag validation 28574","config":"default","split":"validation","body":"{\"activity_label\":\"Vacuuming floor\",\"ctx\":\"Several people are walking down a subway and the train takes off. once there\",\"endings\":\"[', they begin to fall off their sides when a person comes along and paddles them back to the surface.', ', several more clips are shown of men falling down throwing rocks as well as running to retrieve them and speaking to one another.', 'are less people, the man walks in and plugs up a vacuum and begins to clean.', ', people are riding sleds and going down busy roads.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}