{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f00ca9a8-a027-5523-902b-762d91e083cc","task_key":"default--validation--48176","task_revision_id":"1","upstream_id":"48176","short_description":"HellaSwag validation 48176","config":"default","split":"validation","body":"{\"activity_label\":\"Rope skipping\",\"ctx\":\"Three people are seen standing in the middle of a large gymnasium holding onto a jump rope. The people then jump around one another while people watch on the sides. they\",\"endings\":\"['continue to jump around one back and fourth on the jump ropes and end by jumping down down together.', 'end their stunts spinning around on the rope.', 'continue jumping up and down while others clap and cheer on the side and shake their hands.', 'continue swinging the rope around and end by giving a pose.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}