{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a1b72ae3-aa39-5d71-a4a5-b1c314ddca56","task_key":"default--validation--16320","task_revision_id":"1","upstream_id":"16320","short_description":"HellaSwag validation 16320","config":"default","split":"validation","body":"{\"activity_label\":\"Disc dog\",\"ctx\":\"A woman is seen standing in a field holding frisbees with a dog. The woman begins throwing around the frisbee performing tricks with the dog. she\",\"endings\":\"['throws the frisbee all around while the dog catches it by two legs.', 'throws the frisbees around while the dog runs away.', 'continues to do tricks with the dog while others watch on the side.', \\\"come running over and she's shown again in slow motion.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}