{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ebf77b22-742d-51f9-aabf-1cb96e4ef3cc","task_key":"default--validation--40293","task_revision_id":"1","upstream_id":"40293","short_description":"HellaSwag validation 40293","config":"default","split":"validation","body":"{\"activity_label\":\"Layup drill in basketball\",\"ctx\":\"An old man with gray hair, wearing red shirt and black pants is dribbling the ball in an indoor court, he dribbled the ball side by side, between his legs. he\",\"endings\":\"['threw the ball towards the basket, hit it, then simultaneously throw the ball into the basket and the balls splashed on the floor ,.', 'shoot the ball into the ring, then start dribbling again to side to side and between his legs, and shoot the ball into the ring.', 'then transferred the ball to a white net with two orange posts around it, lifted it up and put it on the other side and continue dribbling the ball trying to get it into the center of the court.', 'court jumped, then handed the ball to a young man standing, and the man teammate dribble the ball, then the man threw the ball second time and the line ladies and players high five, and track down the ball.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}