{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"8351b606-366e-58e1-adcf-d00024952493","task_key":"default--validation--2165","task_revision_id":"2","upstream_id":"2165","short_description":"HellaSwag validation 2165","config":"default","split":"validation","body":"{\"activity_label\":\"Tennis serve with ball bouncing\",\"ctx\":\"The man in brown shirt is standing next to a pile of tennis ball while he is talking to the camera. he\",\"endings\":\"['walked along a beach, talking with people, trying to court a ball.', 'started hitting the ball with his racket on his gloved hand.', \\\"pulled out two tennis rackets and toss one to the man and the other's dropped the racket in the air.\\\", 'played tennis ball with another person.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}