{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"97d6a1ac-8c58-5735-bc91-98e5fa74bd9d","task_key":"default--validation--1819","task_revision_id":"2","upstream_id":"1819","short_description":"HellaSwag validation 1819","config":"default","split":"validation","body":"{\"activity_label\":\"Tennis serve with ball bouncing\",\"ctx\":\"A small boy wearing a blue and yellow hoodie and gray sweat pants in standing in an outdoor tennis court. He is holding a blue racket as he bounces the yellow and orange tennis ball on the court. he\",\"endings\":\"['starts with his first serve where the ball hits the net.', 'continues bouncing the yellow haired ball back and forth while looking out over the court passing the ball to one of the players.', 'bounces and throws the ball evenly evenly to continue the game of tennis against the two balls.', 'bounces it and does several different things to hit it.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}