{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"a54dda71-7baf-5ec0-9edc-1c602a3ceb34","task_key":"default--validation--1818","task_revision_id":"2","upstream_id":"1818","short_description":"HellaSwag validation 1818","config":"default","split":"validation","body":"{\"activity_label\":\"Tennis serve with ball bouncing\",\"ctx\":\"A small boy wearing a blue and yellow hoodie and gray sweat pants in standing in an outdoor tennis court. he\",\"endings\":\"['then jumps onto a tennis ball that is positioned in between a set of net and a larger setting of tables on which are arranged various rackets and vegetables as well as various sticks.', 'is lifting a ball over his head to throw it at a birdie on a long wooden table in front of him.', 'lobs the ball back and forth onto the court then puts his hand down to record between his legs to record more practice kicks.', 'is holding a blue racket as he bounces the yellow and orange tennis ball on the court.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}