{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"eb38586c-3951-5f55-a323-ff27dfea99e6","task_key":"default--validation--26073","task_revision_id":"1","upstream_id":"26073","short_description":"HellaSwag validation 26073","config":"default","split":"validation","body":"{\"activity_label\":\"Layup drill in basketball\",\"ctx\":\"Young men play basketball in an indoor court and \\\"how to do a spin move in basketball\\\" overlays the video. \\\" power dribble\",\"endings\":\"[', step over swing \\\" overlay lays the video and the still of the is about to make a ground slide.', '\\\" overlays video of the players demonstrating the steps of doing it.', '\\\" children are shown from several angles as they run down the court to serve.', \\\"dribble back and forth, then dribble on the left hand to click when the ball's moved to go on the left hand side until a stripe is pushed and, the goalkeeper is perplexed.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}