{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d9d25ab1-3627-57bf-ab4e-ccd99797a3d7","task_key":"default--validation--14250","task_revision_id":"1","upstream_id":"14250","short_description":"HellaSwag validation 14250","config":"default","split":"validation","body":"{\"activity_label\":\"Tennis serve with ball bouncing\",\"ctx\":\"A man dressed in a black sweatshirt and black shorts, along with another person dressed in white athletic wear are standing in a tennis court near the net. They are both holding tennis rackets. the person in black\",\"endings\":\"['opens the accordion, inserts a blue string into the racket and begins to play.', 'is playing racket ball with the first player.', 'easily snaps the tennis ball into the court, then the person in black walks over to the man dressed in a black tank top volleyball net tennis clothes.', 'is giving a tutorial on how to return a serve in tennis.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}