{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"553e168a-4bcc-5767-b03e-aa0759f85188","task_key":"default--validation--44597","task_revision_id":"1","upstream_id":"44597","short_description":"HellaSwag validation 44597","config":"default","split":"validation","body":"{\"activity_label\":\"Layup drill in basketball\",\"ctx\":\"A man introduces the basketball concepts of the video to the camera. the player\",\"endings\":\"['starts from one side of the court and runs up and lays up the basketball in the net.', 'tries to make several balls, but only makes very high number of balls.', ', then sits on the end of a pallet and explains that he is demonstrating make boxing moves.', 'drops land, bounces, and lands on the green basketball court.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}