{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8ba90ea6-cf26-5ef6-93c1-c349878c984e","task_key":"default--validation--44599","task_revision_id":"1","upstream_id":"44599","short_description":"HellaSwag validation 44599","config":"default","split":"validation","body":"{\"activity_label\":\"Layup drill in basketball\",\"ctx\":\"The player starts from one side of the court and runs up and lays up the basketball in the net. He repeats around the court from every angle. the man\",\"endings\":\"['returns to his sixteenth attempt and gets a first rate layup of 4-1.', \\\"tails the other player with it when it's the rebound.\\\", 'throws the ball back down the court and runs to make the serve.', 'returns to go over the concepts again as the video closes.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}