{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a84d3cd7-1215-5604-8b9e-ad2e0b833cfc","task_key":"default--validation--6515","task_revision_id":"1","upstream_id":"6515","short_description":"HellaSwag validation 6515","config":"default","split":"validation","body":"{\"activity_label\":\"Kneeling\",\"ctx\":\"The man is bent on his knees and suddenly he jumps up to his feet, but keeps his knees bent while he squats low to the ground continuing to talk. the man\",\"endings\":\"['walked along the rocky beach and jump with the hands on his knees, jumped up in the air and also jump with the arms extended lost.', 'is still explaining the importance of holding hands with someone who has been on the gymnastics equipment for over 15 years until the video is taken actually.', 'then takes the racket, pulls the handle again and returns to the previous position in front of him only this time he takes another hit.', 'then puts his two hands on the ground, and pushes his feet back to go back on his knees and demonstrates one more time how he jumps from his knees to his toes.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}