{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4f6a1f97-4028-5a37-b5a2-57a4861affca","task_key":"default--validation--44685","task_revision_id":"1","upstream_id":"44685","short_description":"HellaSwag validation 44685","config":"default","split":"validation","body":"{\"activity_label\":\"Hula hoop\",\"ctx\":\"The girl spins around with the hula hoop moving from her feet to her knees as well as the rest of her body. at one point\",\"endings\":\"[\\\", the girl closes her eyes and begins waving her arms up and showing what she's capable of doing.\\\", 'she throws the hula hoop accidentally but picks it back up and continues spinning with it.', 'after we capture the spinning, she turns around to two different obstacles that she had previously avoided and does an impressive somersault.', 'the girl changes her stance and flips and throws her hands in the air before spinning back to the object.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}