{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a5f235ad-1f69-56ba-8c72-6c7c349fd7d8","task_key":"default--validation--6514","task_revision_id":"1","upstream_id":"6514","short_description":"HellaSwag validation 6514","config":"default","split":"validation","body":"{\"activity_label\":\"Kneeling\",\"ctx\":\"A fit man is in a mirrored room with wooden floors and is talking while he is moving on the ground and looking at the camera. the man\",\"endings\":\"['is motorinterviewing a person on the white wooden flooring as the camera watches.', 'is brushing the surface of a table and begins to comb with an electric shaver.', 'is now standing on a conference room table and it is full of silverware and bowls and halvers.', 'is bent on his knees and suddenly he jumps up to his feet, but keeps his knees bent while he squats low to the ground continuing to talk.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}