{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6d46ccd0-1c5f-56c5-9829-af7e77cee1a0","task_key":"default--validation--43256","task_revision_id":"1","upstream_id":"43256","short_description":"HellaSwag validation 43256","config":"default","split":"validation","body":"{\"activity_label\":\"Discus throw\",\"ctx\":\"A black screen appears again with red and white words that say \\\"other similar exercises on the same idea: focus on the low point of the implement trajectory. A boy is now standing on the concrete circle and he's practicing his swings with a shirt. a man\",\"endings\":\"[\\\"is now indoors and he's practicing his swings with a golf club.\\\", 'wearing a neatly pressed blue shirt picks up the pace and continues to swing at the swing.', 'is then shown teaching the learned to swing the tried and fail swing and he is shown farther back behind in the weightlifting attempt, swinging the lot of his arms, doing more over and over.', 'shows him how to properly demonstrates and how to properly get the swing to move.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}