{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b0ecb1a3-aa23-59c0-8c14-59dfcd7985d3","task_key":"default--validation--12712","task_revision_id":"1","upstream_id":"12712","short_description":"HellaSwag validation 12712","config":"default","split":"validation","body":"{\"activity_label\":\"High jump\",\"ctx\":\"An athletic man is seen standing ready in front of a large crowd and then runs down the track and jumps over a beam. the man\",\"endings\":\"['springs back into the air and lands a high jump into another mans arms in slow motion while the audience claps and the man stands and walks away.', 'is seen again a bit more and continues running down the track while people clap and the man bounces some more.', 'celebrates with other people and his same jump is shown several times in slow motion.', 'then leaps into the air performing jumps and tricks on the large stage before tossing his arms in the air and cheering to the audience.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}