{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a37e46d7-0d41-5ec7-bb1a-0a641df99487","task_key":"default--validation--26653","task_revision_id":"1","upstream_id":"26653","short_description":"HellaSwag validation 26653","config":"default","split":"validation","body":"{\"activity_label\":\"High jump\",\"ctx\":\"After, more athletes appear and they follow suit by doing high jumps and clearing the bar without scratching. as the guys\",\"endings\":\"['in a chase begins, another person who ran into them accidentally hits them and they all bow down to give remorse to the players.', 'run, they do high jumps off the bar into the water and continue running till their run is over.', 'continue successfully jumping, they jump on top of the mat and throw their hands in the air because in joy from their success.', 'continue their routine, next, a large red banner appears upside down and some of them appear sporting long hair.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}