{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ae078e18-99af-5736-a0d5-25c2c1c0c7c2","task_key":"default--validation--24891","task_revision_id":"1","upstream_id":"24891","short_description":"HellaSwag validation 24891","config":"default","split":"validation","body":"{\"activity_label\":\"Playing squash\",\"ctx\":\"As he is talking, several clips appear in between him where he is in a room with three walls and glass as the fourth wall while playing wall ball. the final scene\",\"endings\":\"['is shown, and the male comes back talking before finally flashing a url at then end.', \\\"is of him punching the ball with his right hand as he's doing moves with the left hand.\\\", 'is shown of him kneeling in the grass.', 'in question is of a fighter splits and while playing, he hits the ball back to his entrance and lands on the bed.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}