{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"69aab15f-d6a0-5003-99a9-a384175acacc","task_key":"default--validation--46472","task_revision_id":"1","upstream_id":"46472","short_description":"HellaSwag validation 46472","config":"default","split":"validation","body":"{\"activity_label\":\"Layup drill in basketball\",\"ctx\":\"A second man in white throws a ball at the first man in white out of camera frame. the first man\",\"endings\":\"['out of camera frame throws a ball towards the second man in red slacks, who in black pants throws the ball back at the first man.', 'throws two balls in camera frame, and then two other men in blue play dodge ball.', 'in white runs in and makes a layup shot while the second man retrieves the ball.', 'throws another ball, the man in black throws a ball that makes the first man run off of camera frame.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}