{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5f6ccd0a-bf66-5c6f-94d3-b80ee75092aa","task_key":"default--validation--13255","task_revision_id":"1","upstream_id":"13255","short_description":"HellaSwag validation 13255","config":"default","split":"validation","body":"{\"activity_label\":\"Long jump\",\"ctx\":\"A large group of people are seen standing and sitting around a sand pit while a runner jumps inside it. the man\",\"endings\":\"['is shown several times jumping with people measuring his jump afterwards and raking the sand.', 'falls off and another athlete steps up and plays with the men.', 'then runs around another man jumping in and playing with a sand pit and then jumps outside performing various different moves and techniques.', 'then stands and begins running with the rope and one chases after him.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}