{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fc2d8194-6428-5bd7-9eb5-e79ce2d02861","task_key":"default--validation--9173","task_revision_id":"1","upstream_id":"9173","short_description":"HellaSwag validation 9173","config":"default","split":"validation","body":"{\"activity_label\":\"Futsal\",\"ctx\":\"Various shots are shown of an indoor stadium followed by two men performing soccer steps around one another. the men\",\"endings\":\"['continue pushing around one another to find other people to play against.', 'continue to move around one another performing various high kicks and punches to one another and ends with them jumping down and bowing.', 'continuously demonstrate how to properly perform soccer moves while the camera captures them from many angles.', 'perform near one another continuously while one man falls and another hits him.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}