{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e087710c-ff21-5a08-8e24-3492db6b4f49","task_key":"default--validation--28614","task_revision_id":"1","upstream_id":"28614","short_description":"HellaSwag validation 28614","config":"default","split":"validation","body":"{\"activity_label\":\"Futsal\",\"ctx\":\"Two teams play soccer in an outdoor field in front of an audience, the black and white team makes a goal and celebrates by smiling and clapping each others hands and waiving at the audience. several more goals\",\"endings\":\"['are made by the black and white team, eventually one of the players wins the game and he gets congratulated by a white lady with blonde hair and her team mates, as is her trend.', 'are made and even two teams continues to rally opposing teams on important events such as soccer.', 'are made and by the end of each of them a score is made and people clap while walking away.', 'are made by the black and white team on the soccer field before the scene fades.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}