{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"dcc98791-fdfe-5cd2-8c44-27565b11adbd","task_key":"default--validation--606","task_revision_id":"2","upstream_id":"606","short_description":"HellaSwag validation 606","config":"default","split":"validation","body":"{\"activity_label\":\"Beer pong\",\"ctx\":\"One of the ping pongs lands in the cup and one of the boys begins to drink the beer. the group\",\"endings\":\"['of teens is sailing down the river with others sailing in the background.', 'gets up together and poses with their vests to celebrate.', 'continue to play beer pong when one final player is able to throw and land a ping pong ball while standing with his back to the cups at which point the boys celebrate, and laugh.', 'does a trick and the liquid is poured on a second table.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}