{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c8c4c3a7-a32e-59e9-ab91-08644948bb3b","task_key":"default--validation--11267","task_revision_id":"1","upstream_id":"11267","short_description":"HellaSwag validation 11267","config":"default","split":"validation","body":"{\"activity_label\":\"Skiing\",\"ctx\":\"As he skis, more men appear before he reappears by himself. Once alone, the man skis rapidly down the slopes and does spins and kneels as he skis. eventually\",\"endings\":\"[', the skier stops talking and sits up.', ', the man makes it to the last slope before another man appears on his gloves and nobody notices.', ', his story is aired and he hands off to the camera and another reporter reporters appear in the car while viewers show them all.', ', he goes to fast and ends up flipping over into a pile of snow.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}