{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"e5f00d4d-9775-5ba4-9550-0fc98930e04f","task_key":"default--validation--2499","task_revision_id":"2","upstream_id":"2499","short_description":"HellaSwag validation 2499","config":"default","split":"validation","body":"{\"activity_label\":\"Ice fishing\",\"ctx\":\"They continue to pull the string while the man beside him grabs a hook. He places the hook near the hole but the man grabs a fish on a string. the fish\",\"endings\":\"['fizzles and the man drops the hook.', 'splashes into the hole bigger and bigger as the others continue to pull the string one by one.', 'flops around and the man grabs the fish and holds him up.', 'crawls out and the man catches it.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}