{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"8e776c2e-55ef-5176-bb53-e6f3de1d94fd","task_key":"default--validation--1066","task_revision_id":"2","upstream_id":"1066","short_description":"HellaSwag validation 1066","config":"default","split":"validation","body":"{\"activity_label\":\"Baton twirling\",\"ctx\":\"The girl begins to incorporate her arms and does a flip and continues to flip the baton over her hands and arms. as the young lady\",\"endings\":\"['continues her performance, she twirls the baton between her legs and over her head, catches it and does a gallop.', 'is still serving the baton, she places the extension on her left hand and does a full 180 degree spin in one flip.', 'becomes more involved in the ballet, she does a final dismount to bring her elbows down and her upper body straight.', 'is zapping her arms and is practicing her baton twirling technique, she grabs the baton from the ground and shows herself victorious while her coach claps.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}