{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"a5d86ddf-1ad8-509e-9cbd-acfff14cb1ce","task_key":"default--validation--1067","task_revision_id":"2","upstream_id":"1067","short_description":"HellaSwag validation 1067","config":"default","split":"validation","body":"{\"activity_label\":\"Baton twirling\",\"ctx\":\"As the young lady continues her performance, she twirls the baton between her legs and over her head, catches it and does a gallop. the rest of the young girl's performance\",\"endings\":\"['is replayed in slow motion.', 'is a set of twirls and twist and then she runs off the floor to finish her performance.', 'is shown several times and ends by showcasing her back to the both the camera and a final focus of her performance.', 'is replayed, ending with the young lady clapping her hands joyfully.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}