{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"3c7db32e-5b8e-53e2-9964-9a9bbf811c28","task_key":"default--validation--2477","task_revision_id":"2","upstream_id":"2477","short_description":"HellaSwag validation 2477","config":"default","split":"validation","body":"{\"activity_label\":\"Rollerblading\",\"ctx\":\"There's a woman with red colored hair wearing a black tank top and printed black tights skating on roller blades through the streets of a city. the streets\",\"endings\":\"['showed various advertisements advertised for skate skating.', 'are covered hip hop and dressed in purple and red leotards as the individual rhythmically spins around.', 'are filled with crowds, people and cars, and she is wearing a white and orange outfit.', \\\"don't have any cars but people are riding bikes or skateboarding.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}