{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"08a5f4ea-473d-5f78-9e23-f03231bf1806","task_key":"default--validation--40892","task_revision_id":"1","upstream_id":"40892","short_description":"HellaSwag validation 40892","config":"default","split":"validation","body":"{\"activity_label\":\"Building sandcastles\",\"ctx\":\"There are so many awesome and different things being made out of sand, it looks like it takes a lot of time and energy. there\",\"endings\":\"['are screens explaining what to do with the sand, how you should use it, and what you should wear to do it.', 'are people walking around and looking at all of them.', \\\"are so many plants and flowers on the sand, it's unbelievable.\\\", 'are waterlog covered in sand, shingles, and areas filled with a lot of sand, sand castles shown, and miscellaneous sand on the ground and concrete.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}