{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"84dd9d01-d611-578d-b00a-c67bfe41c17a","task_key":"default--validation--21416","task_revision_id":"1","upstream_id":"21416","short_description":"HellaSwag validation 21416","config":"default","split":"validation","body":"{\"activity_label\":\"Cheerleading\",\"ctx\":\"The cheer leading team walked towards the stage, while in the audience, their teammates, who are wearing red and white uniforms are cheering. the team\",\"endings\":\"['huddle and particular ways they look for points away from them in the stadium.', \\\"started their cheer dance, where women are stood up on the the men's hands, they did pinwheels, jumped in the air, held some banners in the air, jumped high in the air and fell on their teammate's arms.\\\", \\\"started to run to where the backstage were when suddenly a black and white squad car comes in the crowd with yellow u's on the side that make the team of seven, a red team jersey and white usd's covered with glittery glittery christmas garland.\\\", 'who was wearing red, hit the putts and cheered, then they continue playing.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}