{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"ef19f351-caae-57d8-85e3-8a17f4964b9a","task_key":"default--validation--1628","task_revision_id":"2","upstream_id":"1628","short_description":"HellaSwag validation 1628","config":"default","split":"validation","body":"{\"activity_label\":\"Surfing\",\"ctx\":\"The surfer who is in front is happy to riding the wave for so long that he raises his two arms up in the air and then lowers it back down. the surfer in the back\",\"endings\":\"['kept one arm raised up and the surfer in front of him raise his left arm.', 'wave has a big wave almost bigger than he is paddling, and this makes it pretty fun because it causes the surfer riding the wave to put up a lot of resistance so he just does it and crash onto the waves making the wave even bigger then it was.', 'does not have to go as far as he needed to to go as he is ready to go.', 'begins to lose balance and falls off of his board while the man in the front is still standing up.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}