{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e28c1029-83bc-5a9b-a6a2-6f3d54b65753","task_key":"default--validation--32471","task_revision_id":"1","upstream_id":"32471","short_description":"HellaSwag validation 32471","config":"default","split":"validation","body":"{\"activity_label\":\"Surfing\",\"ctx\":\"An intro leads into clips of people riding surfboards on the water as well as jet skis. the camera\",\"endings\":\"['follows the girls as they slip and slide through rough waters and stop to do sunscreen on take breaks.', 'pans around the area and continues to show more clips of people surfing.', 'is panned around to show a wooden security fence along side the water.', 'captures several people riding tubing in the water while people stand in a long line and watch.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}