{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"ff02918a-055e-542a-ab4f-46836a7b47c7","task_key":"default--validation--1626","task_revision_id":"2","upstream_id":"1626","short_description":"HellaSwag validation 1626","config":"default","split":"validation","body":"{\"activity_label\":\"Surfing\",\"ctx\":\"There are two men in the water wearing wetsuits and laying on their surfboards as they ride a wave. the camera man\",\"endings\":\"['is shorter than the men and has a shorter range.', 'is in a fast moving boat and he continues to video the surfers who are still riding the wave but now they are standing up.', 'catches one of the surfers as he slips into the goop.', 'is surfing as fast as he can on the water as the waves crescendo.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}