{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"f5518f4c-29c9-5df7-aebe-ba6c92737561","task_key":"default--validation--503","task_revision_id":"2","upstream_id":"503","short_description":"HellaSwag validation 503","config":"default","split":"validation","body":"{\"activity_label\":\"Surfing\",\"ctx\":\"A sea is shown with a green forest on seashore. Blond man is standing in seashore and talking to the camera and surfing big waves on the sea. man\",\"endings\":\"['is on ocean and swimming in white water snapping his head.', 'is snowboarding up in the sea.', 'is doing flips in the waters and on the rocks and he be water surfing.', 'is walking on seashore through a lot of people and talking to the camera and holding an award and talking about surfing the seashore.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}