{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b551ff18-48fc-579d-b3ac-edf312550e8d","task_key":"default--validation--16170","task_revision_id":"1","upstream_id":"16170","short_description":"HellaSwag validation 16170","config":"default","split":"validation","body":"{\"activity_label\":\"Ice fishing\",\"ctx\":\"The action of the lures is shown underwater as several different fish go after the lures. The men are also shown above the ice working their poles and catching several fish. the video\",\"endings\":\"['continues to show other fish swimming around the sport on the ice.', 'ends with the closing credits and graphics shown on the screen.', 'ends with the crowd applauding.', 'ends with the closing faces of the men and smiling to the camera.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}