{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"a49b512c-318a-5e2c-abc5-463df764b851","task_key":"default--validation--1663","task_revision_id":"2","upstream_id":"1663","short_description":"HellaSwag validation 1663","config":"default","split":"validation","body":"{\"activity_label\":\"Ice fishing\",\"ctx\":\"Several people are seen drilling holes into the ice while looking down and speaking to the camera. Many pictures are shown of the people fishing as well as pulling fish out of a hole. more people\",\"endings\":\"['are seen swimming around in the lake and the men speak to the camera.', 'are seen having their pictures taken and speaking to the camera while water flows all around them.', 'sit by the holes and continue to pull out fish.', 'are seen drilling holes as well as swimming around in the water.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}