{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e30f29a8-29b5-5d58-b5dc-6f314df32f90","task_key":"default--validation--33397","task_revision_id":"1","upstream_id":"33397","short_description":"HellaSwag validation 33397","config":"default","split":"validation","body":"{\"activity_label\":\"Ice fishing\",\"ctx\":\"Some booth serve hot dogs to the people as they ice fish. A little boy and girl attempt to catch a fish. they\",\"endings\":\"['catch a fish and keep swimming around.', 'are interviewed as they fish.', 'try repeatedly, wandering them all the way next to their fish.', 'are quickly caught in the fast current as they kick and push against the bank of the river and paddle away from a slight fish infesting the water.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}