{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"68028f3d-01b3-59e0-afb3-5e6ea0d8d2f4","task_key":"default--validation--8909","task_revision_id":"1","upstream_id":"8909","short_description":"HellaSwag validation 8909","config":"default","split":"validation","body":"{\"activity_label\":\"Ice fishing\",\"ctx\":\"A man is seen sitting on the ice speaking to the camera and leads into several shots of him grabbing fish from a pole. the men\",\"endings\":\"['presents the fish to the camera as well as speak to the camera and continue to grab and hold up fish.', 'then grab rocks and fish around in the ocean while people watch on the sides.', 'then are seen skinning a fish and holding back food.', 'hold clips around a camp site all while holding trophy in the end followed by pictures of a trophy.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}