{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d5f76f8e-47fa-5421-b093-f5cfc6606142","task_key":"default--validation--4123","task_revision_id":"1","upstream_id":"4123","short_description":"HellaSwag validation 4123","config":"default","split":"validation","body":"{\"activity_label\":\"Waterskiing\",\"ctx\":\"A man is seen skiing behind a boat. He holds on tight as he is pulled through the water. the man\",\"endings\":\"['spins himself around the boat several times and continues skiing away.', 'is water skiing until the end of the clip.', 'gets out of the boat and skis along the water.', 'continues skiing and ends by jumping off the boat.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}