{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8a68c78f-d4e7-5798-8bd1-e3c0d0b554e5","task_key":"default--validation--38346","task_revision_id":"1","upstream_id":"38346","short_description":"HellaSwag validation 38346","config":"default","split":"validation","body":"{\"activity_label\":\"Swimming\",\"ctx\":\"Several girls are then shown deep underwater doing a shimmy motion and then back stroking in the pool. One female then begins breast stroking, waves to the camera and then gets out. two other girls\",\"endings\":\"['then stand side by side on the outside of the pool and jump in together and race to the other side of the pool.', 'afterwards go in the water to go clothes and with the same technique.', 'begin breastpooing and the end results of the performance is shown.', \\\"appear from below and the one with the ramen stick then jumps in the water, holds the written testimonials and both give the commentary of one of them who's underwater.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}