{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f37f8176-0837-5a4f-9c40-21673e9a8289","task_key":"default--validation--49509","task_revision_id":"1","upstream_id":"49509","short_description":"HellaSwag validation 49509","config":"default","split":"validation","body":"{\"activity_label\":\"Windsurfing\",\"ctx\":\"A person is seen kite surfing along the water and eventually falling in. he\",\"endings\":\"['speaks to the camera and adds more sand into a bucket in front of him while surfing.', 'floats along the water while the camera captures him from several angles.', 'climbs back on the board and continues riding along while the camera pans out.', 'paddles alongside a student guide who is shown from several angles.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}