{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"03a23c3e-d7d4-5c5e-993a-342f5c7df9ca","task_key":"default--validation--47178","task_revision_id":"1","upstream_id":"47178","short_description":"HellaSwag validation 47178","config":"default","split":"validation","body":"{\"activity_label\":\"Playing harmonica\",\"ctx\":\"Then, the person stops playing and talks. After, the person continues playing while marking the compass of the music with his hands. next\",\"endings\":\"[', the person play on his front feet, then turns to look at the camera and plays until the end.', ', the person zooms in the manmade instruments, then with his eyes side to side, then they play again.', ', the person stops playing and talks.', \\\", the person fill water holes with lime water and place it in the water, then cleans the player's marks with a towel.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}