{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"87f41cce-11d0-54b8-8991-8a894a86a104","task_key":"default--validation--33728","task_revision_id":"1","upstream_id":"33728","short_description":"HellaSwag validation 33728","config":"default","split":"validation","body":"{\"activity_label\":\"Hopscotch\",\"ctx\":\"A little girl in green takes her turn and then more of her friends follow after. All three girls stand and then drop a bottle cap to move forward with their jumps. they\",\"endings\":\"['skip their jumps up till all three jump into the air to finish.', 'continue to do an arm wrestling match with others who give them several shots where they are successful.', 'each do a flip that takes them to the base of their leap.', 'play different variations of the game just having fun doing so and when its over one of the girls throws her hands in the air.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}