{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0fc6f053-28dd-5dc5-9396-7fd926337318","task_key":"default--validation--11200","task_revision_id":"1","upstream_id":"11200","short_description":"HellaSwag validation 11200","config":"default","split":"validation","body":"{\"activity_label\":\"Doing karate\",\"ctx\":\"The girl ends her routing a little further from the point she started and firmly drops her hands to her side and does a little bow, then opens her legs shoulder length apart and goes back into the same position as she was in when she started. after she\",\"endings\":\"['circles a couple times several more times, there is still no sign of her ending her gardingaing.', 'is done holding the position for a while the girl relaxes and then walks away.', 'finishes, the girl moves closer to the camera on her knees while holding a cylindrical object, and then straightens her legs out and stops her bow.', 'is done with her camera, she begins talking to the camera again and looking at the screen before finishing.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}