{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9d145365-9c9f-5797-a5a1-3b3fb40abba2","task_key":"default--validation--19989","task_revision_id":"1","upstream_id":"19989","short_description":"HellaSwag validation 19989","config":"default","split":"validation","body":"{\"activity_label\":\"Paintball\",\"ctx\":\"People speaks in the tv, and a crowd appears covered the face and holding firearms. Mr kissinger and other man speaks, and then the crowd walks on an unpaved road. after\",\"endings\":\"[', president kennedy talks, then the crow start to run and shoot.', ', a person come in front of the woman and shoot a black calf and receive tattoos.', ', two men is in beach and the guy says something guys around him do other things as red board.', ', the man holds his hands in crowdedness.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}