{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d8e384e1-8f84-59c2-a00b-01a184626d6f","task_key":"default--validation--31119","task_revision_id":"1","upstream_id":"31119","short_description":"HellaSwag validation 31119","config":"default","split":"validation","body":"{\"activity_label\":\"Dodgeball\",\"ctx\":\"A man is seen standing around with a large group of kids and leads into the kids play dodgbeall in a large gymnasium. a woman\",\"endings\":\"['walks in and out of frame and leads into them playing dodgbecata in another room.', 'sits a boy on a table and then a girl goes next the kids playing dodgnog and two again fall in.', 'is seen speaking to the camera while more shots of the game are shown and the man speaking to the kids.', 'in black does a back flip off a horse to the forefront of the kids and a guy is seen holding it up.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}