{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bda057ca-3d32-5ca5-b74b-7f38d366b678","task_key":"default--validation--28465","task_revision_id":"1","upstream_id":"28465","short_description":"HellaSwag validation 28465","config":"default","split":"validation","body":"{\"activity_label\":\"Having an ice cream\",\"ctx\":\"A teenage boy is sitting at a table with his friends and begins sticking his face in his food then they all begin to laugh about it and he gets up. after\",\"endings\":\"[\\\", a group of twelve boys begin hitting each other with soccer balls and here is the child really scared, he runs the other way when the boys start to try to get his attention and he doesn't take the time to look into the camera.\\\", \\\", he grabs his brush and runs down them happy that's it's done.\\\", ', all of the boys at the table do the same thing and begin smashing their face and rubbing ice cream around their face.', 'he continues to sit, a small fruit is poured onto his face and he rubs his eyes in frustration.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}