{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"937e4633-5be1-5551-85c4-60fb3d74f90b","task_key":"default--validation--36125","task_revision_id":"1","upstream_id":"36125","short_description":"HellaSwag validation 36125","config":"default","split":"validation","body":"{\"activity_label\":\"Washing face\",\"ctx\":\"The baby is looking at the camera and then the hand wiped his face. the hand\",\"endings\":\"[\\\"wiped the baby's face and the baby opened his mouth.\\\", 'start wrapping a gift the hand moved down to the bottle and puts some tissue paper on the top part.', 'lays on the table with the baby on it and then the little girl hits the camera again.', 'talked and raised its hand.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}