{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"22988a62-ef56-530e-83ff-df2fbc565ae3","task_key":"default--validation--942","task_revision_id":"2","upstream_id":"942","short_description":"HellaSwag validation 942","config":"default","split":"validation","body":"{\"activity_label\":\"Washing face\",\"ctx\":\"Two children are seen sitting in a tub rubbing their faces and another person's hands interacting. the boy\",\"endings\":\"['continues to rub water all over his face as well as play with toys and smile at the cameraman.', 'walks out of the bathroom laughing to the camera and the girl stands up to rinse her face off.', 'then begins washing the other child and ends by copulating with him.', 'is now seen pointing down by himself and leads into him holding a comb and blow dryer and speaking to others.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}