{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5aa8f05e-c374-509b-ac72-92efb520fbdf","task_key":"default--validation--19215","task_revision_id":"1","upstream_id":"19215","short_description":"HellaSwag validation 19215","config":"default","split":"validation","body":"{\"activity_label\":\"Rope skipping\",\"ctx\":\"A large group of people are seen standing around a gymnasium with many standing in the middle holding jump ropes. the people\",\"endings\":\"['continue jumping the rope and looking to the camera while the camera always watches over their movements.', 'then begin jump roping one after the other while people stand on the sides recording their times.', 'jump over one another and end with the girls jumping the rope and cheering.', 'moving around each other pull on the ropes while the camera captures them from many sides.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}