{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2c7f5220-c4dd-5600-a922-9a3d57317713","task_key":"default--validation--8571","task_revision_id":"1","upstream_id":"8571","short_description":"HellaSwag validation 8571","config":"default","split":"validation","body":"{\"activity_label\":\"Zumba\",\"ctx\":\"A dance team dressed in black with pink letters shows a routine put to reggae music. The women jump bounce in place before the lyrics of the song starts. when the lyrics\",\"endings\":\"['start they go into an intricate dance routine that fits with the song.', 'change, they are all standing preparing the floor.', 'are done, they smile and cheer over each other.', 'finally fade the cheerleaders enter the pit, make a pile and cheer continuously.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}