{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7660feec-bdb6-52bc-8217-7f20c6e9dd13","task_key":"default--validation--8572","task_revision_id":"1","upstream_id":"8572","short_description":"HellaSwag validation 8572","config":"default","split":"validation","body":"{\"activity_label\":\"Zumba\",\"ctx\":\"The women jump bounce in place before the lyrics of the song starts. When the lyrics start they go into an intricate dance routine that fits with the song. after the song\",\"endings\":\"['stops the women stand up and begin to walk away while the man is still playing.', 'is complete they continue dancing with each other, and stop.', 'the women continue to hop and spin while the man in the red jacket hold up a flag.', 'is over the group of winded ladies disbands from the formation.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}