{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"eb4e05a1-7109-5714-98bf-f2e9101eeeb3","task_key":"default--validation--14845","task_revision_id":"1","upstream_id":"14845","short_description":"HellaSwag validation 14845","config":"default","split":"validation","body":"{\"activity_label\":\"Doing fencing\",\"ctx\":\"When they are done fighting the women are shown standing close together, smiling, and the woman on the left has her arm resting on the shoulder of the woman on the right. the outro\",\"endings\":\"['screen appears with a banner at the bottom, \\\" chor 20010\\\\'s abbiya #224 amazon \\\".', 'appears with an african language covered in purple lines and both begin to talk to each other.', 'is a white screen with multiple colors on it that include logos, website and the name of the event.', 'leads into two hockey players bare arm wrestling a woman and reeling her in and the crowd is cheering.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}