{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"58930b66-1d9a-599b-a209-628112e5768c","task_key":"default--validation--3662","task_revision_id":"1","upstream_id":"3662","short_description":"HellaSwag validation 3662","config":"default","split":"validation","body":"{\"activity_label\":\"Playing violin\",\"ctx\":\"The violinist dances as she plays the instrument along with five other dancers. They dance in a modern and hip hop style of dancing. then only two of the dancers\",\"endings\":\"['talk and one dancer lightly dances.', 'stay on the stage with the violinist and dance in a salsa style.', 'join her as she continues playing to both the camera and the audience.', \\\"are left to dance and sit together while they cover themselves with their capes and join in the dj's rendition of the fountain of youth.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}