{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a5cec007-95d6-5490-8aa0-cf82ef6fcae8","task_key":"default--validation--49812","task_revision_id":"1","upstream_id":"49812","short_description":"HellaSwag validation 49812","config":"default","split":"validation","body":"{\"activity_label\":\"Playing harmonica\",\"ctx\":\"A man is sitting at a chair inside of his home all by his lonesome. He has a harmonica in his hand which he begins to play a song on. he\",\"endings\":\"['hits the harmonica line and leads into him playing it again and a still of him playing it in turn.', 'is struggling throughout the song as he tries hard to to keep his composure.', 'is very into this song that he is playing, he sits there very concentrated just playing his song.', 'progresses along the tune as he plays and moves his hands up and down the keys intently.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}