{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"715a7de4-e4b5-59d1-89aa-6706419ba145","task_key":"default--validation--21959","task_revision_id":"1","upstream_id":"21959","short_description":"HellaSwag validation 21959","config":"default","split":"validation","body":"{\"activity_label\":\"Playing drums\",\"ctx\":\"A boy in blue shirt hit the drum, but the big drum keeps moving, then it fell. A boy with earphones is drumming and the stick got stuck in the instrument. the band\",\"endings\":\"['stopped drumming and the tall boy named cuanspwe was smiling and tapping his left foot as the drummer continue to drum.', 'is playing, while the man is singing, the drummer fell off the his chair.', 'stopped playing, one of the boys put the earphones on the drums.', 'pass the road, then cars are passing by the hill.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}