{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bfc58082-2a71-5445-a65b-6f3168959226","task_key":"default--validation--34366","task_revision_id":"1","upstream_id":"34366","short_description":"HellaSwag validation 34366","config":"default","split":"validation","body":"{\"activity_label\":\"Playing harmonica\",\"ctx\":\"He plays rhythmically as he moves the harmonica rapidly through his mouth to play a fast tune. He continues playing the tune as he closes his eyes in deep concentration. he\",\"endings\":\"['is now singing as he plays with hands in lower order to play as he moves his mouth up and down fast.', 'then puts the harmonica down as his facial expression changes.', 'moves the harmonica side ways as he continues playing till he comes to an end.', 'stops the song and speaks to the camera.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}