{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"282af3b6-0db3-5e2b-830f-99bd4ca77ddd","task_key":"default--validation--45329","task_revision_id":"1","upstream_id":"45329","short_description":"HellaSwag validation 45329","config":"default","split":"validation","body":"{\"activity_label\":\"Playing harmonica\",\"ctx\":\"Four men in the studio are playing different instrument. The man with big guitar is singing while playing his guitar. the man playing harmonica\",\"endings\":\"['is hanging next to the man playing his guitar which started playing very rapid, the man twirled his legs with his harmonica.', 'is rapping and the other men were singing.', 'passed the scene and went and see another time.', 'stopped playing and looked at the man singing, then he continue to play his harmonica.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}