{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"128e8782-49b0-5e6f-971c-26abe442b595","task_key":"default--validation--5460","task_revision_id":"1","upstream_id":"5460","short_description":"HellaSwag validation 5460","config":"default","split":"validation","body":"{\"activity_label\":\"Playing harmonica\",\"ctx\":\"They are enjoying themselves as they rhythmically play their instruments and dance along to the tune. the player on the extreme right\",\"endings\":\"['is showing off his abdomen while dancers on both sides of him get distracted.', 'is playing a longer harmonica while the others are playing a smaller harmonica.', \\\"is also enjoying herself dancing with another male and she seems very into what she's doing.\\\", 'starts to struggle as he tries to get up, but they continue to soothe him with their hand movements.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}