{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e964b2ea-2b78-5e0f-b896-29e88c8de15f","task_key":"default--validation--12751","task_revision_id":"1","upstream_id":"12751","short_description":"HellaSwag validation 12751","config":"default","split":"validation","body":"{\"activity_label\":\"Playing violin\",\"ctx\":\"Some is carrying an instrument case walking around outside, they put the case down on the ground and take out a violin. The woman plays the violin everywhere she goes and she dances while she plays too. she\",\"endings\":\"['stops to stop and grab a change of clothes and start playing.', 'does some movements of her body making her back bend down almost touching the ground while still jamming out on the violin.', 'continues singing as she pulls the violin across as she plays, and her face appears showing awe at the sight of the instrument.', 'stops to play all around and act really silly.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}