{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f40f8452-f4f4-5d9b-8388-5a896d86def9","task_key":"default--validation--47820","task_revision_id":"1","upstream_id":"47820","short_description":"HellaSwag validation 47820","config":"default","split":"validation","body":"{\"activity_label\":\"Putting on makeup\",\"ctx\":\"This woman is shown putting on her makeup while looking in the mirror. first she\",\"endings\":\"['begins to apply some more makeup and make eye shadow to her lashes to cover the marks.', 'makes her eyes up and winks at the camera.', 'puts foundation on her cheeks and then she places lip sticks over it.', 'takes her hair out and puts it in a ponytail.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}