{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"401a7ad1-d2f7-5319-a07a-95db0a1e5c1a","task_key":"default--validation--38345","task_revision_id":"1","upstream_id":"38345","short_description":"HellaSwag validation 38345","config":"default","split":"validation","body":"{\"activity_label\":\"Swimming\",\"ctx\":\"Several girls are then shown deep underwater doing a shimmy motion and then back stroking in the pool. one female\",\"endings\":\"['then begins breast stroking, waves to the camera and then gets out.', 'is now dipped underneath the water and water is being pumped out of her mouth thoroughly.', \\\"steps on while another kneels and flips to her back on the house while a male holds the woman down so she doesn't fall and hurts herself.\\\", 'is joined by several others and they continue to email to various websites.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}