{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4b68eb85-4086-587f-9df5-b6b7b4fd715e","task_key":"default--validation--14553","task_revision_id":"1","upstream_id":"14553","short_description":"HellaSwag validation 14553","config":"default","split":"validation","body":"{\"activity_label\":\"Hitting a pinata\",\"ctx\":\"The man rotate the little girl and then let walk a little and the girl hit the pinata. a young man\",\"endings\":\"['from the end walked to that side to womb with the little girl, and he hit the pinata, then he grabs a stick from the gate and push the girl.', 'shake as the girl hit the pinata.', 'spins the little girl and get pattern move ability.', 'hit the pinata ad the kids are lined up to take their turns to hit the pinata.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}