{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8999dadd-248d-5d85-9e23-5356a080d8d5","task_key":"default--validation--7413","task_revision_id":"1","upstream_id":"7413","short_description":"HellaSwag validation 7413","config":"default","split":"validation","body":"{\"activity_label\":\"Hitting a pinata\",\"ctx\":\"The man spins the blindfolded woman around while she's holding the bat in her hand. But then she starts swinging the bat and the man moves out of the way. next\",\"endings\":\"['he walks through a storm and into the dojo dojo fighting with the other man standing in the fourth circle.', 'he flips her and holds up his hand.', ', the woman throws a small ball and they catch it and continue playing.', ', she swings the bat at the pinata and she hits the candy out of there, then she takes the blindfold off and swings the bat again getting more candy out.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}