{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"334032fe-ff5b-58a6-a34d-d7612bd14251","task_key":"default--validation--5182","task_revision_id":"1","upstream_id":"5182","short_description":"HellaSwag validation 5182","config":"default","split":"validation","body":"{\"activity_label\":\"High jump\",\"ctx\":\"The guy seems to be practicing how high he can jump by jumping over the pole many times, each time a person raises the pole higher and higher to see if the guy can jump over it. One jump the guy knocked the pole down when he tried to hump over it. the guy\",\"endings\":\"['knock the pole down six time out of all his jumps.', 'jumps and does several tricks over the pole, trying to do better than the other people who did it.', 'jumps over the pole but does not get over it.', 'try to balance far too far and he hits the dirt catching himself next to other guy.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}