{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ef787e32-4451-5b03-a676-3161ae0b9f9d","task_key":"default--validation--35972","task_revision_id":"1","upstream_id":"35972","short_description":"HellaSwag validation 35972","config":"default","split":"validation","body":"{\"activity_label\":\"Doing crunches\",\"ctx\":\"Once she has secured herself on the bag, she begins doing a set of crunches by pulling herself up. in\",\"endings\":\"['the long stripe there is another goal of throwing the ball down the track.', 'the end, she arrives to the top of the slide and glides down looking off into the distance.', 'between the crunches, she sits up and makes punches out into the air, before going back down.', 'between, she explores a few different sponsors before she continuously does crunches.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}