{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4c69cc21-607c-5816-a93d-be1b772d4f65","task_key":"default--validation--32951","task_revision_id":"1","upstream_id":"32951","short_description":"HellaSwag validation 32951","config":"default","split":"validation","body":"{\"activity_label\":\"High jump\",\"ctx\":\"A large group of people are seen standing around when several clips are shown of a man attempting to jump over a bar. The man completes the jump after a couple tries and is shown again in slow motion. the man\",\"endings\":\"['runs into one more man near a bar still jumping in slow motion while people watch on the side and tower over him.', 'attempts jumping higher several more times and ends up knocking the bar down each time.', 'is shown again in slow motion jumping how many times high up on the bar.', 'then jumps off the who and is shown again ending with him having various drink courses and doing flips.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}