{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b0b9b347-56f9-598e-819c-ac5e539503aa","task_key":"default--validation--32950","task_revision_id":"1","upstream_id":"32950","short_description":"HellaSwag validation 32950","config":"default","split":"validation","body":"{\"activity_label\":\"High jump\",\"ctx\":\"A large group of people are seen standing around when several clips are shown of a man attempting to jump over a bar. the man\",\"endings\":\"['runs a large jump rope while going across an elevated bar and lands heavily on the stage before jumping down beautiful.', 'falls back onto the bar and another man does the same while people watch on the side.', 'completes the jump after a couple tries and is shown again in slow motion.', 'continues jumping all around while his friends stand around and watch and cheers him on.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}