{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c6dedf46-42f6-5cd6-b247-6b457ab7249e","task_key":"default--validation--9680","task_revision_id":"1","upstream_id":"9680","short_description":"HellaSwag validation 9680","config":"default","split":"validation","body":"{\"activity_label\":\"High jump\",\"ctx\":\"Now the man is getting ready to jump and this one is 6'2 feet high, he clears it, lands on the padded landing, gives himself a thumbs up and walks off. the man\",\"endings\":\"[\\\"runs and now he's jumping 6'4 feet high but unfortunately he knocks the pole off and as soon as he realizes it he gets up and is very disappointed in himself.\\\", 'rises again and runs to the tree and jumps between the jump ropes and lands on a red border trampoline and does a flip.', 'is just walking his board as his excitement gets the better of him and signals the start of his jump.', \\\"smiles while he's jumping through the air, grabs a pose and goes to the girls, flips and lands and then walks away.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}