{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"60021b72-9f49-56e5-a12c-69401aeca9cb","task_key":"default--validation--9679","task_revision_id":"1","upstream_id":"9679","short_description":"HellaSwag validation 9679","config":"default","split":"validation","body":"{\"activity_label\":\"High jump\",\"ctx\":\"The man is now running and jumping over the pole that is now 6 feet high, he clears it like he did all the other heights and once again it replays. now the man\",\"endings\":\"['he done jumping is jumping from a long rope and once again it replays itself in slow motion.', 'is back in the warm body of water again and the young lady in green releases the rope and he falls into the warm body of water once more until he lands into it with a splash and a smile.', 'rushes over and submerges his butt, while a tall one comes into frame and the curling man sliding on the roof comes to a stop.', \\\"is getting ready to jump and this one is 6'2 feet high, he clears it, lands on the padded landing, gives himself a thumbs up and walks off.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}