{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7807321f-fa4a-51fe-a6a4-39a3df0d586c","task_key":"default--validation--16717","task_revision_id":"1","upstream_id":"16717","short_description":"HellaSwag validation 16717","config":"default","split":"validation","body":"{\"activity_label\":\"Rope skipping\",\"ctx\":\"There's a lot of children and adults in a large indoor gymnasium with a wood floors and some are rope jumping while others are watching while sitting or standing. A brown skinned girl in the forefront begins to jump rope very quickly. as\",\"endings\":\"['the boy slows down, he jumps the highest but then falls because he let go of the rope and his left leg gets injured in the fall too and is released.', \\\"soon as she's done a white skinned girl next to her begins her solo jump rope as the two girls on each side of her watch her.\\\", 'soon as the girls are done jumping, they are given hugs by a man who holds one of their belts while they jump rope and signals with another hand.', \\\"she gets closer to the ground stage the other children begin to jump rope and then the kid at the rear begins to jump rope as well and near the backs of the field, but he gets a c and doesn't even make it to the knees.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}