{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b14e1fb4-86a8-5686-8cbd-bd4a2d1d73ec","task_key":"default--validation--48592","task_revision_id":"1","upstream_id":"48592","short_description":"HellaSwag validation 48592","config":"default","split":"validation","body":"{\"activity_label\":\"Doing karate\",\"ctx\":\"A woman in white kimono is doing a karate moves with is sempai, she is standing on the side of the carpeted room. the sempai\",\"endings\":\"['go close to her to teach her the proper moves, and attack her lightly, she in return counter attack by moving forward while kicking.', \\\"'s watched as the girl slowly forms her arm's on the floor, getting ready to move to the one leg so she can hit the back of the body before the body hits the floor and disappear.\\\", 'makes a karate move in the floor with her hands.', 'is moving around on the floor smoothly and has a rubin and a white wooden kick in his back, he then stops in waist height and hands that remain on the ground.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}