{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"888dd77c-f689-5460-ba04-adf181223fb9","task_key":"default--validation--49492","task_revision_id":"1","upstream_id":"49492","short_description":"HellaSwag validation 49492","config":"default","split":"validation","body":"{\"activity_label\":\"Slacklining\",\"ctx\":\"A strong young man is shown on the other side of the tree pulling the line to ensure that is the tightest it can be around the trunk of the trees. then two couples\",\"endings\":\"['appear and the men are assisting females tight rope across the slack line.', 'are shown down to the cards in the and the girls begin to toss the cards.', 'begin to walk towards the camera where the filming ends with a close up of their lips.', 'are shown sitting on a chair talking and the camera taking the names of the people that are pulling the rope as well as the woman on the right next to the tree standing with her knees.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}