{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"24a756e9-7f9b-5f02-b643-7f1074851aa4","task_key":"default--validation--40788","task_revision_id":"1","upstream_id":"40788","short_description":"HellaSwag validation 40788","config":"default","split":"validation","body":"{\"activity_label\":\"High jump\",\"ctx\":\"We see a lady from china run and perform a high jump and see her score and we see her manager and a slow motion. An different lady from uzbekistan performs a high jump and we see her score. the first lady\",\"endings\":\"['jumps hard in fifteen seconds and scores, we see the closing screen.', 'runs off the mat and trampolines and shakes her pom poms right and left.', 'stands in front of the stage and smiles and says something.', 'performs again and knocks the bar over and looks disappointed.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}