{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d870740b-8003-54c7-84c1-6ace053f8462","task_key":"default--validation--22672","task_revision_id":"1","upstream_id":"22672","short_description":"HellaSwag validation 22672","config":"default","split":"validation","body":"{\"activity_label\":\"Waterskiing\",\"ctx\":\"We see a short flashback as we then see him in the boat in then in the water being assisted with his skis. once\",\"endings\":\"[\\\"the v-jet is inflated we see he is having a hard time tying the boat down now he's out of rope and tied down.\\\", 'he is successfully tied up, the man lowers his ski in the water and and we see the rest of the video.', 'again he is in the water, and the boat slowly picks up speed, pulling the young man upright until he is skiing full-speed behind the boat.', 'he has his skis on, he grabs his in the boat and skis behind a group lined up.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}