{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"cc8d3deb-8620-502c-8b26-70928b0509eb","task_key":"default--validation--1758","task_revision_id":"2","upstream_id":"1758","short_description":"HellaSwag validation 1758","config":"default","split":"validation","body":"{\"activity_label\":\"Mooping floor\",\"ctx\":\"Another boy attempts to walk through where he is mopping. The man takes a drink of beer and complains to the camera. he\",\"endings\":\"['gets back in the raft to go back to the store.', 'continues to mop mopping the floor.', 'mops through the room in between the people.', 'continues scrubbing the mop with another spray and the boy walks back.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}