{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a9c53ed0-2676-5800-8fe8-f58071667438","task_key":"default--validation--47923","task_revision_id":"1","upstream_id":"47923","short_description":"HellaSwag validation 47923","config":"default","split":"validation","body":"{\"activity_label\":\"Running a marathon\",\"ctx\":\"The camera pans into a man and leads into shots of him running as well as pictures of him. A camera then shows a large group of people running and the camera panning around the man. more shots\",\"endings\":\"['are shown of him performing various tricks as well as speaking to the camera and stopping to play with a dog.', 'are shown of people running and waving to the camera.', 'are shown of the people playing soccer and running around the area and ends with text across the screen.', 'are shown as well as clips from a race and people racing on a dirt bike.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}