{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"a251299f-358d-5341-99eb-4c829f00d83c","task_key":"default--validation--888","task_revision_id":"2","upstream_id":"888","short_description":"HellaSwag validation 888","config":"default","split":"validation","body":"{\"activity_label\":\"Cleaning windows\",\"ctx\":\"A person is shown looking around a city and begins washing windows with his tools. He goes up and down the windows at a rapid pace and continues filming himself through the glass. he\",\"endings\":\"['continues to work by scrubbing the windows and then by pouring water out.', 'takes his clothes off and hangs them up before throwing his plaid shirt into the laundry.', 'finishes the glass in record time and turns off the camera.', 'spins around to show the different angles of the board and shows pictures of the house in the end.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}