{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"561d3673-18cb-5a97-8840-c81b1c34fc13","task_key":"default--validation--10635","task_revision_id":"1","upstream_id":"10635","short_description":"HellaSwag validation 10635","config":"default","split":"validation","body":"{\"activity_label\":\"Playing beach volleyball\",\"ctx\":\"A large group of people are seen standing around a beach as well as several shots of cars and people riding bulls. various people\",\"endings\":\"['are then seen diving into the water, hitting the bulls back and fourth as well as playing a game of volleyball and cheering along.', 'then run to the bull and the bull fights them off while one stands by and watches.', 'are shown speaking to the camera and others riding bulls around one another.', 'ride the bulls and sit in the cars as well as end with a game of volleyball and celebrating.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}