{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"20b41415-da69-5016-a19d-8706dde7c7e6","task_key":"default--validation--30601","task_revision_id":"1","upstream_id":"30601","short_description":"HellaSwag validation 30601","config":"default","split":"validation","body":"{\"activity_label\":\"Waterskiing\",\"ctx\":\"A black screen appears and the yellow words are displayed on the screen and read \\\"the disk\\\", and once again the brunette woman is speaking while standing by the water. the screen\",\"endings\":\"['eventually fades to black and a potted plant with leaves has one of its branches fallen to the floor.', 'fades to a black screen and the woman is on the beach while stirring the cocktail, and begins to talk.', 'then appears and the website it contains is then shown with the website logo and some more comments including \\\" made public sensation cock block being made \\\" along with an ad for a start date of if you are interested \\\".', 'then goes to black and displays their website in yellow letters.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}