{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"69772cad-b09f-5c89-ac74-49505aa891eb","task_key":"default--validation--247","task_revision_id":"2","upstream_id":"247","short_description":"HellaSwag validation 247","config":"default","split":"validation","body":"{\"activity_label\":\"Cheerleading\",\"ctx\":\"They get into formation, then begin dancing and flipping as male cheerleaders join them. They all continue dancing and flipping, doing hand springs. when they\",\"endings\":\"[\\\"are done, they shake hands and walk away, but cheers are heard all around as they're finished.\\\", 'are finished, they jump up and down happily and walk off the stage, exhausted.', 'are done, a male cheerleader is crowned and an instant replay is shown.', 'come down from their workout, they all stop to bow and applaud, then they continue dancing.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}