{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"043a8297-789a-5bfd-acb0-fc2ff861991d","task_key":"default--validation--246","task_revision_id":"2","upstream_id":"246","short_description":"HellaSwag validation 246","config":"default","split":"validation","body":"{\"activity_label\":\"Cheerleading\",\"ctx\":\"A group of cheerleaders run onto a stage before a cheering audience. They get into formation, then begin dancing and flipping as male cheerleaders join them. they all\",\"endings\":\"['dance and flip on stage while hanging their hats on the sides.', 'continue dancing and flipping, doing hand springs.', 'dance together with flip up their sleeves.', 'perform their routines as the studio rashes to a halt with cheerleaders in the audience.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}