{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c375c73b-81bc-5ddf-af52-0d68d4102173","task_key":"default--validation--28168","task_revision_id":"1","upstream_id":"28168","short_description":"HellaSwag validation 28168","config":"default","split":"validation","body":"{\"activity_label\":\"Disc dog\",\"ctx\":\"After, the man holds up the frisbee and the dogs stands in two legs. After, the dog pass between the legs of the person. next\",\"endings\":\"[', the person takes off the ear pieces of another dog.', ', the man picks up a brown dog with three legs and helping the man to carry it.', 'to the man, the man throw the frisbee, then puts it in the air and catch the frisbee again.', ', the person holds a frisbee high and the dog jumps to get the frisbee.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}