{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a54aacfb-7c61-582b-a030-8ab5ea61ba3d","task_key":"default--validation--41805","task_revision_id":"1","upstream_id":"41805","short_description":"HellaSwag validation 41805","config":"default","split":"validation","body":"{\"activity_label\":\"Cutting the grass\",\"ctx\":\"A man is looking all over for his dog. He said he paid his dog 10 dollars to cut the grass and now he's gone. he\",\"endings\":\"['said he was going to buy the dog esp and could give it a home on his family farm.', 'finally finds the dog who is driving the riding lawn mower.', 'shows his dog and pet, the dog is sitting on the ground stretching its front legs and then washing his front paws.', 'is mowing his backyard and walking to the store.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}