{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"75d7b8e4-843e-506a-ab5b-136132e43115","task_key":"default--validation--10314","task_revision_id":"1","upstream_id":"10314","short_description":"HellaSwag validation 10314","config":"default","split":"validation","body":"{\"activity_label\":\"Grooming dog\",\"ctx\":\"A woman is seen hosting a news segment with a man and leads into a person bring out a dog and petting the dog. the man\",\"endings\":\"['then cuts up a potato and shows the woman how to peel it and shows off the final product as well as taking the potato out and speaking to the camera.', 'then pets the dog and lights a flame to add some heat and leads back into the dog being tv interviewed and shown off again until it goes back to the other arm wrestling event.', 'then demonstrates how to properly groom the dog while the host continues to speak and pet the dog.', 'finishes grabbing the dog and putting the dog in his coat.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}