{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"96761197-db95-5f97-9325-272793ce5b84","task_key":"default--validation--30251","task_revision_id":"1","upstream_id":"30251","short_description":"HellaSwag validation 30251","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to adopt a service dog [title] Look for service dog organizations in your area though the internet. [step] Many service dog organizations have dogs up for adoption. These dogs may either be ones that couldn't make it through the program or who have retired from service.\",\"endings\":\"['There are also programs that offer purebred service dogs with genetic disabilities. Be sure to identify your country of origin.', 'If possible, find one of these organizations by searching in one or another of the following search engines to find a city or region where the dog was raised. [substeps] Southern shelter shelters rescue dogs involuntarily after becoming homeless.', 'A good list of organizations is on the internet. [title] Decide what if you are preparing for the adoption.', 'Try searching \\\" adopt service dog \\\" with your city. [substeps] These organizations are more likely to be in larger cities, so you may need to travel a bit if you live in a smaller city.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}