{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6bf8d804-25ad-56f8-9574-7b771a031352","task_key":"default--validation--21105","task_revision_id":"1","upstream_id":"21105","short_description":"HellaSwag validation 21105","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to be a good guest (for mothers in law ) [title] Rent a car, or have an alternate way to get around. [step] Don't expect your hosts to cart you around to wherever you want to go. Additionally, your hosts may be not having their vacations as you are and working throughout the day-the last thing they want to do when they get home is drive you places.\",\"endings\":\"[\\\"[title] Plan other things during your visit, and let them know in advance. [step] If they are working during your stay, they may need some time to themselves away from' entertaining'.\\\", \\\"Likewise, don't expect to be away much. Leaving several random nights of your period in your room is a waste of time, the worst thing ever.\\\", \\\"It's best to try any alternate route so as to avoid a poor rental. [title] Look through a phone book, internet guide, or library.\\\", '[substeps] Be sure you bring an alternate route. [title] Buy a gym membership (for women) and workout gear.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}