{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"28ee8aa6-bdec-5d92-9891-6f0acc2c377b","task_key":"default--validation--20604","task_revision_id":"1","upstream_id":"20604","short_description":"HellaSwag validation 20604","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear biker boots [title] Pick a waterproof pair if you plan to wear them in all weather conditions. [step] Biker boots are great for the colder months as they are functional yet fashionable. Read the labels carefully to ensure the pair you pick are waterproof so you can rock them any time of year.\",\"endings\":\"[\\\"[substeps] Dress up this look with a solid piece of clothing like a t-shirt, short sleeve or appropriate jacket. Make sure the thickness of the boots isn't too tight.\\\", '[substeps] Suede boots are more versatile than leather if you can take any type of severe weather and wear them as if you were going out. Many people opt for suede boots because they are waterproof in the winter and functional throughout the summer months.', \\\"[title] Opt for a colored or detailed pair if you want a statement shoe. [step] If you're looking to make biker boots the focal point of your outfit, you may not want a plain, black pair.\\\", 'You can also get a pair made for over-dressing at any outdoor store. [substeps] When picking a waterproof boot, err on the side of simplicity.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}