{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9d9eab07-e3ef-5e62-b302-e429c59f0863","task_key":"default--validation--32645","task_revision_id":"1","upstream_id":"32645","short_description":"HellaSwag validation 32645","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to clean white vans shoes [title] Mix up a cleaning solution. [step] Using a mixture of laundry detergent and warm water is an easy way to clean vans. Into a large bowl, mix 1/4 cup laundry detergent and 2 cups warm water.\",\"endings\":\"['Dump in 1/4 cup fabric softener and 1 cup bleach. Mix together with a spoon or a rubber scrub brush.', \\\"If your vans are extremely dirty, you may need to mix up a clean batch for the second shoe. If you're low on laundry detergent, here are other cleaning solutions you can try.\\\", 'Pour the cleaning solution into the shoe, scrub the inside of the shoe, and rub in the rinse with a cloth or rag. The rinse will return the shoe to its original clean state.', 'Swirl the solution on a rag or cloth to get suds into the shoes. As you rub the solution in, notice if the color of your vans shoes gets darker or lighter.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}