{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2342e9e1-c214-57ce-98d5-35d4322176da","task_key":"default--validation--34272","task_revision_id":"1","upstream_id":"34272","short_description":"HellaSwag validation 34272","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to get dog smell out of your car [title] Wash any removable parts of your car. [step] Any parts of your car that can be removed should be washed in a washing machine. This will get the dog smell out, and leave your car smelling fresher.\",\"endings\":\"[\\\"If you notice your car smells like you when you unplug it, you should soak the parts for 5 minutes in warm water and 20 minutes in cool water. [substeps] If you're doing a street fair, use a funnel to transport plastic parts into the washing machine with the outside parts discarded.\\\", '[substeps] Rubber mats, car seat covers, and any blankets you keep for your dog can be removed and washed. Use a cool temperature to be safe when washing parts of your car.', 'A coat of antiperspirant, on the other hand, will only remove the product, and your car likely already smells like urine. [substeps] If your car has leather seats, wipe away any product that collects there.', '[substeps] The safest option is to remove all the removable parts of your car, including the \\\" fifflers \\\". These components can get stinky easily in hot weather, but they can also sweat and produce your dog\\\\'s own odor.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}