{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a110b82a-01cf-5a68-93d3-40996d1d7be3","task_key":"default--validation--46204","task_revision_id":"1","upstream_id":"46204","short_description":"HellaSwag validation 46204","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to buzz cut hair [title] Use high-quality clippers with secured guards. [step] If you don't already have a pair of high-quality clippers, you can purchase a pair costing between $20 and $120 online. Before you start buzzing, make sure the guard is securely fastened to the clippers.\",\"endings\":\"['Otherwise, you could end up with a bald spot if it falls off while in use. [substeps] Many clippers will come as a kit, including not only the clippers but the charging station, guards, combs, and barber scissors.', \\\"[substeps] Go ahead and set the guard so it's to your right when you start to buzz. This will ensure your hair doesn't get in the way if you accidentally buzz it.\\\", \\\"[title] Shave down the sides of your head and across your temples. [step] In order to buzz your hair, you'll want to shave down to the tips of your ears.\\\", '[substeps] Wear a pair of safety gloves to protect your hands from getting scratched by the prongs of the clippers. [title] Decide how many pieces of hair you want to buzz.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}