{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bda349cd-4135-5dd1-b570-57ee831aeaa4","task_key":"default--validation--24222","task_revision_id":"1","upstream_id":"24222","short_description":"HellaSwag validation 24222","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make a mustache [title] Flat iron your craft hair. [step] You can purchase craft hair online or at a craft store, such as michael's. Make sure to purchase hair designed to be used in costumes or on things like dolls.\",\"endings\":\"['Use a comb to flatten your hair or shape your hair into a mustache. If the texture of your hair is not the same as what you want, you can do a manicure or mod podge job on the area where you want the mustache to go.', \\\"Run a flat iron over your craft hair to make it smooth and easy to work with before you start forming your mustache. [substeps] Craft hair can sometimes be very fine, so it's best to err on the side of caution and use a low heat setting.\\\", 'Two methods are the flat iron and the round-tooth mod 2. Both methods work, but all your hair must be flat.', 'You want the hairs on your mustache to be smooth, and go to the hair where they attach in your mustache. [substeps] Be extremely careful which side of your mustache you flat iron on.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}