{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d9b2b7ea-a36b-567c-9929-377539b41b31","task_key":"default--validation--16439","task_revision_id":"1","upstream_id":"16439","short_description":"HellaSwag validation 16439","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to extend hair color [title] Wait 48 hours after your most recent shampoo to have your hair colored. [substeps] Natural oils on your scalp help the dye to better adhere to your hair. Make sure to let your stylist know you'd prefer not to have your hair shampooed prior to coloring.\",\"endings\":\"['[title] Practice taking a shower, or soak your hair in warm water after you shampoo it, if desired. [substeps] The faster you wash and condition your hair, the better your results.', '[title] Wait 48 hours after your color treatment is applied before shampooing. [substeps] This allows time for the color to completely penetrate the hair.', 'Linseed oil helps the color adhere better to your hair. [title] Heat up the oil in shampoo according to your prior instructions.', '[title] Mark your hair as dry before coloring. [substeps] Before applying your dye, dab a bit of lukewarm water over the dry film on your hair.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}