{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"780a43d6-7bc4-5927-b7ad-6fd3e815f353","task_key":"default--validation--30646","task_revision_id":"1","upstream_id":"30646","short_description":"HellaSwag validation 30646","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to go grey [title] Use coverage products while you grow out your roots. [step] There are several types of products available that are designed to temporarily color grey roots. Most of them come in the form of chalk, spray, or crayons, and can be applied directly to your roots and washed off at the end of the day.\",\"endings\":\"['Apply these products only in warm, dry weather, as this is the length of time necessary for the roots to dry completely. [substeps] Read reviews about unique products before purchasing them.', '[substeps] Choose a color that matches your dyed hair so that your roots will blend with the rest of your hair. Crayons are generally less likely to rub off on fabrics, such as your pillowcase or hat.', \\\"These products do not alter the color of your roots, and they prevent the color from spreading. [substeps] If you feel as if you're not adding color, you may have naturally lighter grey roots.\\\", '[substeps] If you grow your roots in gel form, you can see a very drastic reduction in your hair color. You can also use perm strips to completely cut down your hair color.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}