{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"aefcd0b3-4cc8-5783-bd63-782ff2f34696","task_key":"default--validation--36871","task_revision_id":"1","upstream_id":"36871","short_description":"HellaSwag validation 36871","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to do a strand test before dyeing your hair [title] Put on disposable gloves. [step] Pull on the plastic gloves provided in your dyeing kit to protect your hands from the chemicals in the dye. You should leave these gloves on throughout the strand test.\",\"endings\":\"['Wait a few minutes to ensure that they do not stain your hands as any color can be unwanted. [substeps] You can also use a pair of plastic gloves.', '[substeps] Check for any adverse reactions that may arise during the process. For example, you may experience dry or brittle strands.', \\\"[substeps] If you don't have gloves included with your dyeing materials, purchase any disposable latex or latex alternative gloves from the store. It's important that you avoid contact of the dye with your skin.\\\", \\\"To ensure that you didn't accidentally transfer too much dye, cover your hands with a soft, thin bandage before you begin. The bandage should cover the entire strand of your dyed hair.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}