{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d6a81ae5-5539-5660-888f-863795c15ce5","task_key":"default--validation--34712","task_revision_id":"1","upstream_id":"34712","short_description":"HellaSwag validation 34712","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make cocoa lip balm [title] Place cocoa in the bowl. [step] Use your spoon to scoop 3 tablespoons of cocoa powder into your bowl. Make sure the bowl is safe to microwave.\",\"endings\":\"[\\\"[substeps] Choose a high quality cocoa powder and be sure that you're not using bars of chocolate or hot cocoa mix. These contain additional ingredients that could make your lip balm spoil or go bad.\\\", '[substeps] When you microwave or refrigerate your ingredients, watch the size of the container the powder is in. Adult who is not a certified cocoa drinker should only use 1 cup of cocoa.', 'Do not add too much or the cocoa may burn. [title] Transfer cocoa mix to an airtight container.', \\\"If you're using liquid cocoa, skip this step. [title] Add 1 tablespoon of peanut butter for each 1/2 cup of liquid.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}