{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8cbf3b5c-ff14-5505-b5e4-a395787c11da","task_key":"default--validation--777","task_revision_id":"1","upstream_id":"777","short_description":"HellaSwag validation 777","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make aloe moisturizer [title] Add the almond oil, jojoba oil, and beeswax to a double boiler. [step] To start, you will melt your oils together. Add your almond oil, jojoba oil, and beeswax to the top portion of your double boiler.\",\"endings\":\"['[substeps] A double boiler has two pans: a larger, deeper pan and a smaller, more shallow pan. You should add your oils to the smaller pan and add a few inches of water to the larger pan.', 'Use a wooden spoon to mix the ingredients. [substeps] The heat will activate your oils in the melted-together mixture.', 'Use a small spoon to stir or heat your oils together. [substeps] If you do not have a double boiler, warm both to start.', 'Use one boiler to melt your oils and one to melt your milk. Stir or stir your oils in between constantly heating them in the boiler as they continue to heat together.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}