{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"905b4d79-968b-518b-95b8-57c2d354801a","task_key":"default--validation--25003","task_revision_id":"1","upstream_id":"25003","short_description":"HellaSwag validation 25003","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make lip gloss with honey [title] Clean out old lip gloss containers. [step] Wash them carefully with soap and water, and then sterilize them with rubbing alcohol. Rinse them out once more with water so that you won't be able to taste the alcohol when using your new lip gloss.\",\"endings\":\"['Place them in your small lip gloss container or bottle. [substeps] Try to have a small container, not too big enough to hold up to 50 milligrams of honey in.', 'Do not dip the containers in pure honey, as it can damage or destroy the enamel on the container. [substeps] You can protect your lip gloss with an old lip gloss bottle, or a container on a metallic glass.', '[substeps] Any petroleum based formula will work, but to apply a fresh, perfect lip gloss, stick to an alcohol-free formula. [title] Mix the honey and egg yolks together.', \\\"[substeps] If you don't have old containers on hand, you can purchase new ones from a beauty supply store, drugstore, or even a dollar store. [title] Place 3 tablespoons of petroleum jelly and honey in a microwave-safe dish.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}