{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d90fb830-48cd-51f7-bd7a-55f0285b255a","task_key":"default--validation--46306","task_revision_id":"1","upstream_id":"46306","short_description":"HellaSwag validation 46306","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make organic face masks [title] Assemble the ingredients. [step] For this wonderful method you'll need : [substeps] Organic ground cinnamon organic honey organic lemon juice (squeezed from an organic lemon. [title] Take a cup or an object to put the ingredients into.\",\"endings\":\"['[step] Such as a small bowl, a small raisin, a wooden spoon, or any other food item that you wish. Mix well before you add the other ingredients.', '[step] The low sides of the cup provide the best access to the mask. The large side is used to hold the mask in place and be simple so that no tiny bits or particles get easily absorbed.', '[step] Put the lid on and leave it ajar (open while allowing the honey to combine). [title] Let the honey soak in and take it out.', '[step] First, put in 2 tablespoons of organic honey; soften it by standing the container in a wider bowl of warm water. [title] Add 2 teaspoons of organic ground cinnamon.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}