{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"62b599b6-4e52-5cc6-88b9-0dce7725a609","task_key":"default--validation--29388","task_revision_id":"1","upstream_id":"29388","short_description":"HellaSwag validation 29388","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make a moisturizing deep treatment for hair [title] Mix up the ingredients in your blender. [step] Make sure the mixture is very smooth, or it will get stuck in your hair. [substeps] If you're working with any fruits (banana or avocado), lots of honey, and other natural ingredients, it's better if they're well blended.\",\"endings\":\"['Cut up fruits like bananas so they blend well. If the mixture is too thick, you can always add more conditioner.', 'You could also use soy sauce instead of milk for a subtle fruit treatment. Mix up some honey and milk together and set it aside for now.', '[title] Add a moisturizing hair serum. [step] You can select these over any hair products.', 'Be sure you find enough honey in your blender to work, such as pure brown sugar, honey, or olive oil. Milk can also work, as long as there is not cream.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}