{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"51bbd817-5b20-5cc9-8f98-371b999c1774","task_key":"default--validation--39438","task_revision_id":"1","upstream_id":"39438","short_description":"HellaSwag validation 39438","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make face wash [title] Find a suitable container for your face wash. [step] You can use an old face wash bottle, a mason jar, or even an empty bottle. Whatever you choose to use, make sure that it is clean and that it has a lid or cap you can close tightly.\",\"endings\":\"['[title] Pour some liquid castile soap into the container. [step] You will need ¼ cup (56.25 milliliters) of liquid castile soap.', '[title] Add enough detergent and supplies to cover your face with the face wash. [step] Add enough dye or softener to pool and stain the pores of the face, then wait a few minutes for the skin to soak up the new color.', '[substeps] Do not use a regular bottle because the lid is much too small and might not work properly. Old toilet paper works best, though you can use any type of disposable container as long as it is clean.', '[substeps] You can use either a bottle cap or a plastic tube lid. Porcelain lids can get wet or damaged, if they are not put under a proper seal.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}