{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"40e93551-a154-5f94-aab3-a4f5e634a44a","task_key":"default--validation--39696","task_revision_id":"1","upstream_id":"39696","short_description":"HellaSwag validation 39696","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to fix dried out eyeliner gel [title] Sanitize your brush. [step] Before you fix your liner, you need a clean brush. Wash an eyeliner brush using your favorite brush cleaner or baby shampoo and hot water, then wring it dry.\",\"endings\":\"['[title] Create a watertight liner by taking a very small dab of gel or liquid concealer. [step] Squirt a thin layer onto the brush you will be using.', '[title] Consider using an entirely non-alcohol-based powder eyeliner. [step] These types of products are designed to be nonflammable so they are more forgiving than all-natural eyeliner.', \\\"Then, swirl the bristles of your brush around in a shallow cup of isopropyl alcohol and allow it to dry. [substeps] Once dry, the alcohol will not hurt or sting on your skin, and it doesn't hurt your makeup.\\\", \\\"Then paint the brush's bristles with a few drops of a mild mustard, such as lemon juice or boiled water. [substeps] On your brush, dip the brush into a bowl of water until the oil and water have separated.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}