{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6d4ca3b0-098d-5c27-8a65-4129587b8d6a","task_key":"default--validation--4881--variant-63e98c71ad9f","task_revision_id":"1","upstream_id":"4881--variant-63e98c71ad9f","short_description":"HellaSwag validation 4881","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to apply eyeliner to sensitive eyes [title] Read the ingredients. [step] Some eyeliners say on the label that they are safe for contact wearers or are hypoallergenic. This, however, is not a guarantee that the eyeliner will be comfortable for sensitive eyes.\",\"endings\":\"['Make sure eye makeup is formulated without sulfates or parabens, which can cause skin irritation. Talc and kohl should also be avoided to prevent irritation.', 'People who use flavored eyeliners are the only ones who know for sure. How much if you use double-or triple-colors or stay away from black.', 'Check the ingredients or instructions on the label to make sure they are safe to use with your eyes. [title] Clean your lips before applying eyeliner.', 'To avoid irritation in sensitive eyes, read the ingredients list displayed on the lip gloss store. If it says \\\" hypoallergenic, \\\" then it\\\\'s likely the product is hypoallergenic.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}