{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"50ff3e60-0f53-5a37-a5aa-a521e0db760b","task_key":"default--validation--10002","task_revision_id":"1","upstream_id":"10002","short_description":"HellaSwag validation 10002","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to use a collagen lip mask [title] Decide on a collagen lip mask. [step] Lip masks can vary by several factors, such as size and taste. You may want a bigger lip mask to get the dry skin around your lips or if you have plumper lips to begin with, for instance.\",\"endings\":\"[\\\"However, some people enjoy larger, heavy-coverage masks because they're heavier and longer lasting. Pay attention to difference between different lip masks and what type of face paint you like most and stick with the one you wear most often.\\\", 'Some also have tasty flavors, and that can be a deciding factor. [substeps] Most of these masks have hydrating ingredients, so you if you have an allergy, be sure to check out the ingredients before buying.', \\\"A large lip mask will be ideal for dry skin by reducing swelling and redness. [substeps] If you're trying to achieve light coverage, you should use a slightly brighter lip mask.\\\", 'These characteristics make mouth masks ideal for this cause. [substeps] Your mouth mask determines the size of your face, as well as how your mouth will fit through it.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}