{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5e4f4a0c-d481-5a97-a1d7-136499577deb","task_key":"default--validation--10758","task_revision_id":"1","upstream_id":"10758","short_description":"HellaSwag validation 10758","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to do winged eyeliner with pencil [title] Use eyelid primer. [step] Before you begin the process of applying eyeliner, dab some primer onto your eyelids. Use your fingers to gently rub in the primer.\",\"endings\":\"['Next, run the eye primer over your eyelids using circular motions. Dab around them and smudge the primer in with your fingers.', 'This will make your eyeliner and any eyeshadow you use stand out more. [substeps] Make sure to wash your hands before you apply your eyeliner.', 'You should get a smooth, even coverage that covers both the bottom of your eyelid and the area around your lips. [title] Know the right brush for the right shade.', \\\"If it's too greasy (too dry), the eyeliner may not hold up on you. [substeps] If you're wearing eye primer, it's recommended to apply it after cleansing your eyeliner.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}