{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"604a83cf-6246-59bf-8018-4ed1058cab92","task_key":"default--validation--261","task_revision_id":"1","upstream_id":"261","short_description":"HellaSwag validation 261","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to apply false eyelashes [title] Size the lashes. [step] Before gluing the lashes on, you will need to make sure they are not too wide for your eye. Hold the lash strips up against your eyelid, and trim them down on the sides if necessary.\",\"endings\":\"['[substeps] If the lashes are too long for your personal taste, consider trimming the individual lashes down to achieve a more natural look. Lashes should be longer toward the outer corner of the eye.', '[substeps] If you do not like the length of your lashes, opt for a couple of inches for the most dramatic look. You can use thick lashes that you already have.', 'Also, double check that the end of the lash strips do not touch your eyelid line. If they do, then they may give you mistaken lashes.', 'Tape the fake lashes to the front of your eye using eyelash glue. Move your lashes down along your eyelid to even them out.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}