{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1d916461-c160-50c0-be35-922f34c7dc57","task_key":"default--validation--30290","task_revision_id":"1","upstream_id":"30290","short_description":"HellaSwag validation 30290","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to do ombre eye makeup [title] Prime your eyes. [step] It is important to prime your eyes to prevent creasing. You can use the makeup primer of your choice, or if you don't have primer, you can cover your eyelids with concealer and powder.\",\"endings\":\"['While you will have to use the primer to soften your eyes, it will provide you with heavier coverage. [substeps] Ask a friend or family member to prime you if you are not sure how to do it.', 'Use a small brush to prime your eyes. Do this for one hour, according to the directions on the packaging.', 'Use a small amount of concealer to cover your eyelid and then cover it in a nude powder. Or, you can use a nude cream eye shadow.', '[substeps] Then, in an empty makeup-jar, two 1 oz. ( 237 ml) makeup or powder primer (without the one you used to prime your eyelids), two 1 oz.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}