{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ff90443b-ebe1-5a15-a1cd-8ac9ce1be3b0","task_key":"default--validation--7509","task_revision_id":"1","upstream_id":"7509","short_description":"HellaSwag validation 7509","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to look like you are wearing makeup [title] Enhance your eyelashes. [step] Instead of mascara, take a bit of lip balm (flavoured works best) on your index finger and thumb and run it through your eyelashes. This will darken and lengthen your eyelashes, and over time, it will increase their actual length.\",\"endings\":\"['[substeps] Re-apply mascara daily for better results. This protects your lashes from harmful antioxidants.', 'You can use petroleum jelly instead of lip balm. [title] Use a little bit of soft lotion of your eyelids.', \\\"[title] Bring your eyebrows up so that the top of them are in the middle of your eyes, and relax them gently. [step] Don't try to contour these brows.\\\", 'Practice this step without a lipstick or gloss first. [substeps] If you have thick eyelashes, try using a matte lipstick or gloss.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}