{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"aa406247-7ddf-5dde-a594-16c08cf306ed","task_key":"default--validation--45677","task_revision_id":"1","upstream_id":"45677","short_description":"HellaSwag validation 45677","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make sure you don't have too much makeup on [title] Pick one feature you want to bring out the most. [step] If you chose eyes, then wear more makeup on the eyes. Use colors that accentuate your eye color.\",\"endings\":\"['Consider choosing neutral shades of your skin color. This will make your freckles stand out more and help avoid blackheads.', 'Choose a colorful eyeshadow at least three shades darker than your skin tone. For darker skin, choose a greenish/blue eyeshadow instead.', 'If you choose to wear powder foundation, wear it on the eyelids and only on the lids. Makeup is the best tool for keeping looks fresh and clean.', 'Avoid anything on your lips, except chapstick or a clear lip gloss. If you want to accentuate your lips, choose a color that goes well with your skin tone.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}