{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b729516a-81df-5959-9171-9bff55da1639","task_key":"default--validation--35182","task_revision_id":"1","upstream_id":"35182","short_description":"HellaSwag validation 35182","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to pretend to be a girl [title] Get a wig or style your hair. [step] If you have long hair already, then you can just style it. If your hair is short, then you may want to get a wig to enhance your feminine look.\",\"endings\":\"['If your hair is long, then you may want to get a wig to transform your hairstyle. Try ponytails, pigtails, or several similar hairstyles.', \\\"Wigs make your hair appear like a little frizz at the top. [substeps] Do not get a wig that is very short on top, unless you're sure that you want it to be.\\\", 'When you are choosing a wig, make sure that you purchase one that is high quality if you want your look to be more convincing. A poor quality wig may not look as good.', \\\"[title] Spend some time creating your own hair style. [step] For some girls, this may mean shopping at a hair salon; for others, that may mean posing at a hairstylist's home instead.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}