{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0eff333a-5398-539f-9828-14f2c96c2f05","task_key":"default--validation--21960--variant-19fed0f9995c","task_revision_id":"1","upstream_id":"21960--variant-19fed0f9995c","short_description":"HellaSwag validation 21960","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to trim a mustache [title] Grow the hair above your lip. [step] If you can wait until it's thick and long, you'll have more to work with when it comes to achieving a certain look with your trim. [substeps] If this is your first mustache, it may take several weeks until your hair is long enough to trim.\",\"endings\":\"['This will give your mustache some time to grow in and blend in with your skin. You may need to trim the area every day.', 'You should snip your mustache as straight as possible before doing so. Loosen the hairs using a pair of sharp scissors and trim them off with a barber knife.', \\\"Begin etching out the shape of your mustache by shaving your cheeks and chin while you wait for your mustache to grow. Have some patience: if you start to trim your mustache too soon, you won't get the best results.\\\", 'A laser-straight trimmer will be more effective but less efficient than a handheld trimmer. To make it harder to trim your mustache properly, tape down your trim to your hand.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}