{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5a81042c-d34c-5fc1-b9c3-46504f0f8e81","task_key":"default--validation--6660","task_revision_id":"1","upstream_id":"6660","short_description":"HellaSwag validation 6660","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make an emergency makeup bag [title] Make sure that the size is right. [step] When you choose a makeup bag, it's important to choose one that's large enough to hold all of your items. However, you don't want it to be too big or it may not fit in your purse, backpack, or other bag.\",\"endings\":\"[\\\"Look for a makeup bag that's just large enough to hold all of your essentials. [substeps] It's also a good idea to consider the color of your makeup bag carefully.\\\", 'Find a larger bag or something that you can hold all of your tools, like a makeup applicator, makeup brush, mascara or lip gloss. [substeps] If possible, order something larger than your entire bag or case to allow you to comfortably assemble your items.', \\\"If you don't know what size you're looking for, then don't wait until a day or two after you have your bag picked out to determine what it can hold. [substeps] Choose a bag that's not too small.\\\", 'Choose a bag you can seal tightly so that everything stays in place instead of being opened. [substeps] If you use a bag with lids, try to use loose lids, so that each item can fit on its own.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}