{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ba41ff08-49fc-501b-b67f-66dc0d193aff","task_key":"default--validation--6220--variant-3c96cb497fd3","task_revision_id":"1","upstream_id":"6220--variant-3c96cb497fd3","short_description":"HellaSwag validation 6220","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to do tree braids [title] Find someone to braid your hair, or find someone that you can practice your tree braids on. [step] Unless you are very skilled at braiding, it would be very difficult to do this technique on yourself. [substeps] If you want someone to give you tree braids, offer to do a trade with them.\",\"endings\":\"['Make sure the person is nicer than you. If they are not, they may not be the best person to braid your hair for you.', '[title] Decide what kind of twists to braid. [step] Once you have chosen your style, you can decide how long your braids should be.', 'Suggest that you can give them braids if they do them to you as well. This way you both get practice and a new hairstyle.', \\\"You can also try braiding other people's hair, as long as the seller doesn't accept credit. Ask someone to accompany you through the process, if possible.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}