{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c7259110-9d1f-5e97-9f8f-f46528259cd5","task_key":"default--validation--32311","task_revision_id":"1","upstream_id":"32311","short_description":"HellaSwag validation 32311","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to do a messy bun (for curly hair ) [title] Start with dry hair, and have everything you need on hand. [step] Once you start making your bun, it will be difficult to open boxes and drawers. Keep a bunch of bobby pins handy, and your hair tie around your wrist.\",\"endings\":\"['Try to use a hair tie that matches your hair color. You can find them in black, dark brown, and light brown/beige.', '[title] Part your hair under the near shoulder. [step] Take the top part of your hair in the right direction and split it down the middle, making a very heavy ponytail.', \\\"[substeps] After you've done that, check the hair color you want your hair to be. Sure, there are lots of bright reds, but there are also lots of colors to choose from, so don't stop there.\\\", '[title] Bring your hands together for a different style (blue, yellow, purple, or white are best). [step] Put the bobby pins you will use to the side with the hair tie.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}