{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"74be738e-81c9-5e32-b8ba-debce667cc57","task_key":"default--validation--487","task_revision_id":"1","upstream_id":"487","short_description":"HellaSwag validation 487","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to be a redhead [title] Pick a brand. [step] The two most popular red dye brands are schwarzkopf and l'oreal, but a trip to your local drugstore beauty aisle will reveal dozens of reliable and affordable dyes. This step can be overwhelming since there are so many options.\",\"endings\":\"['Read the descriptions on boxes that grab your attention. [substeps] Decide if you want semi-permanent dye or permanent dye.', 'You may choose either bleached blonde or tinted blonde if you only want your look to lighten up a bit. Selecting the right brand is the first step to achieving this look.', 'Pair the shades of blue and purple with whatever shade of red or soft pink is alive in the outdoors in september or august. It can also influence the tone of your fabric, so pick bright shades of red or purple for your bolder shade.', \\\"But remember, consider what you want to wear, and don't give up if you don't find something you like yet. Dyeing your hair can be scary and ridiculous, and dye won't change that.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}