{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5ff03804-476d-5d89-8634-6bb192ba8931","task_key":"default--validation--46998","task_revision_id":"1","upstream_id":"46998","short_description":"HellaSwag validation 46998","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make a fake nose piercing [title] Gather your materials. [step] Whereas making a stud relies on glue, making a hoop requires a bit more effort. You'll need a pair of pliers, a pen or pencil, a file, and either some wire or a headpin.\",\"endings\":\"['[substeps] Use wire or a head pin depending on how you want your nose ring to look. For example, do you want your hoop to be thick (head pin) or thin (wire), gold or silver.', \\\"[substeps] For the hoop you'll need two 2-inch lengths of wire, two small metal cases, and a large cylindrical object to hold the hoop together. Since you'll be piercing your nose, you'll need several additional pieces of wire to hold the hoop together, so make sure you have several.\\\", '[title] Glue the ends together at the \\\" top \\\" edge of your work surface. [step] Lay the hoop down flat with the pigmented side touching.', \\\"[substeps] Wire to make your nose piercing can be found at most chain-and stationary supplies stores. If you don't have access to wire, you can also find a pre-made hoop.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}