{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ec28f37a-ed41-5aec-b82e-4c47cc297a81","task_key":"default--validation--25646","task_revision_id":"1","upstream_id":"25646","short_description":"HellaSwag validation 25646","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to shape your eyebrows with a razor [title] Purchase a razor made specifically for eyebrows. [step] A standard razor will not give you detailed brows, and can be dangerous to use near your eyes. Instead purchase an eyebrow razor, whose small blade will give you the brows you desire.\",\"endings\":\"['Head to the store and buy a kit. The kit will contain an assortment of tools and scrubs, all of which you can use to shape your eyebrows.', 'Ask your doctor for advice, or check your health insurance before attempting to purchase or shave your eyebrows. Your doctor can help you choose the right razor for your preferences and needs.', \\\"Eyebrow razor razors come in a variety of sizes, so it's important to know your eyebrow size, and look at the sizing when deciding which one to use. [substeps] To create your eyebrow shape, use a shape bar to outline your brows, creating a hole on top of the eyebrow bar, marking off your brows.\\\", '[substeps] You can order an eyebrow razor online or purchase one at your local beauty supply store. [title] Use a small amount of shaving cream.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}