{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2489ab59-8c4c-5e58-b9cb-84671bd9a224","task_key":"default--validation--45616","task_revision_id":"1","upstream_id":"45616","short_description":"HellaSwag validation 45616","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to choose a tattoo design [title] Look up tattoos online. [step] Visits image banks or galleries online to view images of different tattoos or designs. Images are usually grouped by category, which can reduce your search time if you have a basic idea in mind.\",\"endings\":\"['Save your favorite images in a folder on your computer, or on a pinterest. [title] Check out binders at a tattoo shop.', '[substeps] Look for a tattoo artist who specializes in a specific tattoo or design. [title] Decide what you want your tattoo to be.', 'The way you get started is how big and intricate you want the tattoo to be. Some images may depict more than one tattoo, although you may be able to create a larger design by matching two or three different designs.', \\\"Tattoo artists often look for images of potential tattoos or designs which they believe are right for your skin and body. [substeps] Keep in mind that you may not want to be bored with all your options at once, so don't ask.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}