{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e34b48b3-a1da-5d30-a6f3-8ae82cde529e","task_key":"default--validation--8387","task_revision_id":"1","upstream_id":"8387","short_description":"HellaSwag validation 8387","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to get a medical tattoo [title] Learn about \\\" medical alert \\\" tattoos. [step] A \\\" medical alert \\\" tattoo is a tattoo that communicates important medical information about a patient, such as a severe allergy or the presence of a disease. These tattoos are meant to replace traditional medical alert bracelets, which can get lost.\",\"endings\":\"['They sometimes even go unregarded for collectors and may eventually become difficult to leave alone. [title] Get a tattoo at a local tattoo parlor.', 'Although these tattoos are growing in popularity, there is still no consensus about the design or placement of such tattoos. This can make them more difficult for medical professionals to detect.', \\\"The tattoo outline looks like a spot on a patient's body. The tattoo's design often has been stenciled onto a patient's skin to show evidence of the tattoo's importance and significance.\\\", 'You can find them online or at a tattoo store, or at a tattoo parlor. [substeps] Specialized tattoo artists (paramedics, physicians, and others) typically do not provide special graphic-design or detail design to their tattoos.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}