{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8c6d9be4-9bcb-5143-a466-a59a493c70c7","task_key":"default--validation--26225","task_revision_id":"1","upstream_id":"26225","short_description":"HellaSwag validation 26225","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to act evil [title] Choose the right clothing. [step] Evil is typically associated with dark, foreboding colors, so when you're planning a costume for a villainous character, that's the color palette that you should start with. Black is the traditional option, but any dark shade, such as midnight blue or eggplant, can work well.\",\"endings\":\"['You need a long sleeved shirt, dark jeans, and a long black sweater or skirt with fur to make you look predatory. [substeps] Tan pants are good choices, though dyed straight from pormen, but you could also try dr.', \\\"Dressing up in the winter will work just fine for an evil character, so wear a dark blue-white or dark-blue eye color. If you're struggling with identifying the kind of costume you want to choose, see the math section below.\\\", \\\"Since dark, foreboding colors are not accepted in any wizard society, that's okay, too; whatever you do, don't overdo it. Above all, this is the main costume you want to create.\\\", 'Opt for clothes that have sleek, clean lines too. For example, a well-tailored suit is an attractive option for a male, while a form-fitting cocktail dress is an ideal costume for a villainess.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}