{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8ec24ff5-503d-5a3c-9f2e-697b344aae37","task_key":"default--validation--3171","task_revision_id":"1","upstream_id":"3171","short_description":"HellaSwag validation 3171","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to act like an adult [title] Dress like an adult. [step] Act the part by investing in clothing that makes you feel like a grown-up. Trade in your band shirts for collared button downs, go for a nice blazer instead of a denim jacket, and invest in a nice pair of shoes.\",\"endings\":\"['[substeps] Go shopping with an adult and ask for their opinion on your selections. Look for clothing that is high quality and well made.', \\\"You don't have to shop for expensive goods or dive into fashion, but do buy a super fashionable dress, one that fits you well. Wear accessories for a totally street persona that also has effects on being a grown-up.\\\", \\\"Just below that, wear what makes you feel and are comfortable. [substeps] If you're tall, go for a nice yellow cardigan or a navy trench coat with black trousers.\\\", '[substeps] Invest in trendy chic accessories, like cufflinks, colorful headbands, and lip-gloss. [title] Keep an open mind about what you like and what you dislike.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}