{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0102fa87-95bb-5e37-a25d-7f72088bbaf5","task_key":"default--validation--6587--variant-ffc1301447cc","task_revision_id":"1","upstream_id":"6587--variant-ffc1301447cc","short_description":"HellaSwag validation 6587","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to style mom jeans [title] Pick a pair of jeans that fit your waist well. [step] Mom jeans are meant to sit high on your waist. When picking out a new pair, try them on and see how the waist fits-you don't want the jeans too tight or too loose.\",\"endings\":\"[\\\"Try on any pair and see if it's even snug. [substeps] If your mom jeans tend to dig into your legs, look for jeans that come up above the knee.\\\", '[substeps] Pick out jeans that are at the leg length of your legs. If you have shorter legs, opt for what looks best to you, rather than what might look nice with longer pants.', \\\"[substeps] The high waist will show off your waist and elongate your legs. [title] Choose a leg style that's appropriate for your figure.\\\", 'Try buying pairs that are so low that you can still walk in them but they are a little looser. [substeps] Ask your mom about how to size your jeans.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}