{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a5525686-0f1f-50df-bbb4-0ad36f6f7112","task_key":"default--validation--46054","task_revision_id":"1","upstream_id":"46054","short_description":"HellaSwag validation 46054","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear a puffer jacket [title] Select a puffer jacket with a sleek silhouette. [step] In the past, most puffer jackets were thick, bulky and cumbersome. These days, puffer jackets come in a range of stylish silhouettes and cuts.\",\"endings\":\"['Look for jackets that are nipped at the waist, since these will be more form-fitting. Consider a cropped, bomber-style cut instead of the more traditional coat-length version.', 'And also, more to the point, recent styles have passed the cra test. Fitted fitted puffer jackets are worn with pants instead of shirts, and they are generally more lightweight.', 'Consider the length and shape of your jacket and pick one in a versatile shape. Think between : [substeps] Long or short jackets stand out best in warmer weather.', \\\"You can wear long, slimmer puffer jackets if you are on a budget but that doesn't really matter now. [title] Choose a tailored and well-made buttoned zip-down coat.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}