{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ae0fae11-ab3c-57ba-9a2b-3179b4f3e311","task_key":"default--validation--38774","task_revision_id":"1","upstream_id":"38774","short_description":"HellaSwag validation 38774","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear a cowboy hat properly [title] Get the right size. [step] The first step to properly wearing a cowboy hat is making sure the hat is the right size and fits your head properly. Cowboy hats should typically fit snug around your head.\",\"endings\":\"[\\\"You can wear them under the brim or up around the neck. [substeps] It's not necessary to measure your head before buying a cowboy hat, and actually no matter what accessory you wear, you should probably start sizing your head first when deciding on a size.\\\", 'If the brim is too small or too large for you, it will limit your head movement. [substeps] Use a thick leather cord or strap around your head to help you decide if you can comfortably pull the brim up or down.', 'If you are buying one, know your hat size ahead of time but also try the hat on before you buy it. [title] Fix your current hat to fit better.', \\\"Depending on how long your head is, and how much muscle it needs to support its weight, your hat may fit tighter or narrower around your head. Generally, that means it's too small.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}