{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e7107532-ad56-58f8-ad69-3119bc69c9eb","task_key":"default--validation--47215","task_revision_id":"1","upstream_id":"47215","short_description":"HellaSwag validation 47215","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear a fedora [title] Make sure the hat fits. [step] Big hats look better on big heads, whereas small hats look better on small heads. Make sure the hat fits well and feels comfortable before you purchase it.\",\"endings\":\"['[substeps] Try a fedora with a wide brim. There is a wide brim that should fit snugly over your head without letting it hang down down.', \\\"If your hat is too tight or constantly falls off your head, you're not going to want to wear it often. [substeps] To measure your head, place a tape measure 1/8 of an inch above your left ear and then wrap it around the circumference of your head to determine your head size.\\\", '[substeps] If the fedora is too small, try a cowboy hat. This style is brand new, but they tend to look great on large heads.', \\\"If you're opting for a fedora with a wide brim, make sure it reaches down to the bottom of your hat. [substeps] The brim size doesn't really matter, as long as the hat is long enough to cover your head.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}