{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9b2b3fc0-0d53-5d6b-b295-072af1779a03","task_key":"default--validation--5861","task_revision_id":"1","upstream_id":"5861","short_description":"HellaSwag validation 5861","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make perfume [title] Know the different notes. [step] Perfumes are a blend of different levels of scent, also called \\\" notes \\\". When you spray a fragrance on your skin, it moves through these notes in the following order : [substeps] Top notes are what you smell first.\",\"endings\":\"['They are, essentially, notes that you make in your nose before applying perfumes. Other notes are: top notes linger in your nostrils.', \\\"Shoes with high heels ahead of you and quickly put on socks when you roll your toes down. Smaller (comparatively, tiny) notes may remain on your other foot for a short time while you're in the store.\\\", 'They are also what disappears first, usually within 10 to 15 minutes. Middle notes appear as the top notes die off.', 'You should be able to tell which notes a puff of air is moving through instead of always smelling the same notes when you are in the air. Down notes are whatever you inhale, just underneath those bubbles.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}