{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5dc8b00c-71b3-5d3e-be82-a0c945be756e","task_key":"default--validation--26141","task_revision_id":"1","upstream_id":"26141","short_description":"HellaSwag validation 26141","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to accessorize with a bucket bag [title] Choose a reasonably sized bag. [step] Bucket bags can be a fashionable alternative to a purse, backpack, or briefcase, but you need to make sure your bag can fit everything you need. Take inventory of the largest items in your current bag, and find a bucket bag that is at least half an inch (about 1 cm) larger than your largest items.\",\"endings\":\"['[substeps] Look at the depth of your current bag, as well. Make sure your bucket bag will be able to expand in all directions to hold what you need.', 'Think about weight when picking a bucket bag. You want a bag that is firmly packed and compact to accommodate multiple items, and safely holds the things you would need most.', \\\"[substeps] Keep all the essentials you need, like keys, basic clothes and accessories, into your bucket bag. You'll need these items when you wear a bucket.\\\", \\\"[substeps] There's not much specific size detail about bucket bags. Stick with whatever works for you, while you're at it.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}