{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7c87b693-112b-523f-9fef-2445c56b11e5","task_key":"default--validation--13031","task_revision_id":"1","upstream_id":"13031","short_description":"HellaSwag validation 13031","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make a silk scarf [title] Go shopping. [step] The best silk fabrics are usually light georgette, organza and crepes. The best designs and widest variety can be found in privately owned fabric stores, but keep your eyes open.\",\"endings\":\"[\\\"Check at your local thrift store, fabric supply store, craft store or online. Since they charge thousands of dollars per wrap, you'll end up paying even more than the sales associate will, at a bargain price.\\\", 'A fresh silk scarf can last in storage for a long time. [title] Snag a scarf from a fifth-tier sale.', \\\"You might find a piece when you least expect it. Burned out velvet's and even crisp fabrics can be used for scarves, especially for wearing with an outdoor coat.\\\", 'A cheaper scarf at a thrift store or a secondhand store can be a perfect contrast. [substeps] Fabrics find in secondhand stores are very essential for draping over silk.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}