{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e025e84a-50ca-5f6f-b40d-2ace84a91540","task_key":"default--validation--35418--variant-432af7542980","task_revision_id":"1","upstream_id":"35418--variant-432af7542980","short_description":"HellaSwag validation 35418","config":"default","split":"validation","body":"{\"activity_label\":\"Holidays and Traditions\",\"ctx\":\"[header] How to make a gift bag [title] Choose a material to work with. [step] You can make paper bags out of just about any type of paper you want, including: brown kraft paper, patterned scrapbooking paper, and wrapping paper. Cardstock is not recommended, however, because it is too stiff.\",\"endings\":\"[\\\"[substeps] If the paper is not the right size, cut it down. Make sure that it's a rectangle.\\\", 'If you want to make paper bags from scratch, get sheets of cardstock that are from ibm or another company. But what about fiber-packed paper? [substeps] If you use either plain or thick paper, the paper will shrink with age.', 'Some possibilities include : [substeps] Colored scrapbooking paper. Black cardstock is an option, but any color will work.', 'If you are making a gift bag for a friend or family, choose cardstock, patterned wrapping paper, or sewing paper. Felt or cotton materials are not good options, as they are too sticky and may cause tear marks.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}