{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"222b6923-f3b2-5ed6-97d6-d184f5828fe7","task_key":"default--validation--49662","task_revision_id":"1","upstream_id":"49662","short_description":"HellaSwag validation 49662","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to remove lint from clothes [title] Use a lint roller. [step] You can find them in the laundry department of a supermarket, as well as in fabric stores and pet stores. Peel the wrapper off the tube part, and roll it across the garment.\",\"endings\":\"['It should pick up the excess lint after approximately a minute or two. [substeps] Give the lint roller a break for an hour or so when you are finished.', 'Use and up-and-down motion. As you continue to roll, you will notice that the lint roller is become less tacky.', \\\"[substeps] Do not use a lint roller, it's highly flammable. Likewise, as sharp objects can scratch or poke you, try and use a lint roller as close to the needle as possible without breathing.\\\", 'Then, carefully strip the entire item, making sure to get away any lint that may have filled the tube. Then, take the lint roller and throw it in the trash.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}