{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7a1fe22f-9412-56f9-a84a-c556a88bbceb","task_key":"default--validation--1385","task_revision_id":"1","upstream_id":"1385","short_description":"HellaSwag validation 1385","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to remove clothing labels [title] Assess your label. [step] Labels are made of diverse materials and sewn into clothing in a variety of ways. You must remove them with care, or you risk accidentally tearing your clothing with the seam ripper.\",\"endings\":\"['Remove all labels from your clothes, and if need be, sew to remove them completely, then clean with a butter knife, water, detergent, or any other special techniques you might wish. When you do need to remove a label, the first step is to call a cleaners using a machine-type kit and then scrape off the label.', '[substeps] Some labels can be stripped off by applying a new layer of fabric with little seam allowances. Peel this off completely if you intend to remove the label.', '[substeps] With labels on, you usually just slip the label off using tweezers. For clothes that are synthetic, you can remove the label using a cosmetic tool.', '[substeps] Look for the best approach and point of entry to begin removal. Make a mental note about the type of material the label is made of-is it made of soft fabric or something a little more rigid and paper-like? [title] Check for multiple tags or labels.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}