{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"24294e4f-4ed4-5642-9b75-e72c37349f89","task_key":"default--validation--28345","task_revision_id":"1","upstream_id":"28345","short_description":"HellaSwag validation 28345","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to clean mold from leather [title] Apply a layer of petroleum jelly. [step] Test the effect of the petroleum jelly on a small area that won't be noticeable. Apply the petroleum jelly on the moldy areas creating only a thin layer.\",\"endings\":\"['[substeps] The jelly can also be rub on with a clean cloth or simply applied onto painted leather. Avoid rubbing petroleum jelly on leather, as it will also cause the paintwork to flake off.', 'If the mold works quickly, then the petroleum jelly will remain on the entire jacket. [title] Press on mold with cotton balls to make gentle circular motions.', '[substeps] Avoid applying petroleum jelly straight to your leather. This may damage the fabric or cause mildew.', 'You may also use a suede cleaner but read the label of the cleaning product to ensure that it works on mold and mildew. [substeps] Nubuck can easily change colour so make sure the cleaning product you use has been tested before applying.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}