{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f41f119a-bf50-5b1b-9e3a-b079b5bd2513","task_key":"default--validation--27196","task_revision_id":"1","upstream_id":"27196","short_description":"HellaSwag validation 27196","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to clean silk [title] Determine if the color will fade or run. [step] Spot test the silk fabric by blotting a water-moistened cotton ball on a hidden point in the garment, such as an inside seam. If the color does not fade or come off onto the cotton ball, the garment is safe to hand wash.\",\"endings\":\"[\\\"Water-soaked cotton will achieve the same effect but worsen hue. [title] Feel the fabric for the color change you're looking for.\\\", \\\"[substeps] Whenever using any cleaning product, such as vinegar or alcohol, first test it out on a hidden portion of whatever item you're washing. If any discoloration occurs, bring the item to a local dry cleaner to have it professionally cleaned.\\\", 'If color blends or discoloration remains after the stain, then you may need to take the garment to the dry cleaners to deal with the stain. [title] Decide whether you want to hand wash or just wash silk fabrics.', 'If you are not certain whether the color will fade or run, you should still wash the garment. [substeps] If dye or waterproofing the silk helps to keep it supple and creamy, it is safe to hand wash the silk.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}