{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f7ff7e1d-c7fa-5210-ab5d-682836c305ba","task_key":"default--validation--1081","task_revision_id":"1","upstream_id":"1081","short_description":"HellaSwag validation 1081","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to remove hard water stains from glass [title] Use an acidic cleaner for best results. [step] Combine salt and water and use the mixture to remove the mineral buildup. The salt acts as a scouring powder and can cut through the stain.\",\"endings\":\"['Apply the mixture to the area and use a clean cloth to rub the stain in a circular motion. Rinse the glass thoroughly when finished to remove all salt water residue.', '[title] Dab the stain with a paper towel to remove ink, but do not rub the stain. [step] The acidic solution works well on hard water stains due to it soaking into the fabric and removing the ink from the glass.', 'If you notice persistent deposits after use, you may want to try an acidic cleaner to get rid of them. A natural acidic cleaner will be less likely to damage glass and may be less expensive.', \\\"While you're cleaning, rub or file the stained area with a soft touch and rinse with water. [substeps] Any acidic cleaner should be fine regardless of whether you're using salicylic acid, cetaphil or natural household bleach.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}