{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a7693d1a-448d-5bb3-902d-64fce1d0629b","task_key":"default--validation--9546","task_revision_id":"1","upstream_id":"9546","short_description":"HellaSwag validation 9546","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to clean jewelry [title] Use a soap and water solution. [step] Gold jewelry is most easily cleaned using a simple solution of soap and water. Fill a bowl with warm water and a few squirts of mild dishwashing liquid.\",\"endings\":\"['Place your gold jewelry in the bowl and let it soak for about 15 minutes, then use a soft-bristled toothbrush to gently scrub away the grime. Dry your jewelry with a soft cloth to restore its shine.', 'Add a few drops of food coloring (ie, green, etc. ) and a dab of mild dish detergent.', 'Set the jewelry down on a clean towel, and gently swish it in the solution until it removes dirt and debris. Be sure to stop and dry the jewelry as soon as the soap runs clear.', 'Place the jewelry in the bowl and use a small cleansing sponge to spread the solution over the jewelry. Rinse the jewelry with warm, soapy water as soon as you are done, as rinsing it off may dry your jewelry in the long run.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}