{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c4fae590-7a7b-5b47-9bd5-1dc17f5b7b7a","task_key":"default--validation--28152","task_revision_id":"1","upstream_id":"28152","short_description":"HellaSwag validation 28152","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to clean silver jewelry with vinegar [title] Immerse your jewelry in white vinegar. [step] Place your jewelry in a clean mason jar or another suitable container. Cover it with white vinegar, so that it is completely immersed.\",\"endings\":\"['[substeps] Try adding fresh tea tree oil to your vinegar solution as well. The tea tree oil helps the coating hold on to the silver.', 'Turn on the temperature of the white vinegar and wait at least 15 minutes to let the vinegar dissolve. [substeps] Do not remove your jewelry as soon as it is submerged in white vinegar.', 'You can let it sit for two or three hours, depending on how much tarnish you need to remove. Then, rinse and dry the jewelry.', 'Make sure that the lid is tightly sealed before handling the glass bottle. [substeps] White vinegar will dissolve the crystals and help remove the crystal polish from your jewelry.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}