{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"27959254-eeb3-53c1-a1f1-103614490ad7","task_key":"default--validation--6412","task_revision_id":"1","upstream_id":"6412","short_description":"HellaSwag validation 6412","config":"default","split":"validation","body":"{\"activity_label\":\"Cars & Other Vehicles\",\"ctx\":\"[header] How to preserve peppadews [title] Wash the peppadews. [step] Rinse the peppadews under running water, using your fingers to gently scrub away any dirt or debris. [substeps] Completely dry the peppadews with clean paper towels before continuing.\",\"endings\":\"[\\\"If your peppadews weren't washed, you may need to rinse them again if they get dirty after washing. Do not use bleach or other harsh chemicals if you're deciding to preserve them.\\\", 'Do not leave the peppadews in the water for more than one day or they may become spoiled. Consider transplanting (covering, decolonizing, curing) the peppadews yourself.', '[title] Package in freezer-safe bags. [step] Pack the whole peppadews in a thick resealable plastic bag labeled for freezer use.', '[title] Run a rectangular ash pan under warm running water. [step] Once the peppadews are thoroughly coated, run them under water again to remove the dirt and debris.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}