{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cb1bd717-c33d-59c5-9f8a-1aa7485bc16b","task_key":"default--validation--34670--variant-bf1b8c8b7746","task_revision_id":"1","upstream_id":"34670--variant-bf1b8c8b7746","short_description":"HellaSwag validation 34670","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to clean shoe insoles [title] Remove the insoles from your shoes. [step] Lift the insoles from the footbed and slide them out the opening. Brush off any loose dirt, dust, lint or other debris so that you can focus your attention on the more soiled spots.\",\"endings\":\"[\\\"Rinse the shoes off under cold, running water. [substeps] If you prefer to put less debris on the shoes' insoles, you can skip this step.\\\", 'Do this repeatedly as you reach for the dirty shoes. While you remove the insoles, be sure not to toe the insole to prevent it from sliding out.', 'Do not stuff the insoles too deep into the shoe, or you risk pushing dirt and debris into the shoe itself. [substeps] Gently smooth the insole with a rubber buffing tool to prevent scratching.', \\\"Place recently worn insoles somewhere nearby to air out before you begin cleaning. [substeps] If your shoes don't have removable insoles, you'll have to carefully clean inside the shoe by hand.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}