{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9926f84d-4116-585a-b59a-dfaac6fcf13e","task_key":"default--validation--9199","task_revision_id":"1","upstream_id":"9199","short_description":"HellaSwag validation 9199","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to rip jeggings [title] Put your jeggings on. [step] It's hard to estimate exactly where distressing lines will fall if you're not wearing the jeans. Prior to distressing, slip the jeans on so you can pinpoint the spots where you want your jeans to rip.\",\"endings\":\"[\\\"[substeps] Get into the habit of searching around for these lines whenever you're ready to rip. Ask a coworker to give you a hand.\\\", \\\"[title] Mark the areas you want to distress with pen or chalk. [step] While you're wearing the jeggings, take a piece of chalk or a pen.\\\", '[substeps] With jeggings on, you will have to remove them one at a time. You will have to pull them off one by one.', 'Help your jeggings get wet in order to wash them efficiently. [substeps] Rub some food dye on your jeans before you begin distressing them.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}