{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d28185ae-30eb-528f-9fcb-adcee6e13353","task_key":"default--validation--160","task_revision_id":"1","upstream_id":"160","short_description":"HellaSwag validation 160","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make a donut bun [title] Buy a donut form or find a clean sock. [step] Donut forms can be purchased in the hair accessories section of the drugstore, but it's just as easy to make your own with a sock. You want a large, stretchy tube sock that's easy to roll.\",\"endings\":\"['[substeps] The roll should be approximately 50 millimetres (11 in) in length, with a hole in the middle for stuffing. The tube sock is washable, making it impossible to change a simple roll you rolled yourself.', 'If you want a slightly less large tube sock, buy a sock that is smaller. Roll up the sock into an accordion-like sock shape.', \\\"Use an old pair because you will be cutting the toe off. [substeps] It's best if you have a sock that's similar to your hair color, but any sock will work.\\\", 'Running in place when you roll the sock up helps distribute the hair evenly throughout the donut as it gets folded. [substeps] Pull out as much hair as possible.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}