{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d7ba542b-8134-5cf5-8389-b05c5cfe666a","task_key":"default--validation--1747","task_revision_id":"1","upstream_id":"1747","short_description":"HellaSwag validation 1747","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to layer sheer clothing [title] Put a sheer shirt on over a bra. [step] This is the easiest and most common way to layer sheer clothing. Put on a bra or tank top (if you don't want to show your stomach) for the underneath layer.\",\"endings\":\"[\\\"If you want to make sure your shirt doesn't completely cover your breast, you can wear tights to cover your skin. This creates the appearance of your bra through the shirt.\\\", \\\"[substeps] You don't have to layer a bra with top. Keep in mind it might get lumpy as you layer.\\\", 'For the top, sheer layer, you can wear a sheer t-shirt, button-down, or a lace shirt. You can find lace shirts with designs, such as flowers, for a more interesting look.', '[substeps] Cut a slit into the cotton about 1/8th of an inch from the edge of the neckline. Tie a piece of yarn to your blouse-this will wrap around and hide the slit between the top and bottom of the shirt.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}