{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6a1756f5-9716-5062-9c5e-905e9171f972","task_key":"default--validation--38562","task_revision_id":"1","upstream_id":"38562","short_description":"HellaSwag validation 38562","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to take out the trash [title] Secure the bag. [step] Use the extra room at the top of the bag to securely tie a single knot so that the garbage does not fall out of the bag. Sometimes trash bags come with twist ties.\",\"endings\":\"['Using these ties makes it easy to secure the contents of the bag. Gather the entire opening of the bag, close and place the twist tie around it.', 'Avoid using these as this can easily give the bag away easily. Also be careful when hanging bag, these bags are also easier to tip over and might break if tossed in the trashcan.', 'You should fasten these ends using zip ties or hair clips. Some bags even have tabs around the edges to keep the trash from falling out of the bag.', \\\"This prevents rats from getting into your bag and will keep you safe if you get caught. [substeps] It's most commonly used by rats when they are looking for a food source.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}