{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c6e7edc0-a204-5057-a7ff-a878351fe4f3","task_key":"default--validation--22801","task_revision_id":"1","upstream_id":"22801","short_description":"HellaSwag validation 22801","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to clean playtex sippy cups [title] Make sure to wash new sippy cups before using them. [step] Before giving a new playtex sippy cup to your child, wash it to remove any dust or dirt that may have come from the packaging. Even sippy cups that appear clean can contain germs and bacteria; washing the cup before use gives you a clean and fresh start.\",\"endings\":\"['[substeps] Remove all the cleaner that came with the new toys. Use vinegar or distilled water to clean the cup.', '[substeps] Different ways to wash the cups will be covered in the following steps. [title] Separate all of the parts of the cup.', '[title] Remove the barbell and loosen the screws holding the small cup. [step] How you do this may depend on your settings, so consult your settings on your small cup.', '[title] Clean the sippy cup outside before using it. [step] Before taking your new playtex cup outside to wash it, take your child outdoors to cleanse it.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}