{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0bea80f1-168a-59c7-8084-7b57f16f1bb6","task_key":"default--validation--48060","task_revision_id":"1","upstream_id":"48060","short_description":"HellaSwag validation 48060","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to store nail polish [title] Get some blank sticky labels and a hole punch. [step] Sticking labels painted with nail polish on the tops of your nail polish bottles is an easy way to organize them. By placing a painted label on top of each polish bottle, you will be able to quickly locate the color you want.\",\"endings\":\"['Find some blank white sticky labels and a hole punch to get started. [substeps] You can also use a hole punch that is in the shape of something, such as a heart or butterfly.', \\\"It's also somewhat less likely that the labels in your nail polish bottles will match up on a way the labels are painted. [substeps] To keep your nail polish bottles neat, make sure you place them in various spots where they won't get bumped.\\\", \\\"[substeps] Make sure the label states the color you need as well as the type of nail polish you'll be using. You could spray paint the label one side of the bottle white, the other side another black, and so on.\\\", '[substeps] Keep in mind that you will need to repaint each label before you can repaint the labels. It is perfectly fine to repaint the labels now, but be cautious because you will likely allow paint or chemicals to accumulate.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}