{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d443c97e-8ed4-54f0-a111-cd3f5e359e46","task_key":"default--validation--30629","task_revision_id":"1","upstream_id":"30629","short_description":"HellaSwag validation 30629","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make rainbow bracelets [title] Select rubber bands. [step] Choose a rubber band for each color of the rainbow, e.g., blue, red, green, etc. Rubber bands.\",\"endings\":\"['Gold gold gold bands make for large bracelets, while silver gold bands provide a nice rainbow bracelet. You can purchase rubber bands from sephora or look at the bug, ball and snake stores for enthusiasts.', \\\"If you can't find all the colors, use markers to color plain rubber bands the colors that you are missing. [title] Put the rubber bands on your wrist.\\\", \\\"[substeps] If you're making rainbow bracelets for a child, you can choose colored rubber bands as a gift. How you use the bands will depend on whether you want to draw the pattern or use a contrasting color (e.g., blue wouldn't work well with black).\\\", 'Choose as many different colors as possible, since these rubber bands will be used to hold on concert tickets. They also create the movement that the artist will do while walking.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}