{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0fcaf560-7e36-5522-908f-433ba0ff2219","task_key":"default--validation--42648","task_revision_id":"1","upstream_id":"42648","short_description":"HellaSwag validation 42648","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make a 4 strand braided bracelet [title] Choose your colors. [step] You can have any number, from 1 to 4. 2 colors is pretty easy to work with, but have fun and be creative.\",\"endings\":\"['Many people wearing a strand of hair dye on their head color their hair a simple color. Use dyed hair dye for haircolors or a color like black.', \\\"You can use muted tones for an earthier look, or bright colors for a more festive feel. For this example, we'll use black and blue strands.\\\", \\\"[title] Put your jewelry in a pot and use that to mix greens and blues together. [step] If you do not have anything handy, you may want to see this tutorial, if you haven't already.\\\", 'Decorate the braids with ribbon and beading thread, as well as other details. You can draw \\\" flowers \\\" or \\\" snow stars \\\" across the braids.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}