{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d4fbba07-cba3-5a7a-acfa-e26fb6de5ce4","task_key":"default--validation--36482","task_revision_id":"1","upstream_id":"36482","short_description":"HellaSwag validation 36482","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make a bracelet out of safety pins [title] Open up a safety pin and slide seed beads onto the pin. [step] Leave enough space on the top of the pin so that you can close it. You can use any color of seed bead you want, but avoid using ones that are too small, or you won't fit them onto the pin.\",\"endings\":\"['A size \\\" 10/0 \\\" would be ideal. [substeps] This design will create a thick cuff-like bracelet that is covered with seed beads.', \\\"Choose a color that's not too heavy. [substeps] These are green, rose, and gold beads.\\\", 'Many safety pins are filled with seed beads, which you can tie into ribbons. [substeps] Plastic bands would work, though you should also get ones with a snap in the middle.', \\\"[title] Slide a slip of how-to tape to the pin. [step] You can use any color of tape you want, or avoid ones that are too small, or you won't fit them onto the pin.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}