{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6d87964e-2b48-5b82-b3d1-d9932621c46a","task_key":"default--validation--41827","task_revision_id":"1","upstream_id":"41827","short_description":"HellaSwag validation 41827","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make a bracelet out of safety pins [title] Open up a safety pin and slide seed beads onto the pin. [step] Leave enough space on the top of the pin so that you can close it. You can use any color of seed bead you want, but avoid using ones that are too small, or you won't fit them onto the pin.\",\"endings\":\"['[substeps] Be sure that the beads are small enough to comfortably fit to the pendant without rubbing against the pins. [title] Take a strand of ribbon and tie it around the top of the safety pin.', '[title] Consider the size of your bead. [step] Use any size pin, but avoid sizes that are too small for your bead.', '[title] Pick up beads that are the same length as the pin, so that the sides match up. [step] Use a squeegee or a knife to carefully slide beads onto the pins.', 'A size \\\" 10/0 \\\" would be ideal. [substeps] This design will create a thick cuff-like bracelet that is covered with seed beads.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}