{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"66569d89-699d-5ecd-8a81-83fcbb1d1e10","task_key":"default--validation--30107","task_revision_id":"1","upstream_id":"30107","short_description":"HellaSwag validation 30107","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make a candy stripe friendship bracelet [title] Buy or collect your supplies. [step] For the four string weave, you will need four pieces of embroidery thread (sometimes called embroidery floss) in two different colors. You can find these at your local craft store, or in the craft section of most major retailers, like walmart or target.\",\"endings\":\"['Look for two colors that complement each other, such as blue and green or red and orange. [substeps] You can also make an ombre bracelet with two shades of a color, like blue or red.', 'You will be able to buy wire cutters if you want to customize the bracelet. [substeps] The simplest ring necklace pattern can be found online.', 'You will also need three sets of stitches for each color of striped yarn. Once you have used all of the yarn, tie it into a knot.', 'All you need is enough thread to tie your stringing together to form a bracelet. [substeps] You can also sew together your pieces of embroidery floss to create a long series of small shirts or flower-print themed scarves.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}