{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"96bc4e11-1837-562f-b350-ce9560880eeb","task_key":"default--validation--48396--variant-989eec82ac2a","task_revision_id":"1","upstream_id":"48396--variant-989eec82ac2a","short_description":"HellaSwag validation 48396","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to decorate school supplies [title] Use stickers. [step] One of the easiest and best ways of decorating your notebooks, binders, and other school supplies is to use stickers. Head to the craft store or the hobby store and find some stickers that you like, and decorate your notebooks.\",\"endings\":\"['Consider pressing and gluing stickers for extra fun. [substeps] Glue art or designs onto your notebooks and binders in colorful boxes.', \\\"[substeps] Apply the stickers in unusual patterns for a neat effect. Even if you've just got a bunch of football stickers, if you make them into the shape of a face, or a peace sign, it might be cool.\\\", 'Brush (or paste) your stickers onto the outside of your notebooks. [substeps] You could also use a tacky glue on the inside or edges of the corner of the tabs of your notebooks as well.', 'They can be as small as math stars, as large as bike wheels, or as big as the words in english. [substeps] You can often find stickers online in large baskets and inside themed packs of things.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}