{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bc2af5c5-6454-5629-8cac-e66ad219b2d5","task_key":"default--validation--15688","task_revision_id":"1","upstream_id":"15688","short_description":"HellaSwag validation 15688","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear novelty knits [title] Explore a variety of knitted tops. [step] Novelty knits are often worn as a shirt or other top with elaborate designs, illustrations, or vibrant colors. This can be great for something like a silly sweater party.\",\"endings\":\"['Teams of 50 to 100 people can wear a very silly knitted top with great illustrations, graphics, or interesting designs for a top. [substeps] If this fits with the overall theme of your outfit, subscribe to the evening magazine to get suggestions for different holiday sweaters.', 'Novelty christmas knits are popular around the holiday season. You can also look for things like novelty knits in vest form, or jumpers, turtlenecks, and cardigans.', 'A different colour or pattern looks great on a novelty pair. [title] Find a piece of knitted fabric or felt.', 'Or for something more decorative like a pony collar, you can explore a handful of novelty knits. Here are a few categories of novelty knits you should consider : [substeps] Novelty knits are most often made in those days when there is hand print printing printed on the fabric.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}