{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"96a4b31e-9886-5233-87e2-163413371082","task_key":"default--validation--29905","task_revision_id":"1","upstream_id":"29905","short_description":"HellaSwag validation 29905","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to decorate your nails [title] Pick a good spot to decorate your nails. [step] You'll want your nails to match, so they should all be the same length. Start by cutting them down, and then use a file to smooth out the rough edges.\",\"endings\":\"[\\\"Then, use nail trims around the entire nail, making sure you get all of the way around. [substeps] On the wrong nail, or get nail trims that are similar in appearance as your real nails, you'll end up with redness and blistering.\\\", '[substeps] See below for tips on picking the right size nail to fit the personality of your doll. [title] Choose a decoration that is appropriate for your doll.', 'You can also use your file to shape your nails into either a rounded or squared edge, depending on your preference. [substeps] Be sure to file in one direction, from the outside toward the center of the nail, rather than in a back-and-forth motion.', '[title] Decide if you want to apply cuticle oil or polish, and pick whichever works for you. [step] For example, if you have very coarse cuticles, ask to have your cuticles dry between treatments.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}