{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e9f6daed-4d03-5257-ac92-af75205b26bf","task_key":"default--validation--26926","task_revision_id":"1","upstream_id":"26926","short_description":"HellaSwag validation 26926","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to color your nails with sharpie markers [title] Trim your nails and file them! [step] Your nails should be neatly trimmed and filed. [title] Look for regular nontoxic markers. [step] Try using crayola markers, they do make good fake nail polish! Try to have them be washable, in case you don't want it and want to wash it off.\",\"endings\":\"['Plus, it makes your nails look fun! [substeps] If you are using glitter, be sure to cut the back of the dip tool up to be completely bare. [title] Before you start coloring your nails, use the glitter around them.', \\\"Also, although the fat markers make it messier to paint nails, use them because they won't run out of ink as quickly as thinner ones. Watercolor paints work well too, as they wash off better.\\\", '[title] Hold your old bottle of nail polish in your hand, and go over your nail with the plain colored polish. [step] Make sure to wash it in warm water before putting in your new nail polish.', \\\"Sharpie marker are flammable and can ruin them if you don't wash it off sooner. Do not use regular, non washable markers.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}