{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"18d3e4f0-f76d-50d8-9951-24e81a0448af","task_key":"default--validation--12622","task_revision_id":"1","upstream_id":"12622","short_description":"HellaSwag validation 12622","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to rock red lipcolor [title] Clean up your lips of any foreign particles (even peeling skin, even though that's not foreign) with cool water. [step] Dab until just dry (rub in some lip balm if your kissers are chapped, and let sit until lips are moist and smooth). [title] Apply a single coat of red lip color, using medium pressure.\",\"endings\":\"['[step] It doesn\\\\'t have to be black, because this is just lipstick for you. You can use many different shades, and you can\\\\'t have a crazy orange \\\" la la la la \\\" lip color.', \\\"[step] Once this is wet, smudge it around the outer corner (edge of lips) to smooth it out. Don't move it up too far because that can cause the color to run away and ruin the makeup.\\\", '[step] Rub it in so that your lips look perfectly red, the colour not too faint and not too brash either. A lip brush, if you have one, will work wonders, but your fingers should do too.', \\\"[step] Make sure it's applied evenly throughout your lips, and the brush color will be lost. This step is not absolutely necessary but most people will appreciate the finishing touches.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}