{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4da9639c-f842-5906-ac5d-10fe88ed4766","task_key":"default--validation--34152","task_revision_id":"1","upstream_id":"34152","short_description":"HellaSwag validation 34152","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wear a purity ring [title] Take the time to really give it some thought. [step] Make sure you're are dedicated enough to wait. If you're already thinking about sex all the time and you don't plan on staying pure, you shouldn't wear a purity ring.\",\"endings\":\"[\\\"[substeps] If you aren't sure if you really do want a purity ring, it's okay to give it a couple of weeks to let the crystal on your ring become dry. That will help make it slightly thicker and make it easier to insert it into a woman's vagina.\\\", \\\"Don't make a promise to yourself you know you won't keep. But if you've decided that you are going to wait for sex until you get married, and you want the world to know it.\\\", 'The old saying says, \\\" to be virtuous, you have to have a sense of permanence. \\\" but if you don\\\\'t cultivate a sense of permanence, you won\\\\'t have the option of purity.', \\\"But that doesn't mean you aren't sufficiently committed, either. That makes it important to find the right ring for your body.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}