{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6208cd6b-9863-5813-8e0c-b91ec11f3af2","task_key":"default--validation--1830","task_revision_id":"1","upstream_id":"1830","short_description":"HellaSwag validation 1830","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to deal with an unexpected menstruation [title] Carry sanitary products, if you are expecting your period. [step] Have a change of underwear. Wear a thin, comfy panty liner if you are due on or, even if the super thin ones will help protect your clothes for a little while, especially if your flow is light.\",\"endings\":\"[\\\"Remember, all you really have to do is stop the blood making mess of you and your clothes, it's not a huge issue. [title] Use toilet paper, napkins, or tissues as sanitary protection.\\\", 'Launder your pants if you are going to work. Wipe your vagina up with mentheliac or fertile birth control oil if you have one.', 'Always remember not to use disposable underwear; if you are late, you can clean yourself up with a soft, sanitary bag or a tampon. Even if you miss an important moment on your period, you will still be going clean.', 'Cotton should be your safest item of clothing. If you are hoping for a light, breathable look, and have gotten on a pair of jeans and a shirt or dress, do so.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}