{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"403b3a30-f4f0-5c53-a0fc-bd4a9f55894d","task_key":"default--validation--46482","task_revision_id":"1","upstream_id":"46482","short_description":"HellaSwag validation 46482","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make your period eco friendly [title] Be open to trying something new. [step] It's different than what you're probably used to, but definitely worth it. One misconception is that reusable products are unclean and gross.\",\"endings\":\"[\\\"They actually have a lot of health benefits, like less risk of infections and toxic shock syndrome. It's fairly easy to clean them.\\\", 'A good rule of thumb is that you should use sanitary napkins or toilet paper for the most sanitary work. But, you have probably heard that using reusable containers and cans with glass tops are already good options for wearing underwear.', \\\"When you put something in a relatively clean container, it necessarily washes out the germs. This doesn't help you in almost every situation, but it can help you during the first few days.\\\", 'First of all, they are often not sanitary and babies, women and children can sneeze and twitch. [title] Ask for a = next to replace your old one.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}