{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"56b20004-5a21-590a-a7f1-63274c048da1","task_key":"default--validation--22438","task_revision_id":"1","upstream_id":"22438","short_description":"HellaSwag validation 22438","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to know if you're wearing the right size high heels [title] Get your foot measured at a shoe store. [step] Most, if not all, shoe stores will have the necessary tools to properly measure your foot. Some stores have a carpeted scale on which you can place your foot to find your shoe size.\",\"endings\":\"['Knowing your size will keep you from missing out if your foot appears incorrect. [substeps] You can also grab a tiny mirror to use during this process.', \\\"Some stores will have a measuring tape hanging on some of the front walls, allowing the store's clerk to measure your feet. [substeps] Find out which gauge you should buy.\\\", \\\"[title] Pick shoes that don't have high heels. [step] Have a friend or family member grab your foot so you can see your shoe.\\\", \\\"There are also devices that can fit around your foot and adjust to your foot dimensions in order to determine your shoe size. If you can't find these tools around the store where you are shopping, simply ask a store employee if they could help you measure the size of your foot.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}