{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ef3d2d56-78f1-578c-9b04-60b6a1034be1","task_key":"default--validation--32880","task_revision_id":"1","upstream_id":"32880","short_description":"HellaSwag validation 32880","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to get out of a car gracefully without showing your underwear [title] Prepare your clothing. [step] Before you even open the door, get your skirt or dress ready for your exit. [substeps] If you're wearing a miniskirt, pull the cloth down to cover as much of your thighs as possible.\",\"endings\":\"[\\\"Since this is the most embarrassing thing to do in front of an open doors, think ahead about what kind of attire you'll be wearing and how you'll want to hide those covered parts of you. In the worst case scenario, a car wouldn't make it to your destination.\\\", \\\"Keep it tightly on your lower body so that women don't get a glimpse of your crotch. Ensure that you have a seductive neckline and that it reflects soft light.\\\", 'Then, you can avoid showing your underwear. Wear a skirt or dress with a waist that is no more than 1/8 inch.', \\\"If you're wearing a longer skirt, dress, or train, pull it down to cover your knees and push any excess fabric toward the center of the car so that it doesn't get caught around your feet when you exit. [title] Open the door as far as you can stretch your arm.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}