{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2891bcb7-1900-54ab-9374-e32009e09e96","task_key":"default--validation--50092","task_revision_id":"1","upstream_id":"50092","short_description":"HellaSwag validation 50092","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to tie a silk scarf [title] Tie it in the bandit style. [step] This is one of the most classic styles for a square silk scarf. Lay your scarf flat on a table.\",\"endings\":\"['Lay your other scarf on top of the first and use your fingers to trace the opening. The cut edge of the shorter side needs to be closest to the vertical seam.', 'Fold it in half lengthwise, then tuck the folded ends into the side of the scarf. Pinch the folded ends of the scarf together.', 'Fold two of the corners to meet each other, creating a triangle. Place the scarf around your neck with the wide triangle point over your chest pointing down.', 'Hold the corners of your scarf in one hand and slip the ends inside out. This will form a small depression in the center of the scarf.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}