{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e9932b6a-cbde-5e24-bbf3-2289f18c0fa0","task_key":"default--validation--16562","task_revision_id":"1","upstream_id":"16562","short_description":"HellaSwag validation 16562","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to braid beads into your hair [title] Select two to three strands of hair. [step] These are the strands that you are going to braid. You may pick strands on one side of your head or at the back of your head.\",\"endings\":\"['[title] Secure a piece of yarn at the nape of your neck. [step] Take hold of the strand of hair and tie it in an easy-to-knot knot around the middle of your head.', 'It may be easier to start with strands on one side. Pick thin strands if you are using narrow beads and thicker strands if the beads are wide in diameter.', 'Or you may choose strands of hair on the other side, if you are doing two or three strands of hair, and then tie the strands tightly to avoid the strands of hair between them spilling out over the side. [substeps] Loose strands of hair are the most useful.', '[substeps] Tie one of the strands of hair into the top of your head and the other around the side of your head. Form the braid with your fingers and cross the strands so that they form an x-shape with the beads.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}