{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6924c686-0aed-57cd-8263-065fe899f713","task_key":"default--validation--17111","task_revision_id":"1","upstream_id":"17111","short_description":"HellaSwag validation 17111","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make leather gloves [title] Trace your hand onto paper. [step] Lay your non-dominant hand flat on top of a piece of paper, keeping your fingers closed. Your thumb should extend out at its natural angle.\",\"endings\":\"['Trace your hand around the paper, carefully keeping the thumb out of the way. Do not over-cut! This process will require two to three minutes.', 'Draw a line down the middle of the paper from your forefinger to your thumb, from the tip to the tip of your first finger (starting from the first knuckle). Use your other hand to trace a gentle bow shape around the outer edge of your thumbnail.', 'Draw around your entire hand, working from one side of the wrist to the other. [substeps] Your hand should be in the middle of the paper with your index finger and thumb pointed toward the center.', \\\"Trace your hand's shape with your pencil or an exacto knife. [title] Cut around your hand, leaving a 1/3 inch (0.6 cm) seam along the edge of your hand.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}