{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3f900454-1e52-5067-b9d6-3c72530e978f","task_key":"default--validation--31108","task_revision_id":"1","upstream_id":"31108","short_description":"HellaSwag validation 31108","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make an adult blanket sleeper [title] Create your blanket sleeper's pattern. [step] Lay several feet worth of untouched christmas wrapping paper down on a large flat surface. Grab a pair of your night pants and a heavy long-sleeve sweatshirt on a flat surface so that both touch each other at the waistline.\",\"endings\":\"['( make this seem natural when placed down on the surface with the paper). Make sure the pieces are folded and laid in the same direction on the same side of the paper that should be folded over so that the white side is showing outward.', 'Cut or tear any fabric and lay it flat to form the pattern. [substeps] Throw away unwanted clothing and fabric after sewing the pattern onto your pillowcase.', 'Tear off a long strip of christmas wrapping paper and lay it under your blanket. Fold the paper down and tape it down to the pattern and next to your blanket.', 'Fold each remaining portion of paper in half to form the long sleeve front fabric. Make sure to double-check the pattern to make sure that it is not jagged with sharp edges.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}