{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0c989781-8338-59fe-8d6b-15cf37996ad6","task_key":"default--validation--4077","task_revision_id":"1","upstream_id":"4077","short_description":"HellaSwag validation 4077","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to make diaper cupcakes [title] Open up the diaper, so that it is a long strip, and position it horizontally. [step] Make sure that the inside of the diaper is facing you. Do not unfold the adhesive flaps.\",\"endings\":\"['[title] Lay the diaper flat, right side down on your workspace. [step] The shape of your cupcake should resemble a breast, with either a round breast to make it curvy, and another to represent round up.', '[title] Lay the diaper strip on the flat surface. [step] Roll out the palm of your hand.', '[substeps] Try to keep the strips on the bottom of the diaper towards the center. If you unfold one of the strips at the bottom too far it could melt or become stuck in the diaper.', 'Try to use a plain white diaper; this way, it will look more like frosting. [title] Fold the diaper in half lengthwise.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}