{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e011df42-aa7b-533a-82af-97c05946b688","task_key":"default--validation--1210","task_revision_id":"1","upstream_id":"1210","short_description":"HellaSwag validation 1210","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make a king cobra paracord bracelet [title] Pick up some paracord. [step] The amount of paracord you'll need depends on the size you want your bracelet to be. It's a good idea to put one foot of cord aside for every inch of bracelet.\",\"endings\":\"['Take this into consideration the size of your bracelet and how much of it you plan to braid. Each inch of paracord needs one inch of cord braided into three strands.', \\\"Measure your wrist with measuring tape and gauge how much you think you'll need based on that. [substeps] Paracord can be purchased relatively cheaply from a wholesale outlet.\\\", '[title] Cut 2/3 inch strips of stiff, flexible cord. [step] Start at your pinky pinky pinky pointer finger and place your thumb, middle finger, and pinky on one long strip.', 'It may seem ideal to have four ones, but can be a lot of work for very small children, but thinner rings hold up well. [title] Lay the metal end of each piece of paracord between three or four layers of tape.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}