{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"63821644-7259-5950-9b57-816f745962ca","task_key":"default--validation--35597","task_revision_id":"1","upstream_id":"35597","short_description":"HellaSwag validation 35597","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to tie a lasso [title] Grab a length of rope. [step] For a lasso, the exact length of rope you use isn't important, provided you have enough length to tie your knot, form the loop, and twirl it above your head. Any extra length can be coiled up and carried with you.\",\"endings\":\"['[title] Wrap the rope around your hand in a spiral shape. [step] A single loop wraps around the point of your hand, forming a loop with the rope.', \\\"For adults, about 30 feet (9.1 m) of rope is plenty; for children, shorter may be better. [substeps] If you're just practicing, nearly any type of rope will suffice.\\\", '[title] Tie the loop by tying the end of the rope as far forward as possible. [step] Make sure that the rope stops at the top of your head.', '[title] Wrap the rope around your index finger. [step] When your thumb and forefinger are at rest, place the loose ends of the rope around the center of your index finger and thumb.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}