{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"577dfbba-ce3a-50e0-b89b-0bc1ee818535","task_key":"default--validation--15492","task_revision_id":"1","upstream_id":"15492","short_description":"HellaSwag validation 15492","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to edge a lawn without an edger [title] Make a straight line with 2 wooden sticks and string. [step] Set 2 wooden sticks into the ground 1-2 inches (2.5-5.1 cm) from the edge of your flower bed or walkway. Then, tie a piece of string to each stick to form a straight line that will become the edge.\",\"endings\":\"['[substeps] Any stand-apart string will suffice. It can consist of any age and material, but white, black, or even red is the standard.', 'There should not be any loops or gaps between the sticks. [substeps] Tie the string to the stick to secure it to the height you desire in your garden bed as the bow points outwards.', 'Make sure to leave at least 3 feet (1.9 m) of spare string running between your 3 sticks. [substeps] Make sure to use equal lengths of long string and cut it so that the entire surface of the flower bed is packed with string.', '[substeps] Setting the line before you start edging the lawn will keep your work neater. You can also try to edge by eye, but it may look sloppy.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}