{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0173176c-ae36-5f0c-8300-cf8e3947ee2d","task_key":"default--validation--31756","task_revision_id":"1","upstream_id":"31756","short_description":"HellaSwag validation 31756","config":"default","split":"validation","body":"{\"activity_label\":\"Cars & Other Vehicles\",\"ctx\":\"[header] How to row a drift boat [title] Practice using a backstroke. [substeps] Until you are comfortable with your skills, practice your backstroke in a calm area of the river where there are few obstacles and you can easily control the boat. Sit in the row seat facing the bow of the boat (the bow is the high end of the boat).\",\"endings\":\"['Place each oar in its oarlock. Face the drift boat downstream so that the bow (and you) are facing downstream.', \\\"[title] Open the gap in the nose of the boat by placing your heels on the keel (the rounded rear end of the boat). [step] It should move you a bit so that the boat's nose remains against the keel.\\\", 'Your head should be level with the water, above the bow, and your feet should be hip width apart. You should bend your knees and put weight on them to keep you from falling backward while rowing.', 'Bring your line towards you, but rather slightly out to the side, without using just the line. Lean forward in your seat by slipping your rod underneath the surface of the water and pushing through the water until your line is at the right angle to the water.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}