{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"98e7a03b-8639-5879-9c5d-6858947a9e5d","task_key":"default--validation--5528","task_revision_id":"1","upstream_id":"5528","short_description":"HellaSwag validation 5528","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to skin a squirrel [title] Get the squirrel wet. [step] To keep unwanted hair from getting all over your meat while you are skinning a squirrel, place the squirrel in the water at the edge of a stream or in a bucket. Lay the squirrel belly-down on a flat surface.\",\"endings\":\"['[substeps] You can also douse the animal with a hose prior to continuing if you are back at home. This step is optional and dependent on a water source; however, it is a handy way to reduce the spread of hair onto the meat.', 'If the squirrel is left outdoors your hands will rub against the water. As the squirrel ages, he will begin to form a bristle pelt.', \\\"Pour water into a bucket and swish it around to saturate the squirrel. Once you have soaked the squirrel's belly, the water will flow into the bucket and spread around the squirrel.\\\", \\\"Tip the squirrel's head backwards and place the wet spot directly over its tailbone. Use a stick to gently push back the squirrel's tail which cuts off the head and separates the back and upper body.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}