{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"33834bf1-86cb-508e-8192-62f57fab67f4","task_key":"default--validation--13529","task_revision_id":"1","upstream_id":"13529","short_description":"HellaSwag validation 13529","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to keep dogs out of gardens [title] Put up a fence. [step] Perhaps the most obvious way to keep dogs out of your garden is to put up a fence. You can pay to have a fence installed professionally (which can be expensive), or consider.\",\"endings\":\"['But obviously, it is recommended to make sure that your dog is not poking their face through the wall to sniff it. [substeps] If you do not have a fence, you could use a tall line as a barrier between your yard and the backyard.', 'The more expensive an fence, the less expensive the recourse will be. [substeps] If you want to put up a fence, estimate the amount of pedestrian access and space you need to accommodate dogs, and add or remove it in time.', \\\"It can provide some protection from hazards and keep dogs away from the garden. After all, you don't want your gardener to fall off his horse and stab you! [title] Hold your dog in a similar position.\\\", 'You may choose a fence made from wood, wire, or mesh, which can all be found at your local home repair store. [substeps] For small dogs, a 16 \\\" fence should be adequate.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}