{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2e183688-4d95-59ae-a36f-8ffb27b70baf","task_key":"default--validation--6295","task_revision_id":"1","upstream_id":"6295","short_description":"HellaSwag validation 6295","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to keep owls away [title] Yell or clap your hands near the owl if you see it. [step] Owls typically try to avoid human contact and noises. If you see an owl lurking around your property, be as loud as you can to chase it off.\",\"endings\":\"['Instead of yelling and clapping your hands at the owl, say something like \\\" no! \\\" and firmly follow the owl as it escapes. [substeps] If you\\\\'re calling a librarian to gather your supplies and then arrive promptly, look around for owls waiting to open their doors.', '[title] Close the gap between vehicles and trees. [step] Owls are particularly attracted to small vehicles, so if your tent is short distance, you may not need to move away from everything near the owl.', 'Sometimes simply increasing human activity in an area, like setting up and using a fire pit and chairs in your yard, can deter an owl from trying to hunt near your home. [substeps] The only type of owl known to attack people is the great horned owl, and it only does this if its nest and young are threatened.', \\\"[substeps] This can be loud if an owl chooses to approach your house or other locations, but if you're calling indoors or brushing up a tree, this is also loud and rowdy. [title] Use a glowing owl if it approaches your home.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}