{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8501f6ae-f9af-5073-a66e-f0edc1f33c0c","task_key":"default--validation--30269","task_revision_id":"1","upstream_id":"30269","short_description":"HellaSwag validation 30269","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to build a pergola [title] Research zoning laws in your area. [step] States and towns have zoning laws that restrict certain kinds of construction. Call your local building commission or zoning department to make sure that you can build a pergola on your property.\",\"endings\":\"['The rules usually apply to all sorts of projects, including, but are not limited to : [substeps] Pergola construction your own home. You may end up with unwanted pets or children.', 'You also can download blueprints and maps for municipalities in your area to familiarize yourself with. There are a variety of learning opportunities available through student offices and associations.', 'You will need to find out how much it will cost to construct one of the perches. Search online to find zoning laws that specifically allow you to build pergola perches.', \\\"You can also find zoning ordinances on your local town's website. [substeps] In some cases you'll have to get a permit before you can start building.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}