{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"76865e7b-c3aa-5e1d-98e7-a4ff5df8bd04","task_key":"default--validation--21951","task_revision_id":"1","upstream_id":"21951","short_description":"HellaSwag validation 21951","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to zone agricultural land for use as a wedding venue [title] Talk to your neighbors. [step] Your neighbors can be your strongest supporters or your most bitter rivals if you want to rezone your property. The best way to potentially get them on your side is to keep them informed of your plans every step of the way.\",\"endings\":\"[\\\"[substeps] In the early stages, let them know about your plans and reassure them that you will keep them informed on developments. Give them the chance to ask questions, and do everything you can to assure them that events you host won't cause any disturbances to them or their property.\\\", \\\"[substeps] Make sure you know your neighbors' lives and can help them out if they disagree about your plan. Give them written correspondence so they can be helpful to you if needed.\\\", \\\"This means that if your neighbors are anything but informed, you should talk to them as well. If you don't know them well enough to keep detailed notes, reach out to them and ask them to help keep a record of your plans.\\\", \\\"[substeps] Don't forget that some of your family's acreage may need transplant rights. Always decide ahead of time what kind of liens will be put on what land you are planning on and why.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}