{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4afe52e5-5340-5cef-8df8-f7156ac5ff8f","task_key":"default--validation--38658","task_revision_id":"1","upstream_id":"38658","short_description":"HellaSwag validation 38658","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to design a formal garden [title] Assess the lay of the land. [step] The design you choose for your garden will be influenced by the contours of the land you are using. You should also take into account its placement relative to your house or other nearby buildings.\",\"endings\":\"['[substeps] For example, if you plan on building a master garden in the upper half of your home, you may want to develop a crescent row of hills. At a long distance, this half row may be actually highly steep.', 'Consider the following factors : [substeps] Is the land your are using hilly or flat? This will affect the final look of your formal garden. Formal gardens are easier to create on flatter land, so you may want to look into leavening some of the hills on your property if possible.', 'Take into account the layout of the landscape. [substeps] For example, you might want to have an attractive and open garden with a thick, glistening lawn and a frosted porch.', 'You do not want to have one near your home that must be square. [substeps] For example, putting up one at the top of a corner overlooking a nearby building can prevent the foliage from blowing away.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}