{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2d7f986c-618c-5eaf-9fb7-9d024e43f074","task_key":"default--validation--24281","task_revision_id":"1","upstream_id":"24281","short_description":"HellaSwag validation 24281","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to grow early strawberries [title] Consider the average summer temperatures and level of humidity in your area. [step] Environmental factors like the average summer temperatures and the level of humidity will affect how well certain cultivars grow in your climate. Some cultivars thrive in hot, humid climates while others do better in dry, cooler climates.\",\"endings\":\"['[title] Choose a location with good air circulation. [step] Ideally, a healthy way to grow strawberries is to choose a spot that receives direct sunlight.', 'They do not flourish in dry, cloudy, or temperate climates, so you should do your best to plant the plants in low humidity spaces as well as in unhumid or cool areas. [substeps] Choose the best spot for the growing of the strawberries you wish to grow in.', 'To determine the ideal climate for seedlings, look online or by walking by a local nursery or garden center. [substeps] Try to plant your strawberries in partial shade.', '[title] Purchase strawberry plants from your local nursery or garden center. [step] They should have a good selection of cultivars that will do well in your area.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}