{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e78a832f-6673-58c0-82aa-dd321302e617","task_key":"default--validation--13408--variant-e2d92b7f11f4","task_revision_id":"1","upstream_id":"13408--variant-e2d92b7f11f4","short_description":"HellaSwag validation 13408","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to grow a venus flytrap [title] Buy a venus flytrap bulb. [step] The most common, and easiest, way to start growing a venus flytrap is to purchase a bulb (or several bulbs) from a company that specializes in growing the plants. Search online to find a vendor from which you can mail order bulbs.\",\"endings\":\"[\\\"You'll be able to choose from among several varieties that have differences in shape and color. You may also be able to find a nursery in your area that sells venus flytrap bulbs.\\\", '[substeps] For example, if your own home includes venus flytraps, you could request that they specify specific foods that add glamour and contrast to their winter get-togethers. [title] Cut a bulb by pulling it out of the ground.', 'The people who use that collection will likely want to purchase 4 bulbs separately. When ordering types of bulbs, make sure that the ones that you choose are specialized enough to your species.', 'If a vendor does not have an established base, you will need to purchase a japanese-engineered bulb. For this purpose, you may wish to purchase a male venus flytrap bulb that in its mature stage is more sensitive to light than it is when fully matured.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}