{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"65af6a62-522e-5ff6-b28b-7d8d380e1bfb","task_key":"default--validation--14184","task_revision_id":"1","upstream_id":"14184","short_description":"HellaSwag validation 14184","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to make lavender breath freshener [title] Choose the best dry culinary lavender. [step] Many grocery stores sell lavender stems either in the spices or vegetable aisle. Farmer's markets are another great source.\",\"endings\":\"['You will need an estimated 1 tablespoon (14. 8 ml) of dry lavender for this recipe.', 'Call a local yard sale if your mom packed your garden with lemon/lime or other organic herbs and seeds. [title] Scoop out a cup of air freshener in a jar.', 'These produce more fragrant produce, rather than a more pungent source of odors. The moisture of the berries and herbs causes their leaves to produce more harsh, bitter scents.', \\\"But if your home is very dry or stuffy, you should hire herbs that are hardy. [substeps] If this is a decision for you, go for the organic organic herbs, as they'll be almost entirely natural and can become the freshest scent.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}