{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3b52547c-1638-516a-8605-028cce0a8682","task_key":"default--validation--1871","task_revision_id":"1","upstream_id":"1871","short_description":"HellaSwag validation 1871","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to make tea tree oil [title] Obtain tea tree leaves. [step] This can be difficult, depending upon the region in which you live. Melaleuca alternifolia grows in usda hardiness zones 8 through 11 and it is available for purchase.\",\"endings\":\"['Other tropical areas may also be able to find the small tree at gardening centers or through magazines. [substeps] For people in cooler climates, it will be much harder to track down the raw materials needed to make this oil yourself.', '[substeps] If you live in an area that does not have melaleuca, you may decide to acquire 20-25 leaves from a plant that has melaleuca in its first year of life. [title] Dump 3 cups of water over the dried tea leaves.', 'You can plant this in all locations, including south america, the british virgin islands and costa rica. This oil is safe for home use for you.', '[title] Place the leaves in a glass jar. [step] If your jar has a lid or glass bottom, make sure that all the leaves are placed in as neat a fashion as possible.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}