{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cd846452-c607-563e-b2b2-78717df01f98","task_key":"default--validation--46937","task_revision_id":"1","upstream_id":"46937","short_description":"HellaSwag validation 46937","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to oil trumpet valves [title] Choose a valve oil. [step] Valve oil helps clean and lubricate the trumpet, which will help you play your trumpet more efficiently, and can be purchased at most music stores or at various online retailers. There are two types of oils that are recommended to use for brass instruments: synthetic-based oils and petroleum-based oils.\",\"endings\":\"['[title] Obtain a brass or other hardcase tumbler to store your brass or other materials. [step] Brass needs to have a lid, and the tumbler must be something made of tempered plastic or a pressure plate.', '[substeps] Synthetic oils are commonly found in brass instruments and are much like petroleum oil except they contain less oxidation. Petroleum based oils are not used to clean and lubricate brass instruments.', '[substeps] Synthetic oils use unregulated, high-efficiency lubricants and are not approved by the fda to provide lubrication for brass instruments. Petroleum oil can be purchased at most smaller, household instruments stores or online.', 'Deciding between the two is often a matter of personal preference and determining which works better with your instrument. [substeps] Petroleum-based oil works best with brand new trumpets.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}