{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"73607e84-c738-51e5-a640-54c01b9022ae","task_key":"default--validation--14787","task_revision_id":"1","upstream_id":"14787","short_description":"HellaSwag validation 14787","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to clean oxidized copper [title] Verify the metal is copper. [step] Position a magnet close to the copper. If the magnet sticks to the metal, the item you are cleaning is only copper plated.\",\"endings\":\"['[substeps] Look for components with shiny parts rather than chunky or opaque metals like aluminum. Check, especially if you are a saucepan collector and you use a metal plate for your copper, or vice versa, to protect the metal.', '[title] Clean electronic tools off your copper device. [step] Depending on the tools you use, you may also want to clear the mechanisms that were used to handle the dials and knobs.', 'Oxidization-a chemical reaction between an solvent and water from a source (like a body of water) may cause oxidization. Use a soft cloth to gently wipe down your plated copper.', \\\"If the magnet doesn't react, your item is likely copper. Plated copper generally needs to be treated more gently than pure copper.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}