{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e00aedc9-3a8e-5a64-8375-37f991aceb1d","task_key":"default--validation--18828","task_revision_id":"1","upstream_id":"18828","short_description":"HellaSwag validation 18828","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to clean a laptop [title] Get the right cleaning supplies for your laptop. [step] For the screen you should use a special screen cleaner. For the rest of the laptop, all you should need is a bowl, some soap (if you wish to clean your laptop with the keys removed), a can of compressed air, and some isopropyl alcohol.\",\"endings\":\"[\\\"[title] Turn off the laptop and remove the battery. [step] Don't risk shocking yourself or ruining your machine during the cleaning process.\\\", '[substeps] In the kitchen you can use a pair of tongs or a bottle of rubbing alcohol to clean your laptop. Isopropyl is a strong, thin chemical meant to clean your laptop.', 'Depending on the size of your laptop, you may require more than one cleaner. In addition, you may need different gloves.', 'Commercial computer cleaners (strong, caustic cleaners) can actually harm the screen. The recommended cleaning products will be $3.50 or much less.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}