{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"60384ce9-8279-586d-96df-2a4f8a0e57af","task_key":"default--validation--17050","task_revision_id":"1","upstream_id":"17050","short_description":"HellaSwag validation 17050","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to clean your air filter [title] Locate the air filter. [step] The dry air filter will be made of paper or cotton gauze. Dry air filters tend to get dirty faster but they are also easier to clean than oiled air filters.\",\"endings\":\"['The dry air filter will be under the hood of your car, on top of the engine. The filter will be housed in a rectangular or round box.', 'These filters are also known as dacron air filters, which could also be cleaned by using buffing sticks. You will need to tap the air filter on its side to get the air out of its tube.', 'The dry air filter has a small mouthpiece and stands upright on a hanging wire. [title] Keep a set of cleaning tools nearby.', '[substeps] If your air filter is newer, you may have to buy a new one. Make sure to read the directions carefully so you understand how to properly clean it before you purchase one.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}