{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"623828f3-cecd-5a1a-8106-e19caae89c95","task_key":"default--validation--17105","task_revision_id":"1","upstream_id":"17105","short_description":"HellaSwag validation 17105","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to do a monthly computer clean up [title] Vacuum it out. [step] Dust can cause many parts of a computer to overheat. To clean out dust from inside the computer you will need a can of compressed air or a small air compressor.\",\"endings\":\"['Power up the compressor to be sure it is running properly. [title] Make sure your computer is in good working order.', \\\"You can buy spray cleaner at any hardware store, and if possible, also purchase air fresheners that can clean the system from every possible angle. Follow the manufacturer's directions on the label.\\\", 'This kind of vacuuming will allow you to swish dust around until the inside is cleaner. You can also use a plastic tupperware container.', 'Shut the computer down and remove any devices that are plugged in. Remove the side panel from the computer case and begin to blow the dust out.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}