{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e0ba084f-013e-51bf-beae-50e3ddfe8f71","task_key":"default--validation--12246","task_revision_id":"1","upstream_id":"12246","short_description":"HellaSwag validation 12246","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to prevent your cell phone from being hacked [title] Keep your operating system up-to-date. [step] As soon as apple or android tells you an update is ready, download and install it. Many hackers take advantage of vulnerabilities in out-of-date operating systems.\",\"endings\":\"['Uninstall any viruses or malware that may be affecting any computer. Replace any ware or programs you buy with the latest version of os x.', 'Updates patch these holes and make your phone more secure. [title] Install security software on your android phone.', 'Update your software as needed to protect your smart phone and desktop. For example, updating a kernel kernel is more or less expensive than replacing a cyac kernel if you only have a 64 gb kernel kernel kernel.', '[title] Use your device to avoid multitasking. [step] Technology is always changing and your device may go through many changes.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}