{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0f7bb06d-3c9f-54c5-bb47-f36056426b8b","task_key":"default--validation--17535","task_revision_id":"1","upstream_id":"17535","short_description":"HellaSwag validation 17535","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to use a swiffer sweeper multiple ways [title] Dust walls and hard-to-reach areas. [step] Use your swiffer on more than just your floors. Take advantage of the easy handle and cleaning cloth to reach high-up or vertical surfaces of your home.\",\"endings\":\"['[substeps] Try dusting dirty walls, baseboards, ceiling fans, cabinets, and more with a dry cloth on the swiffer. Check to see how a given surface will respond to cleaner before using a wet cloth for this.', '[title] Use the swiffer to clean furniture or bureaus. [step] Pull drawers and lamps over from the walls in communal rooms and display them at each door.', 'Check underneath to see if there are any nooks and crannies that might be easy to clean. This technique is used as you remove pesticides and chemicals from surfaces.', 'Lightly dust floors and windowsills to inspect for dirt. Rarely, you can also use a remove cloth solution to correct imperfections.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}