{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cf50893b-24f8-5e58-ab28-62019fa55ccb","task_key":"default--validation--19796","task_revision_id":"1","upstream_id":"19796","short_description":"HellaSwag validation 19796","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to clean a gerbil cage [title] Put your gerbil in a safe place while you clean the cage. [step] Your gerbil should not be in his cage during cleaning as the chemicals can be harmful to his skin and fur. Before cleaning the cage, gently take your gerbil out of their cage and put him in a safe place where he cannot escape.\",\"endings\":\"['[substeps] Do not handle or take your gerbil outside during cleaning, as they may bite or scratch you. Make sure the cage is large enough for your gerbil to lay in comfortably.', \\\"For example, put the cage in the corner of the kitchen at work or one of your spare rooms. [title] Move the cage to an area you won't easily reach with a commercial cleaning tool.\\\", '[substeps] Find a secure area where your gerbil will not get hurt. Keep him away from other animals, especially cats and dogs as they might be predatory towards gerbils.', '[substeps] The gerbil should be very secure in his cage so that no one can break his cage or harm the rodents inside the cage. [title] Get a large, long cage for both gerbils.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}