{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"17faabbe-d51c-5c40-a001-09d5f64dee06","task_key":"default--validation--35761","task_revision_id":"1","upstream_id":"35761","short_description":"HellaSwag validation 35761","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to clean a caique parrot cage [title] Replace the liner at the bottom of the cage. [step] The liner is usually made of newspaper or other paper. Using a face mask and gloves, take out the liner and replace it with fresh paper once a day.\",\"endings\":\"[\\\"[title] Clean the caique's eyes as needed. [step] Caique's eyes are flat and do not change colour very often.\\\", '[title] Remove any food feces from the interior of the cage. [step] You need to remove all the other food and feces off of the interior of the cage, along with any droppings, urine, and bacteria.', 'This will keep the cage free of droppings and stains. [substeps] If you have layered the paper in the cage, you can remove and replace only the top layer as long as water or droppings have not soaked through.', \\\"[substeps] As your caique gets older, it will extend its life by 7 , 000 feet (15 , 000 m). Don't spend more than 18 hours each week with your caique bathing.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}