{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8661dd86-0c13-5dac-b218-b767b6b3db76","task_key":"default--validation--44962","task_revision_id":"1","upstream_id":"44962","short_description":"HellaSwag validation 44962","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to clean a cockatoo cage [title] Put a fresh liner in the cage. [step] Take out the old liner at the bottom of the cage and replace it with new liner. You may use newspaper or paper as liner for the cage.\",\"endings\":\"['If the cleaner makes the cage feel sticky, squeeze some and dip it in a cup filled with the old liner and let it sit until it dries. Then, dip another liner and put the old liner back in the cage.', '[substeps] Do not put liners in the part that will be slippery, like the top of the cage. You can also use plastic or glass liners instead.', 'Put fresh liner in the cage at the beginning or the end of the day so it is clean. [substeps] If you have layered the paper in the cage, you can remove the top sheet so the liner is clean and fresh.', \\\"If you replace the old liner then you are ruining the bird's cage. [title] Place the new liner in a bucket.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}