{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0c8a9252-e801-5cf5-9463-88322fd421e8","task_key":"default--validation--7388","task_revision_id":"1","upstream_id":"7388","short_description":"HellaSwag validation 7388","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to trim parrot claws [title] Make sure your parrot has bonded with you. [step] You should know your bird's body language before attempting to trim its claws. If your bird does not feel comfortable with you, you should wait until you've bonded with it a bit before trimming its claws.\",\"endings\":\"[\\\"It will not be willing to listen to you trying to trim its claws down to the correct length. [substeps] When you handle your parrot, pet it gently; don't rub its feathers.\\\", \\\"The experience will be more traumatic if your bird doesn't trust you. [substeps] If your bird sings, talks, or whistles in your presence, this is a sign it is comfortable with you.\\\", '[substeps] At some point, your bird may seem uncomfortable with its claws. If this happens, you may want to try thinning out some of its claws and gently trim their length accordingly.', \\\"[substeps] If you're impatient, your bird may interpret your searching attempts as a sign that you are seeking to trim it's claws. At this point, you can decide if you want to trim the beak or whether you want to trim its claws.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}