{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2ffefd28-c213-544d-8477-83bbabcdfaa0","task_key":"default--validation--41276","task_revision_id":"1","upstream_id":"41276","short_description":"HellaSwag validation 41276","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to train your budgie [title] Make your budgie feel at home. [step] If your budgie is new, it will need time to get used to its cage. You should let it adjust for at least a few days and make sure that the bird is in a quiet environment before attempting to train it.\",\"endings\":\"['By this time, the budgie should start to relax and feel comfortable. [substeps] Make sure to read the wikihow tutorial on how to take care of a budgie stay close to the cage.', '[substeps] Wait at least two weeks before you begin training it, as more alert budgies may not accept verbal commands. If you wait longer than that, your budgie may suddenly become fearful of cage training, and may hide in an area of the home at night.', 'Doing this will help to encourage your budgie to do whatever the environment is doing to make it feel at home in your home. [substeps] Let your budgie rest or run around while you are training it.', '[substeps] Avoid allowing the bird to lounge on the cage floor unattended. It may instinctively place its food bowl on the windowsill, against the wall, or on a desk in the cage.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}