{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"270ec718-43f8-5c81-a478-65779e7e1940","task_key":"default--validation--25527","task_revision_id":"1","upstream_id":"25527","short_description":"HellaSwag validation 25527","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to care for a samoyed [title] Get the proper equipment and set aside time. [step] Your samoyed has two layers of hair: a fine undercoat keeps it warm and an outer layer of coarse hair to protects it from the weather. To maintain all of this fur, you need to comb and brush your dog several times a week with the right kinds of tools.\",\"endings\":\"['[substeps] You will need a rake or comb with rotating teeth (to glide through the hair easily) and a pin brush with rounded ends (to prevent you from scratching your dog while grooming). [title] Groom with a comb or rake first.', 'You can also put your dog in the bath once every ten minutes. [substeps] Make sure your dog has proper: a full head of undercoat protecting the skin from sunburn and scratching.', \\\"Should your samoyed run out of those supplies, have an experienced groomer do it for you. You don't want it to rat you when you get home.\\\", '[substeps] Check with your vet to see what materials he recommends for undercoat. How to prevent heat stroke or body aches is critical.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}