{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b5d57d31-5e7e-5129-9bf8-de505e722944","task_key":"default--validation--9860","task_revision_id":"1","upstream_id":"9860","short_description":"HellaSwag validation 9860","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to train your dog with silent signals [title] Choose a command to teach. [step] Before you begin your training session, choose one command to focus on with your dog so that he will not be overwhelmed. You may want to start with something simple, such as \\\" sit.\",\"endings\":\"['\\\" a quiet click should help the dog put his ears back and relax. You can then give the command to sit by clapping your hands.', '\\\" instead of saying \\\" sit, \\\" teach your dog to sit if he is not already sitting. If he is, have him sit in a quiet area, such as a special spot in your home.', '\\\" or you may want to pick a specific command for your dog. Your dog will most likely respond to the command if you choose a simple one.', '\\\" [substeps] Commands such as sit, stay, heel, come, and lie down are considered basic obedience commands. These are important commands to teach your dog so that he will be well behaved.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}