{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c6a5f7e8-089d-53f9-82d9-5db0c39fc01d","task_key":"default--validation--46844","task_revision_id":"1","upstream_id":"46844","short_description":"HellaSwag validation 46844","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to teach your cat to do tricks [title] Get a supply of treats. [step] Cats need to be constantly rewarded with tasty treats in order to learn tricks. Keep plenty of your cat's favorite bite-sized treats on hand when trying to train it.\",\"endings\":\"[\\\"Some good treats may include : [substeps] Wild-caught hamster treats table scraps your cat's favorite toy kibble dog food [title] Consider training sessions. [step] Most cats have excellent mental and physical abilities that they can use as they learn to perform tricks.\\\", 'Give your cat frequent treats when training it in short sessions. You can also vary treats frequently to keep your cat interested.', 'These should be small enough that your cat can go on the go without biting. [substeps] Also make sure there is plenty of fresh water nearby and a bowl of treats nearby as well.', 'You can keep these treats as treats in your cat\\\\'s \\\" tank. \\\" [substeps] Store a few treats in your cat\\\\'s \\\" tank \\\" bowl or in a plastic egg carton that your cat can open and nibble on with its mouth.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}