{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"65dbc6e4-7eb5-5391-8382-0dea03d146c6","task_key":"default--validation--27947","task_revision_id":"1","upstream_id":"27947","short_description":"HellaSwag validation 27947","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to care for long haired cats [title] Feed your cat two or three meals a day. [step] Don't leave out a constant supply of food. Your cat may be tempted to eat too much, which can lead to obesity and other health problems.\",\"endings\":\"[\\\"Take a closer look at your cat's food stash so you can determine when it is no longer hungry. [substeps] When deciding how much to provide your cat, look for pellets, soft treats and easy access to its bed, food bowls, litters, shelves and furniture.\\\", '[title] Feed your cat a mix of wet food and dry food. [step] This will ensure your cat will get all the proper nutrients.', 'A single serving of nutritious cat food will provide the cat with the energy and nutrition to stay young and healthy. [substeps] Feeding your cat a total of four meals per day should provide the rest and energy it needs.', 'Give your cat a walk at least once a day. Gently feed your cat food between meals so it can pace and get moving.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}