{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e0238229-59d8-5ae7-a856-788834686c05","task_key":"default--validation--27559--variant-86df07a2cac1","task_revision_id":"1","upstream_id":"27559--variant-86df07a2cac1","short_description":"HellaSwag validation 27559","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to breed rats [title] Consider your breeding goals before you begin. [step] Think about your reasons for wanting to breed rats before you start a breeding program. If you plan to breed rats to sell on a large scale, then you will need to have an appropriate venue as well as lots of supplies, such as food, cages, and bedding.\",\"endings\":\"['You will need to have a place for rats to hunt and cross as well as rats to breed. You will also need to have at least 2 litters, even if you are breeding off more than one or two rats.', '[substeps] For small-scale breeding, this might mean having nesting and feeding options, providing your rats with accessible toilet facilities, and providing sure that the rats are sanitary. Another key element in breeding rats is providing space for them to roam.', '[substeps] Breeding rats on a large scale is an expensive, time consuming venture. Ensure that you have the space, resources, and time available to care for the rats that you breed.', \\\"[substeps] The rats' bottom levels, or the areas where you breed them, must be kept and monitored by a professional to ensure that their weight will be correct and their mating rituals will succeed. You can get rutabagas for between fifty and 50 percent of the profit.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}