{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"40c91cd6-c215-5881-b41b-8904ab8c5391","task_key":"default--validation--12840","task_revision_id":"1","upstream_id":"12840","short_description":"HellaSwag validation 12840","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to introduce formula to a baby [title] Choose the type of formula you want to use. [step] Talk to your doctor if you aren't sure which one to use. Not every type of formula is a good choice for every baby.\",\"endings\":\"[\\\"If you don't have specific recommendations, you can simply draft your own formula. For example, a pediatric formula is recommended.\\\", \\\"Some brands, such as sumo plus or almond, are actually more effective for babies who are breastfed. Children who breastfeed from their mothers' breasts are given several types of formula.\\\", \\\"Now that you have found the type you want your baby to be, find a daily formula that is right for your baby. The formula you choose depends on your baby's size.\\\", '[substeps] Milk-based, iron-fortified formula is what most pediatricians recommend until age 1. Hydrolyzed formula is easier to digest, but is more expensive.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}