{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"46e8f5dc-de69-57e5-8635-8f9003f1134a","task_key":"default--validation--23232","task_revision_id":"1","upstream_id":"23232","short_description":"HellaSwag validation 23232","config":"default","split":"validation","body":"{\"activity_label\":\"Computers and Electronics\",\"ctx\":\"[header] How to prepare your own baby food [title] Research baby nutrition needs. [substeps] Learn about the nutritional needs of infants before rushing out to purchase a variety of food with which to prepare baby food. Infants require a different combination of vitamins and minerals than adults do.\",\"endings\":\"['Without information about the ideal combination of infant nutrition, consumers may end up purchasing food that will not provide their babies with the adequate level of significant vitamins and nutrients. Examine the many resources available for new parents with regard to healthy food items that provide the nutrients that infants need to grow healthy and strong.', \\\"To cook and store baby food, consult with an expert in the baby's nutrition needs before trying to prepare it yourself. Hold up a chart indicating your baby's age, sex, and weight for reference.\\\", 'Because infants are so old, they do not really understand how to process nutrition in different ways. Babies will do better with an active diet high in complex carbohydrates.', \\\"Therefore, if you are not up to date on all the research that you can about baby foods and will be preparing them your own way, then you should test the whole thing out beforehand. [title] Read book reviews of your newborn's nutritional needs before planning a trip.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}