{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3a8c2f7a-da14-5a53-99d8-b6b6a126bd09","task_key":"default--validation--12150","task_revision_id":"1","upstream_id":"12150","short_description":"HellaSwag validation 12150","config":"default","split":"validation","body":"{\"activity_label\":\"Work World\",\"ctx\":\"[header] How to bottle feed calves [title] Purchase or gather the appropriate feeding supplies. [step] You'll need a two quart calf bottle with nipple and raw cow's milk or a bovine milk replacement formula, and possibly colostrum. Choose a milk replacement formula that is at least 25% protein.\",\"endings\":\"[\\\"[substeps] Standard cow's milk nipple (gold) is a popular choice for babies older than 4 months. It is 2.5% protein and is also excellent at feeding bulls the correct amount.\\\", \\\"You can get nipple or whey milk backup at your local grocer. [substeps] You'll need 4 pounds (19 kg) of calf's milk for every 1 inch (2.5 cm) of virgin cow's milk that you use.\\\", 'Make sure the formula is fresh and nonfrozen, or you could mess with the calcium content. Plastic or wool milk replacement bottles help support the weight of your calf after feeding.', 'Other supplies can include a syringe or turkey baster, which you might find helpful in the beginning when getting the calf accustomed to the bottle. [substeps] A calf bottle that holds two quarts of liquid is all you should need.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}