{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e626e3e7-b15d-5353-a085-cc4ecb8adaaf","task_key":"default--validation--17852","task_revision_id":"1","upstream_id":"17852","short_description":"HellaSwag validation 17852","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to use a bottle sterilizer [title] Add the recommended amount of water to the sterilizer. [step] Steam sterilizers require the addition of a small amount of water to create steam. Consult your user manual to determine how much water to use.\",\"endings\":\"[\\\"Too little water may not sterilize the bottles properly, but too much could cause the basin to overflow. [substeps] Check the sterilizer's instructions to see if there is a specific type of water you need to use.\\\", '[substeps] Use plenty for the amount of water you need. For example, to sterilize 19 ounces of beer, add 750 ml of water to three pounds of beer, so that 24 gallons is about the maximum that you need to sterilize a 25 gallon pint.', 'For example, a relatively young child weighing around 36 lbs should not add more than one gallon of water to a sterilizer. [substeps] If your child has a serious health problem, you may need more water than the recommended amount.', '[substeps] You can also soak a small cloth in cold water for up to twenty minutes, then stir the cloth with a toothbrush. Do not touch the sterilizer with your hand while it is in the sterilizer.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}