{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"efca538d-9e1f-5ec4-91fd-741b2a2a496d","task_key":"default--validation--26316","task_revision_id":"1","upstream_id":"26316","short_description":"HellaSwag validation 26316","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to store your expressed breast milk [title] Use glass or plastic jars. [step] Either glass or hard plastic jars are fine to use, as long as they have lids that can be tightly closed. Consider sanitizing these items before use by following this method : [substeps] Bring a large pot of water to boil on the stove.\",\"endings\":\"['Add two pieces of ice to the pot; this will warm the contents of the pot and increase the temperature. Place the jar and container in the boiling water and keep them in the refrigerator.', 'Turn on a burner to medium heat, raise the heat, and let the water come to a gentle boil. Place the contents of an empty corn colander over the boil and reduce the heat.', 'Add 3 to 5 drops of peppermint oil , 1 pinch of cinnamon, two tablespoons ground nutmeg and one pinch of rosemary. Place the lid on and let this boil for approximately 10 minutes.', 'Use a pair of tongs to lower the jars and their lids into the boiling water. Let the jars and lids boil for about a minute.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}