{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8e0d68bc-aead-55ca-a2e3-f52b1911062c","task_key":"default--validation--23586--variant-3788cdf0f426","task_revision_id":"1","upstream_id":"23586--variant-3788cdf0f426","short_description":"HellaSwag validation 23586","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to remove an iodine stain [title] Flush the fabric with water. [step] As soon as you notice the iodine stain on your clothes, immediately bring the clothing item to the sink and flush it with cold water. Let the water stream onto the stain until the stained area is completely saturated.\",\"endings\":\"['Do not squeeze any of the liquid out. [substeps] If the stain is superficial, however, the scrape should be discontinued.', \\\"Again, this method will require a bit more effort than a regular stain remover and may take a little longer. However, it's a relatively painless process, and it can potentially be avoided if the stain is completely removed.\\\", \\\"[substeps] These substances give off too much ammonia, which can be deadly to people. The horizontal flush method is most optimal because it uses water directly from the faucet from the tap or from running water for maximum saturation, because you don't have to touch the stain with your finger as you flush.\\\", '[substeps] Flushing the garment with water helps prevent the stain from setting. [title] Create a solution of water, ammonia and liquid laundry detergent.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}