{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"842edd6d-14b3-5ad7-87a9-0b07ed8d5539","task_key":"default--validation--34376","task_revision_id":"1","upstream_id":"34376","short_description":"HellaSwag validation 34376","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to treat botulism in a duck [title] Watch for the symptoms. [step] Botulism poisons the duck, causing what's sometimes known as \\\" limberneck disease. \\\" it causes paralysis, starting with the duck having trouble taking off or going under the water.\",\"endings\":\"[\\\"The duck's legs will be paralyzed, and you may see the duck trying to move only with its wings. The duck's eyelids and neck will droop.\\\", \\\"Chlamydia doesn't have any symptoms, but is caused by consuming ylang-ylang and urinating a lot. It may contribute to seizures, nausea, urination, body odor, and more.\\\", 'The longer a duck is in the water the sicker it gets. These symptoms can include : [substeps] Undigested food in the guts.', 'The duck will not return to normal the following morning. The symptoms first occur when the duck takes off from the water.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}