Most major companies, including Google, Amazon, Microsoft, Uber, and Tesla, have had their artificial intelligence (AI) and machine learning (ML) systems tricked, evaded, or unintentionally misled. Yet despite these high profile failures, most organizations’ leaders are largely unaware of their own risk when creating and using AI and ML technologies. This is not entirely the fault of the businesses. Technical tools to limit and remediate damage have not been built as quickly as ML technology itself, existing cyber insurance generally doesn’t fully cover ML systems, and legal remedies (e.g., copyright, liability, and anti-hacking laws) may not cover such situations. An emerging solution is AI/ML-specific insurance. But who will need it and exactly what it will cover are still open questions.