Companies are increasingly embracing generative AI. But they’re not necessarily considering all the blockers and challenges that come along with folding the tech into their existing workflows.
In one recent survey by KPMG, 90% of businesses said that they had “moderate to highly significant” concerns about the risks of using generative AI and doubts about how to mitigate those risks. Only 6%, meanwhile, said that they felt that their company had a mature AI governance program in place.
It’s partly for these reasons that Neil Sereybrany founded CalypsoAI, a startup developing software to test, validate and monitor internally-developed as well as third-party AI apps prior to their deployment.
“CalypsoAI’s platform promotes the power and competitive advantage that using AI can bring but within a safe, secure and reliable way,” Sereybrany told TechCrunch in an email interview.
That’s setting the bar high. But investors evidently believe in CalypsoAI’s ability to deliver. The startup today announced that it raised $23 million in a Series A-1 round led by Paladin Capital Group with participation from Lockheed Martin Ventures, Hakluyt Capital and Expeditions Fund.
Notable angels contributed to the A-1 tranche as well, including former YouTube CEO Susan Wojcicki and her sister, 23andMe cofounder Anne Wojcicki. To date, CalypsoAI has raised $38.2 million.
CalypsoAI claims that its tools — deployed as a container within an organization’s infrastructure — allow businesses to monitor and shape the usage of large language models such as ChatGPT, via dashboards that show stats related to the toxicity of the models, user engagement and more. Sereybrany says that CalypsoAI can prevent sensitive company data from being shared on models while identifying attacks coming from generative AI tools.
“While every company wants to reap the benefits of generative AI solutions — namely the clear productivity gains — they also want to make sure they aren’t subject to cyberattacks and that employees don’t expose sensitive information to public models,” Sereybrany added. “By implementing CalypsoAI, CISOs and IT leaders can start to seriously consider implementing generative AI solutions across their organizations in a safe and secure manner – allowing them to introduce efficiencies into their business without the added risk.”
Certainly, there’s demand in the corporate sector for some sort of generative AI guardrail solution. Out of fear that employees might accidentally expose proprietary info, companies including Apple, JPMorgan Chase and Verizon have banned or restricted the use of ChatGPT-like tools internally. Many of these same companies are wary of generative AI’s toxicity and misinformatio issues, which continue to plague even the best models available today.
The industry, eyeing the huge potential addressable market, has risen to meet the demand. Microsoft offers the Azure OpenAI Service, which provides governance and compliance capabilities on top of generative AI models from OpenAI, including ChatGPT. Elsewhere, Salesforce’s recently launched the Einstein Trust Layer, which attempts to prevent text-generating models from retaining sensitive data.
How well these guardrails work in practice is an open question, however. Putting one vendor’s claims into question, researchers recently uncovered flaws in NeMo Guardrails, Nvidia’s toolkit aimed at making AI-powered apps more accurate and secure, that allows the toolkit’s protections to be bypassed.
Sereybrany insisted — for what it’s worth — that CalypsoAI’s products are robust to attack. In the same breath, he stressed that CalypsoAI doesn’t hold any data from its customers, instead only facilitating the info flow from employees into the model of a company’s choice.
“Securing LLMs in the enterprise has become a critical venture, which has led to CalypsoAI’s strategic shift to address the needs of enterprises looking to use AI solutions,” Sereybrany said. “With many organizations looking to securely use generative AI solutions to propel their business forward, CalypsoAI feels confident about our ability to develop the best possible product and attract the best talent to secure enterprise organizations.”
Sereybrany says that CalypsoAI will use the funding from the latest tranche to drive product development, hiring and the company’s go-to-market initiatives. CalypsoAI’s current headcount stands at 35 employees, and the startup recently opened an office in Dublin — a part of CalypsoAI’s plan to double its workforce over the next two years.
“As our company continues to grow and expand, we recognize the value of bringing on new investors to support our growth,” he continued. “By partnering with investors who share our vision and goals, we can continue to innovate and drive our business forward.”
CalypsoAI raises $23M to add guardrails to generative AI models by Kyle Wiggers originally published on TechCrunch
