A startup called Letta has just emerged from stealth with tech that helps AI models remember users and conversations. Created in UC Berkeley’s famed labs startup factory, it also announced $10 million in seed money led by Felicis’ Astasia Myers, at a $70 million post-money valuation.
Letta is also backed by a who’s who of angel investors in AI, like Google’s Jeff Dean, Hugging Face’s Clem Delangue, Runway’s Cristóbal Valenzuela, and Anyscale’s Robert Nishihara, among others.
Founded by Berkeley PhD students Sarah Wooders and Charles Packer, this is a highly anticipated AI startup launch. That’s because it’s a child of Berkeley’s Sky Computing Lab and is the commercial entity of the popular MemGPT open source project.
Berkeley’s Sky Computing Lab, led by acclaimed professor and Databricks co-founder Ion Stoica, is the descendent of RISELab and AMPLab, which spawned such companies as Anyscale, Databricks, and SiFive. Sky Lab, in particular, birthed numerous popular open source large language model (LLM) projects like the Gorilla LLM, vLLM, and the LLM structured language SGLang.
“A ton of projects very quickly, within a year’s time frame, came out of the lab. Just people sitting next to us,” described Wooders. “So it was kind of an incredible time.”
MemGPT is one such project and is such a hot commodity that it actually went viral before it even launched.
“Someone scooped us,” Packer told TechCrunch. The founders had posted a whitepaper on Thursday, October 12, 2023, and planned to release a more in-depth paper and the code to GitHub the following Monday. Some random person found the paper, posted it to Hacker News on Sunday, and it “went viral on Hacker News before we had a chance to properly release the code, release the paper, or, like, do a tweet thread or anything like that,” he said.
The reason for the excitement was that MemGPT mitigates a pernicious problem for LLMs: In their native form, models like ChatGPT are stateless, meaning they don’t store historical data in long-term memory. This is problematic for AI apps that depend on getting to know and learn from a user over time — everything from customer support bots to healthcare symptom-tracking apps. MemGPT manages data and memory so that AI agents and chatbots can remember previous users and conversations.
The post on the paper stayed atop Hacker News, the popular site for programmers run by Y Combinator, for 48 hours, Packer recounted. So he spent his weekend and the next few days answering questions on the site while trying to get the code ready to be released. Once the project was available on GitHub, a link to it went viral on Hacker News, again. YouTube interviews and tutorials, Medium posts, 11,000 stars and 1.2K forks on GitHub happened quickly.
VC Felicis’ Myers discovered Wooders and Packer by reading about MemGPT, too, and immediately recognized the tech’s commercial possibilities.
“I saw the paper when it was released,” she told TechCrunch, and she promptly reached out to the founders. “We had an investment theme around AI agent infrastructure and appreciated that a really important component of that was the data and memory management to make these conversational chat bots and AI agents effective.”
The founders still virtually traipsed around Sand Hill Road doing Zoom calls with VCs before going with the one that loved them first.
Meanwhile, Stoica brokered introductions to Dean, Nishihara and other big-name Silicon Valley angels. “A lot of the professors at Berkeley, just as a consequence of being at Berkeley, are very well connected,” Packer recalled about how easy the angel investor process was. “They have their eye on projects out of this lab that are going to be commercialized.”
Competition and the threat of OpenAI o1
While MemGPT is already out in the wild and being used, Letta’s commercial variant, Letta Cloud, is not yet open for business. As of Monday, Letta is accepting requests for beta users. It will offer a hosted agent service that allows developers to deploy and run stateful agents in the cloud, accessible via REST APIs, a programming interface that can maintain state. Letta Cloud will store the long-term data necessary to do so. Letta will also offer developer tools for building AI agents.
With MemGPT, Wooders sees a large span of uses. “I think the number one use case that we see is basically, highly personalized, very engaging chatbots,” she says. But there are also cutting-edge uses like “a chatbot for cancer patients” where patients upload their history and then share ongoing symptoms so the bot can learn and offer guidance over time.
Worth noting that MemGPT isn’t alone in working on this. LangChain is probably its best known competitor, and it already offers commercial options. The biggest model makers also offer AI agent-making tools as well, like OpenAI’s Assistants API.
And OpenAI’s new o1 model may make the need to fix state a moot point for its users. As it is a multistep model, it fundamentally must maintain state to some degree in order to “think” and fact check before it replies.
But Wooders, Packer, and Myers see a few key differences to what Letta is offering versus what 800-pound market gorilla OpenAI is doing. Letta claims it will work with any AI model and expects its users to use many of them: OpenAI, Anthropic, Mistral, their own homegrown models. OpenAI’s tech currently only works with itself.
More importantly, Letta is using open source MemGPT and jumping firmly into the open source side of the FOSS vs. black box LLM debate, saying open source is a better choice for AI application programmers.
“We are positioning ourselves as the open alternative to OpenAI,” Packer says. “I think it’s actually very, very hard to build very good AI applications, especially when you care about, like hallucination, if you can’t see what’s going on under the hood.”