Mind the Gap: Long-term Memory for Conversational AI
Introducing Llongterm
The generative AI revolution is transforming not just the technology industry, but every sector that interfaces with human language. Much like electricity in the early 20th century, AI is becoming a fundamental utility that will reshape how we live and work. And just as electricity benefited from centralised infrastructure and standardised distribution, AI applications need reliable, shared infrastructure to reach their full potential.
As conversational AI (generative AI chatbots and assistants) proliferate, a new ecosystem of applications has emerged, built on foundations like the APIs of OpenAI and Anthropic. These apps offer curated experiences for users to interact with generative AI - but they're missing something crucial: true long-term memory.
The Memory Problem
Human memory doesn't work like a database. We don't store information in neat, chronological orders or perfectly organised categories. Instead, our memories exist in an intricate web of associations, readily available when needed but not necessarily structured in any particular way. This organic, associative type of memory is what makes human conversation feel natural and contextual. Generative AI is perfectly suited to this structure, but we are only making use of a small part of this capability.
Most apps can only maintain short-term conversation history, and rely solely on the history for remembering what a user has said before. Due to the constraints in the number of input tokens, and the cost associated with a large number of tokens, there is a constant sense of starting over.
Information remains siloed between applications, preventing the kind of deep contextual understanding that makes human interactions so natural.
Every new interaction begins from scratch, and privacy concerns often restrict data sharing between applications. The result is an inconsistent user experience where each application maintains its own separate, limited understanding of who they are and what they need.
There's a profound irony here: We're building AI to interact with humans, yet we've given it amnesia. We're asking it to be human-like while denying it one of the most fundamental aspects of human cognition - the ability to form lasting memories and associations.
The Vector Database Challenge
Vector databases seem like an obvious solution for generative AI memory, but they present significant hurdles for individual developers. Setting up and maintaining these systems requires specialised knowledge that many developers simply don't have.
As data grows, managing performance and cost becomes increasingly complex, often overwhelming smaller teams or individual creators.
Each application needs its own database instance, authentication system, and maintenance pipeline – a significant overhead that diverts resources from core development.
The computational and financial demands can be prohibitive, especially for smaller developers or those just starting out. Perhaps most challenging is the expertise needed to fine-tune embeddings and similarity searches – technical skills that many otherwise capable developers haven't had the opportunity to develop.
The New Developer Landscape
In addition, the definition of "developer" has fundamentally changed in the post-AI world. We're witnessing the dissolution of traditional barriers to software creation, opening doors for anyone with ideas and domain knowledge to build meaningful solutions.
Business analysts are creating complex applications using AI-powered tools, content creators are building interactive experiences without traditional coding.
Domain experts are transforming their knowledge into functional software and students are prototyping and deploying ideas without years of technical training.
This new wave of creators needs reliable building blocks – standardised infrastructure components they can trust and build upon. Memory management is one of the most critical of these components, yet it's often overlooked or oversimplified.
Introducing Llongterm: A Unified Memory Layer for conversational AI
This is where Llongterm comes in. We're building a single "mind" for each user – a unified memory layer that can be shared across multiple AI applications.
The concept is simple yet powerful: developers request access to specific parts of a user's "mind," and users maintain granular control, granting read and/or write access to selected portions of their memory. Applications can both utilise and contribute to the user's accumulated knowledge.
This creates a remarkable synergy:
Users enjoy a more coherent experience across all Llongterm-enabled apps, with AI that truly remembers and understands their context.
Developers gain access to rich user context from day one, enabling faster development of more personalised applications.
The ecosystem itself grows stronger with each new connection, as apps are encouraged to contribute back to users' minds.
By providing this infrastructure as a service, we remove the technical barriers that would typically prevent many creators from building sophisticated AI applications. Just as cloud computing broke down the walls around server infrastructure, Llongterm aims to make AI memory management available to everyone.
Get Early Access
We're excited to announce that we'll be opening our API for early access in December 2024. For developers – whether you're a traditional software engineer, a no-code creator, or anywhere in between – we're maintaining a waitlist for those who want to be among the first to integrate this revolutionary memory layer into their applications.
Want to be part of the future of AI memory? Contact us to join the waitlist and learn more about how Llongterm can enhance your application's AI memory capabilities.