LLM assistant with long-term memory
A language-model assistant with persistent memory and hybrid retrieval — semantic and keyword — over a graph-based store.
Personal learning and demonstration project. It is not a deployment for a real client.
Memory
- Conversation
- Information extraction
- Neo4j graph
Query
- Question
- Hybrid retrieval (embeddings + TF-IDF)
- Context
- LLM answer
Conceptual diagram based on the project description. It will be replaced by a diagram of my own.
Overview
A personal experimentation project: a language-model assistant that keeps long-term memory and retrieves it with a combination of semantic and keyword search. It is not a product or a client deployment.
Problem and goal
A language model on its own does not remember earlier conversations, and pasting the full history into every request is expensive and unreliable. The goal was to explore a persistent memory that stores relevant information and retrieves it when needed, with a design that can be reasoned about and evaluated.
Technical architecture
- Graph memory store (Neo4j): information is stored as entities and relationships, which allows traversing connections as well as searching by similarity.
- Hybrid retrieval: semantic search with embeddings, which captures meaning, is combined with keyword search using TF-IDF, which gets exact terms and proper names right.
- Generation: the model receives the question and the retrieved context.
I’m happy to walk through the implementation details (engines, models and specific parameters) in a conversation.
Technologies
Python, Neo4j, embeddings, TF-IDF and a language model (API).
Design decisions
- Why hybrid. Semantic search finds similar ideas even when vocabulary changes, but can miss identifiers, names or exact figures; TF-IDF covers that gap.
- Why a graph. Relationships between people, topics and facts are part of what is remembered, and a graph represents them naturally.
- What to remember. Not everything said deserves storing. Deciding what is stored matters as much as how it is retrieved.
Results and evaluation
Evaluation metrics are not published yet: no unvalidated figures are shown.
Relation to real projects
The same principles — retrieval over your own documents, source citations, permissions and evaluation — underpin the AI assistants on private data service. This project demonstrates technical capability; it does not replace an evaluation on each company’s data.