This is where Retrieval-Augmented Generation (also known as RAG) comes in.
Retrieval-Augmented Generation (RAG)
RAG improves the accuracy and relevance of LLM outputs by augmenting (adding onto) the input prompt with relevant pieces of data stored in an external database or search index. This can involve for example embedding the prompt and documents into a shared vector space, then using similarity metrics to rank the pieces of data by relevance.
When a new user prompt is received (in our case, when the user talks to the shopping assistant), we embed it in the same format as our online store’s inventory (a series of numerical values) which allows us to retrieve the most likely relevant items, which we then include as part of the original prompt to the LLM.
In this scenario, Cymbal Shops was already using a PostgreSQL database to store product information, so we will leverage AlloyDB, Google Cloud’s 100% PostgreSQL-compatible database, to also store vectorized products. Leveraging Vertex AI’s built-in RAG capabilities may be preferable in other scenarios.
