Chat-I-Nette got Vector Database — What is it?

42 WOB AI CLUB — Week X:Episode 1

If you have read our previous note then this is completely different as we worked on several things in between. For now we are going to talk about this sensation that is rocking the whole AI / LLMOPS Field.

Vector Database

It is a type of database that can store vectors which are mathematical representations of objects in space. Unlike traditional databases, vector databases are designed to handle data using vector embeddings and are user-friendly, making them easy to operate even for non-technical individuals. They are versatile and can be applied in various contexts, including e-commerce, healthcare, finance, and other fields.

Let’s go with an example :

To represent the sentence “I eat Apple and Orange” as a vector, we can use a one-hot encoding technique. In this technique, each word in the sentence is assigned a unique index, and a vector of zeros is created with a length equal to the number of words in the vocabulary. The index corresponding to each word in the sentence is set to 1 in the vector, and all other indices are set to 0. So we have [“I”, “eat”, “Apple”, “and”, “Orange”] -> [1,1,1,1,1].

After we have the vector representation of the sentence , one can store them in the vector database. It will allow us to perform similarity search and retrieval can be done based on the vector distance.

Several well-known vector databases are available in the market as I am writing naming Pinecone, Milvus, Chroma, Weaviate and others.

For now this is enough for today. We will continue again with the necessary code soon. If you find any mistakes feel free to reach out or have any feedbacks comment / email : mabbas@students.42wolfsburg.de / slack mabbas