Vector Processing: Understand This New Revolution in Search

“Vectorize EVERYTHING!”

These were the words of wisdom from Nyah Macklin, Neo4j senior developer advocate, in a presentation on vector databases at All Things Open 2024.

Vectorized data is going to be very important going forward, especially in the realms of machine learning and artificial intelligence.

Vector processing, once a niche concept, is now at the forefront of powering innovative applications like semantic search, recommendation systems and advanced chatbots.

“Vector search is a new way of information retrieval, which is up and coming,” agreed Vadim Tkachenko, Percona co-founder and technology fellow, in his ATO 2024 session on vector searches in databases.

Vector: Beyond Keywords

At the heart of vector processing lies the concept of semantic search. Unlike traditional lexical searches that rely on matching keywords, semantic search delves into the meaning and context of words, aiming to understand the user’s intent.

It “allows us to understand the meanings behind words, not just the words themselves,” Macklin said.

It’s about deciphering the nuances of language, recognizing that the same word can have different meanings depending on the surrounding context. For instance, “Apple” in “Apple laptop” is distinct from “apple” in “apple pie.”

Semantic search leverages embeddings, numerical representations of words, phrases, or even entire documents to capture their semantic meaning. By converting textual (and other) information into numerical vectors, semantic search enables computers to understand and compare the meanings of different pieces of content.

Semantic search is all about finding and scoring related data, using context and intent. – Nyah Macklin

Semantic search also gains more context from what is known about the user. Search history or the location of a user can provide more clues about their intent — “Football” means something completely different in the U.S. and Britain, for example.

Semantic search results are scored against the intent of the search so they can be compared. Similarity scores are usually ranked between zero and one, with one representing the greatest similarity.

This numerical representation of a word or phrase is called an embedding. A ‘river bank’ has a different embedding than a ‘savings bank.’

Vector processing extends beyond text and semantic search. It can be applied to various data types, including images, audio and video. For instance, in image recognition, images can be converted into vectors, allowing for similarity searches to find images with similar content or features.

Vectors: The Building Blocks of Semantic Search

A vector is essentially a list of numbers representing a magnitude and a direction. The number of elements in this list defines its dimensionality. In machine learning, vectors with hundreds or even thousands of dimensions are commonly used to represent complex concepts and relationships.

The breakthrough in using vectors for semantic search came in a 2013 paper, “Efficient Estimation of Word Representations in Vector Space.” The paper was authored by Google boffins Tomas Mikolov, Kai Chen, Greg Corrado and Jeffrey Dean, and provided a method to turn sentences into vectors.

The researchers found that vectors could be used to find similarities in very large data sets much more efficiently than the heretofore-used method of training a neural network to do the work.

“We observe large improvements in accuracy at much lower computational cost, i.e. it takes less than a day to learn high quality word vectors from a 1.6 billion words data set,” the researchers wrote. “Furthermore, we show that these vectors provide state-of-the-art performance on our test set for measuring syntactic and semantic word similarities.”

This paper introduced a new model, called Word2Vec, to efficiently convert words and phrases into dense vectors, in a way that captured their semantic relationships. Words with similar meanings have vectors that are closer together in the vector space, while dissimilar words have vectors that are farther apart.

 

In the years since the paper was posted, the industry has built an entire industry, generative AI, around the technology.

“Vector search is just an easy way to find related objects that have similar characteristics,” explained Olena Kutsenko, senior developer advocate for data platform provider Aiven, in her ATO 2024 presentation.

“And in vector search, we rely on a machine loading model to help us to detect those characteristics of different things, whether they’re text, images, audio or something else.”

In the multidimensional space created by the model, images that are similar tend to have have close vectors, while those that are not have very different vectors. Each entity has a set of coordinates, and those coordinates can be compared across different entities.

Vector Match: Searching in a Multidimensional Space

To effectively utilize embeddings for semantic search, specialized databases known as vector databases have emerged, such those offered by Pinecone and the open source Milvus.

Vector support is also rapidly being added to traditional databases as well. PostgreSQL users, for instance can install PGVector for full vector support, including vector search. A 512-vector dimension simply gets stored as a 512-number array.

Kutsenko offered an example of how to use vectors to find movie recommendations.

With both types of databases, a user query (“A movie with cute puppies”) is turned into a vector, so it can then be compared to its “nearest neighbors” of other movies in the database. The vector database then performs a similarity search to find the movies whose vectors are closest to the query vector, effectively recommending movies that match the user’s preferences.

Vector Databases: So Many Vectors

The databases employ algorithms like k-nearest neighbors (KNN) and approximate nearest neighbor (ANN) to quickly identify vectors that are closest to a given query vector.

KNN is a straightforward algorithm that compares the query vector to every other vector in the database, identifying the k-nearest neighbors based on a distance metric. While effective for smaller data sets, KNN becomes computationally expensive for large data sets.

ANN addresses this challenge by using indexing techniques to pre-process the data and speed up searches. These techniques, such as IVFFlat and HNSW, create clusters or graphs of related vectors, allowing the search algorithm to focus on specific regions of the vector space, thereby reducing the number of comparisons needed.

The first option, IVFFlat (inverted file with flat compression), builds clusters around groups of data so the user can specify the chunk of data closets to the query to inspect. This is an index to build only after you’ve entered much of your data.

The second option, HNSW (Hierarchical Navigable Small Worlds), creates a graph with multiple layers, with least-likely material on the outer layers. This is more expensive to build, but it operates faster, Kutsenko noted. This is also the index to use if you are changing out a lot of your data.

Vector Processing: A Search for Tomorrow

While vector processing offers significant advantages, there are challenges to consider. The accuracy of the results depends heavily on the quality of the embeddings and the chosen similarity metric. Additionally, balancing performance and precision is crucial, especially with large data sets. Choosing the right indexing technique and optimizing the search parameters can significantly impact the efficiency and accuracy of vector searches.

But as the presenters at ATO 2024 have shown, vector processing is revolutionizing how we interact with information, enabling machines to understand and process data in a more humanlike manner.

Kutsenko’s full presentation an be enjoyed here:

The post Vector Processing: Understand This New Revolution in Search appeared first on The New Stack.

Three presenters from ATO 2024 show how vector processing is changing how our systems interact with information.