Multi-vector encodings

Multi-vector embeddings represent a single data object, like a document or image, using a set of multiple vectors rather than a single vector. This approach allows for a more granular capture of semantic information, as each vector can represent different parts of the object. However, this leads to a significant increase in memory consumption, as multiple vectors are stored for each item.

Compression techniques become especially crucial for multi-vector systems to manage storage costs and improve query latency. Encodings transform the entire set of multi-vectors into a new, more compact single vector representation while aiming to preserve semantic relationships.

MUVERA, which stands for Multi-Vector Retrieval via Fixed Dimensional Encodings, tackles the higher memory usage and slower processing times of multi-vector embeddings by encoding them into single, fixed-dimensional vectors. This leads to reduced memory usage compared to traditional multi-vector approaches.

from weaviate.classes.config import Configureclient.collections.create(    "DemoCollection",    vector_config=[        # Example 1 - Use a model integration        Configure.MultiVectors.text2vec_jinaai(            name="jina_colbert",            source_properties=["text"],            encoding=Configure.VectorIndex.MultiVector.Encoding.muvera(                # Optional parameters for tuning MUVERA                # ksim: 4,                # dprojections: 16,                # repetitions: 20,            ),        ),        # Example 2 - User-provided multi-vector representations        Configure.MultiVectors.self_provided(            name="custom_multi_vector",            encoding=Configure.VectorIndex.MultiVector.Encoding.muvera(),        ),    ],    # Additional parameters not shown)

import { configure } from 'weaviate-client';

client.collections.create("DemoCollection", col -> col.vectorConfig(    // Example 1 - Use a model integration    VectorConfig.text2multivecJinaAi("jina_colbert",        vc -> vc.sourceProperties("text")            .vectorIndex(Hnsw.of(h -> h.multiVector(                MultiVector.of(mv -> mv.encoding(Encoding.muvera(e -> e                // Optional parameters for tuning MUVERA                // .ksim(4)                // .dprojections(16)                // .repetitions(20)                ))))))    ),    // Example 2 - User-provided multi-vector representations    VectorConfig.selfProvided("custom_multi_vector",        vc -> vc.vectorIndex(Hnsw.of(h -> h.multiVector(            MultiVector.of(mv -> mv.encoding(Encoding.muvera())))))))// Additional parameters not shown);

// Go support coming soon

await client.Collections.Create(
    new CollectionCreateParams
    {
        Name = "DemoCollection",
        VectorConfig = new VectorConfigList
        {
            // Example 1 - Use a model integration
            Configure.MultiVector(
                "jina_colbert",
                v => v.Text2MultiVecJinaAI(),
                index: new VectorIndex.HNSW
                {
                    MultiVector = new MultiVectorConfig { Encoding = new MuveraEncoding() },
                }
            ),
            // Example 2 - User-provided multi-vector representations
            Configure.MultiVector(
                "custom_multi_vector",
                v => v.SelfProvided(),
                index: new VectorIndex.HNSW
                {
                    MultiVector = new MultiVectorConfig { Encoding = new MuveraEncoding() },
                }
            ),
        },
    }
);

The final dimensionality of the MUVERA encoded vector will be repetitions * 2^ksim * dprojections. Carefully tuning these parameters is crucial to balance memory usage and retrieval accuracy.

These parameters can be used to fine-tune MUVERA:

• ksim (int, default: 4): The number of Gaussian vectors sampled for the SimHash partitioning function. This parameter determines the number of bits in the hash, and consequently, the number of buckets created in the space partitioning step. The total number of buckets will be $2^$. A higher value of ksim leads to a finer-grained partitioning of the embedding space, potentially improving the accuracy of the approximation but also increasing the dimensionality of the intermediate encoded vectors.

• dprojections (int, default: 16): The dimensionality of the sub-vectors after the random linear projection in the dimensionality reduction step. After partitioning the multi-vector embedding into buckets, each bucket's aggregated vector is projected down to dprojections dimensions using a random matrix. A smaller value of dprojections helps in reducing the overall dimensionality of the final fixed-dimensional encoding, leading to lower memory consumption but potentially at the cost of some information loss and retrieval accuracy.

• repetitions (int, default: 10): The number of times the space partitioning and dimensionality reduction steps are repeated. Each repetition captures a different perspective of the multi-vector embedding and can improve the robustness and accuracy of the final fixed-dimensional encoding. The resulting single vectors from each repetition are concatenated. A higher number of repetitions increases the dimensionality of the final encoding but can lead to better approximation of the original multi-vector similarity.

• How-to: Manage collections
• Concepts: Vector quantization

Have a question or feedback? Here's how to reach us.