Managing resources (Hot, Warm, Cold)

Weaviate provides flexible resource management features that help you to balance search speeds; search accuracy and recall; and system resource costs.

This guide provides an overview of these topics to help you make allocate resources effectively:

• Storage tiers (hot, warm, cold)
• Vector index types
• Vector compression
• Tenant states
• Tips

We categorize storage resources using three tiers: hot, warm, and cold. Each tier has different performance characteristics and costs.

• Describes memory usage
• Fastest and most expensive
• Primarily driven by HNSW vector indexes
• Always available (active) for use
• Costs increase rapidly with scale

• Describes data stored on disk (SSD)
• Slower than hot tier but less expensive
• Driven by flat vector index, object data, and inverted indexes
• Always available (active) for use
• Costs increase more slowly than hot tier as data grows

• Describes data stored in cloud storage
• Slowest and least expensive tier
• Primarily driven by offloaded tenants
• Resources are not available (inactive) for use
• Requires reactivation to access

Effective resource management in Weaviate involves balancing performance, cost, and data accessibility. The key levers to manage resources are:

• Vector index types: Choose the right index type based on the number of objects and desired performance.
• Vector compression: Use compression techniques to reduce memory usage and improve query performance at the cost of some accuracy.
• Tenant states: Manage tenant states to balance cost and performance.

The choice of vector index type can have a significant impact on performance and resource usage. Weaviate supports the following index types:

The choice of index type depends on the number of objects and the desired performance. As a rule of thumb, use the following guidelines for a multi-tenant collection:

flowchart LR
    %% Define nodes and connections
    subgraph vectorIndex ["Vector Index Selection"]
        direction LR
        multiTenant{"Is multi-tenancy<br> enabled?"}
        objCount{"Does the collection have<br> more than 100k objects?"}
        stayUnder{"Is the collection likely to stay<br> at under 100k objects?"}

        multiTenant -->|No| objCount
        multiTenant -->|Yes| useDynamic["Use Dynamic"]
        objCount -->|Yes| useHNSW["Use HNSW"]
        objCount -->|No| stayUnder
        stayUnder -->|Yes| useFlat["Use Flat"]
        stayUnder -->|"No/Unsure"| useDynamic["Use Dynamic"]
    end

    %% Style nodes
    style multiTenant fill:#ffffff,stroke:#B9C8DF,color:#130C49
    style objCount fill:#ffffff,stroke:#B9C8DF,color:#130C49
    style stayUnder fill:#ffffff,stroke:#B9C8DF,color:#130C49

    %% Terminal nodes
    style useHNSW fill:#ffffff,stroke:#61BD73,color:#130C49
    style useFlat fill:#ffffff,stroke:#61BD73,color:#130C49
    style useDynamic fill:#ffffff,stroke:#61BD73,color:#130C49

    %% Style subgraph
    style vectorIndex fill:#ffffff,stroke:#130C49,stroke-width:2px,color:#130C49

If you are unsure which index type to use, the dynamic index type is a good starting point, as it automatically transitions from a flat to an HNSW index based on the number of objects.

• Starter guide: indexes
• How-to: Set the vector index type

Vector compression techniques reduce the size of vectors by quantizing them into a smaller representation.

This can have the impact of reducing memory usage, or improving performance by reducing the amount of data that needs to be read from disk. The trade-off is that the resulting search quality may be lower.

Weaviate supports the following vector compression methods:

As a starting point, use the following guidelines for selecting a compression method:

flowchart LR
    %% Define nodes and connections
    subgraph compression ["Compression Strategy"]
        direction LR
        startIndexType{"What is your<br> vector index type?"}
        sample{"Do you have a<br> representative sample<br> of your final dataset?"}
        tunable{"Do you want<br> tunable<br> compression?"}
        bqcompat{"Is your vectorizer<br> model compatible<br> with BQ?"}

        startIndexType -->|HNSW| sample
        startIndexType -->|Flat| bqcompat
        sample -->|Yes| tunable
        tunable -->|Yes| usePQ["Use PQ"]
        tunable -->|No| useSQ["Use SQ"]
        sample -->|No| useBQ["Use BQ"]
        bqcompat -->|Yes| useBQ["Use BQ"]
        bqcompat -->|No| noCompress["Do not use<br> compression"]
    end

    %% Style nodes
    style startIndexType fill:#ffffff,stroke:#B9C8DF,color:#130C49
    style sample fill:#ffffff,stroke:#B9C8DF,color:#130C49
    style tunable fill:#ffffff,stroke:#B9C8DF,color:#130C49
    style bqcompat fill:#ffffff,stroke:#B9C8DF,color:#130C49

    %% Terminal nodes
    style usePQ fill:#ffffff,stroke:#61BD73,color:#130C49
    style useSQ fill:#ffffff,stroke:#61BD73,color:#130C49
    style useBQ fill:#ffffff,stroke:#61BD73,color:#130C49
    style noCompress fill:#ffffff,stroke:#61BD73,color:#130C49

    %% Style subgraph
    style compression fill:#ffffff,stroke:#7AD6EB,stroke-width:2px,color:#130C49

If you are unsure which index type to use, scalar quantization is a good starting point, provided that you have a representative sample of your likely final dataset.

• Starter guide: Vector compression
• How-to: Configure vector compression

Multi-tenant collections enable you to efficiently manage isolated subsets of data. Each tenant share the same schema and configuration.

Weaviate supports the following tenant states:

Hot tenants can be deactivated to warm storage to reduce memory usage, and any tenant can be offloaded to cold storage to reduce memory and disk usage. Conversely, any tenant can be reactivated when needed.

flowchart LR
    %% Define nodes and connections
    subgraph tenantData ["Tenant Data Availability"]
        direction LR
        needNow{"Does the tenant data<br> need to be<br> available now?"}
        howQuick{"When it is needed,<br> how quickly does it<br> need to be available?"}

        needNow -->|Yes| active["Active"]
        needNow -->|No| howQuick
        howQuick -->|Quickly| inactive["Inactive"]
        howQuick -->|"Latency is acceptable"| offloaded["Offloaded"]
    end

    %% Style nodes
    style needNow fill:#ffffff,stroke:#B9C8DF,color:#130C49
    style howQuick fill:#ffffff,stroke:#B9C8DF,color:#130C49

    %% Terminal nodes
    style active fill:#ffffff,stroke:#61BD73,color:#130C49
    style inactive fill:#ffffff,stroke:#61BD73,color:#130C49
    style offloaded fill:#ffffff,stroke:#61BD73,color:#130C49

    %% Style subgraph
    style tenantData fill:#ffffff,stroke:#61BD73,stroke-width:2px,color:#130C49

Consider a strategy of deactivating tenants that are not frequently accessed, and offloading tenants that are rarely accessed.

• Starter guide: tenant states
• How-to: Configure tenant offloading
• How-to: Manage tenant states

• Start with the dynamic index type for new collections. This is particularly useful for multi-tenant collections, as it allows each tenant to use the most appropriate index type.
• Use vector compression techniques to optimize storage and query performance, especially for large collections or tenants.
• Conduct thorough testing when changing index types or compression methods to ensure performance meets your requirements.

• Overprovisioning hot storage: Keeping all data in hot storage can lead to unnecessary costs. Regularly assess what data truly needs the fastest access.
• Neglecting to plan for growth: Not anticipating data growth can lead to performance issues. Always design your resource management strategy with scalability in mind.
• Improper tenant management: In multi-tenant scenarios, forgetting to offload inactive tenants can lead to resource waste. Implement automated processes to manage tenant states based on usage patterns.
• Mismatch between quantization techniques, model and data: When using compression technique, ensure that the quantization technique is compatible with the model (e.g. BQ) and that the data is sufficient and representative for training (e.g. PQ, SQ).

• Starter guide: Compression
• Starter guide: Indexing
• Starter guide: Tenant states
• Concepts: Vector Index
• Concepts: Vector Quantization
• Concepts: Multi-Tenancy
• How-to: Set the vector index type
• How-to: Configure vector compression
• How-to: Perform multi-tenancy operations
• How-to: Manage tenant states

Have a question or feedback? Here's how to reach us.