# Scalar Quantization (SQ) :::callout{intent="info" title="Compression by Default"} Starting with `v1.33`, you can set a default quantization for new collections using the [`DEFAULT_QUANTIZATION`](../database-configuration/overview.md#DEFAULT_QUANTIZATION) environment variable. This variable is not set by default, meaning no quantization is applied unless you explicitly configure it. When set (e.g., to 8-bit [RQ quantization](compression-rq-compression.md)), all newly created collections will use that quantization setting. Note that once set on a collection, quantization can't be disabled. Default quantization won't be applied to a collection if the index type isn't supported (for example PQ and SQ aren't supported for the flat index). ::: [**Scalar quantization (SQ)**](../concepts/vector-quantization.md#scalar-quantization) is a vector compression technique that can reduce the size of a vector. To use SQ, enable it in the collection definition, then add data to the collection. ## Enable compression for new collection SQ can be enabled at collection creation time through the collection definition: :::code-group{sync="languages"} ```python title="Python" {7} from weaviate.classes.config import Configure client.collections.create( name="MyCollection", vector_config=Configure.Vectors.text2vec_openai( name="default", quantizer=Configure.VectorIndex.Quantizer.sq(), ), ) ``` ```typescript title="JavaScript/TypeScript" const collection = await client.collections.create({ name: 'MyCollection', vectorizers: weaviate.configure.vectors.selfProvided({ vectorIndexConfig: weaviate.configure.vectorIndex.hnsw({ quantizer: weaviate.configure.vectorIndex.quantizer.sq(), }) }) }) ``` ```go title="Go" {2-4,10-13} // Define the configuration for SQ. Setting 'enabled' to true sq_config := map[string]interface{}{ "enabled": true, } // Define the class schema class := &models.Class{ Class: className, Vectorizer: "text2vec-openai", // Assign the SQ configuration to the vector index config VectorIndexConfig: map[string]interface{}{ "sq": sq_config, }, } // Create the collection in Weaviate err = client.Schema().ClassCreator(). WithClass(class). Do(context.Background()) ``` ```java title="Java" {3} client.collections.create("MyCollection", col -> col.vectorConfig(VectorConfig.text2vecTransformers(vc -> vc .quantization(Quantization.sq()) )).properties(Property.text("title"))); ``` ```csharp title="C#" {11} await client.Collections.Create( new CollectionCreateParams { Name = "MyCollection", Properties = [Property.Text("title")], VectorConfig = Configure.Vector( "default", v => v.Text2VecTransformers(), index: new VectorIndex.HNSW { Quantizer = new VectorIndex.Quantizers.SQ(), } ), } ); ``` ::: ## Enable compression for existing collection :::callout{intent="info" title="Added in `v1.31`"} The ability to enable SQ compression after collection creation was added in Weaviate `v1.31`. ::: SQ can also be enabled for an existing collection by updating the collection definition: :::code-group{sync="languages"} ```python title="Python" from weaviate.classes.config import Reconfigure collection = client.collections.use("MyCollection") collection.config.update( vector_config=Reconfigure.Vectors.update( name="default", vector_index_config=Reconfigure.VectorIndex.hnsw( quantizer=Reconfigure.VectorIndex.Quantizer.sq( rescore_limit=20 ), ) ) ) ``` ```typescript title="JavaScript/TypeScript" const collection = client.collections.use('MyCollection'); await collection.config.update({ vectorizers: [ weaviate.reconfigure.vectors.update({ name: 'default', vectorIndexConfig: weaviate.reconfigure.vectorIndex.hnsw({ quantizer: weaviate.reconfigure.vectorIndex.quantizer.sq({ rescoreLimit: 20, }), }), }), ], }) ``` ```go title="Go" // Get the existing collection configuration class, err := client.Schema().ClassGetter(). WithClassName(className).Do(context.Background()) if err != nil { log.Fatalf("get class for vec idx cfg update: %v", err) } // Get the current vector index configuration cfg := class.VectorIndexConfig.(map[string]interface{}) // Add SQ configuration to enable scalar quantization cfg["sq"] = map[string]interface{}{ "enabled": true, "rescoreLimit": 200, // Optional: number of candidates to fetch before rescoring "trainingLimit": 50000, // Optional: number of vectors to use for training "cache": true, // Optional: enable caching of quantized vectors } // Update the class configuration class.VectorIndexConfig = cfg // Apply the updated configuration to the collection err = client.Schema().ClassUpdater(). WithClass(class).Do(context.Background()) if err != nil { log.Fatalf("update class to use sq: %v", err) } ``` ```java title="Java" CollectionHandle> collection = client.collections.use("MyCollection"); collection.config.update(c -> c.vectorConfig(VectorConfig .text2vecTransformers(vc -> vc.quantization(Quantization.sq())))); ``` ```csharp title="C#" await collection.Config.Update(c => { var vectorConfig = c.VectorConfig["default"]; vectorConfig.VectorIndexConfig.UpdateHNSW(h => h.Quantizer = new VectorIndex.Quantizers.SQ() ); }); ``` ::: ## SQ parameters To tune SQ, set these `vectorIndexConfig` parameters. | Parameter | Type | Default | Details | | :---------------------- | :------ | :----------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `sq`: `enabled` | boolean | `false` | Uses SQ when `true`.

The Python client does not use the `enabled` parameter. To enable SQ with the v4 client, set a `quantizer` in the collection definition. | | `sq`: `rescoreLimit` | integer | `20` (`hnsw`)
`-1` (`flat`) | The minimum number of candidates to fetch before rescoring.

The default depends on the vector index type: `20` under `hnsw`, and `-1` under `flat`, which lets Weaviate pick the limit. | | `sq`: `trainingLimit` | integer | 100000 | The size of the training set to determine scalar bucket boundaries. | | `vectorCacheMaxObjects` | integer | `1e12` | Maximum number of objects in the memory cache. By default, this limit is set to one trillion (`1e12`) objects when a new collection is created. For sizing recommendations, see [Vector cache considerations](../indexing/vector-index.md#vector-cache-considerations). | :::code-group{sync="languages"} ```python title="Python" {7-11} from weaviate.classes.config import Configure client.collections.create( name="MyCollection", vector_config=Configure.Vectors.text2vec_openai( name="default", quantizer=Configure.VectorIndex.Quantizer.sq( rescore_limit=200, training_limit=50000, cache=True, ), vector_index_config=Configure.VectorIndex.hnsw( vector_cache_max_objects=100000, ), ), ) ``` ```typescript title="JavaScript/TypeScript" const collection = await client.collections.create({ name: 'MyCollection', vectorizers: weaviate.configure.vectors.selfProvided({ vectorIndexConfig: weaviate.configure.vectorIndex.hnsw({ quantizer: weaviate.configure.vectorIndex.quantizer.sq({ rescoreLimit: 200, // The minimum number of candidates to fetch before rescoring trainingLimit: 50000, // The size of the training set used to determine the bucket boundaries }), vectorCacheMaxObjects: 100000 // Maximum number of objects in the vector cache }) }) }) ``` ```go title="Go" {2-7,14} // Define a custom configuration for SQ. sq_with_options_config := map[string]interface{}{ "enabled": true, "rescoreLimit": 200, // The number of candidates to fetch before rescoring "trainingLimit": 50000, // The number of vectors to use for training the quantizer "cache": true, // Enable caching of quantized vectors } // Define the class schema with the custom SQ config and other HNSW settings class_with_options := &models.Class{ Class: className, Vectorizer: "text2vec-openai", VectorIndexConfig: map[string]interface{}{ "sq": sq_with_options_config, "distance": "cosine", // Set the distance metric for HNSW "vectorCacheMaxObjects": 100000, // Configure the vector cache }, } // Create the collection in Weaviate err = client.Schema().ClassCreator(). WithClass(class_with_options). Do(context.Background()) ``` ```java title="Java" {3-5} client.collections.create("MyCollection", col -> col.vectorConfig(VectorConfig.text2vecTransformers(vc -> vc .quantization(Quantization .sq(q -> q.cache(true).trainingLimit(50000).rescoreLimit(200))) .vectorIndex(Hnsw.of(c -> c.vectorCacheMaxObjects(100000))) )).properties(Property.text("title"))); ``` ```csharp title="C#" await client.Collections.Create( new CollectionCreateParams { Name = "MyCollection", Properties = [Property.Text("title")], VectorConfig = Configure.Vector( "default", v => v.Text2VecTransformers(), index: new VectorIndex.HNSW { VectorCacheMaxObjects = 100000, Quantizer = new VectorIndex.Quantizers.SQ { TrainingLimit = 50000, RescoreLimit = 200, }, } // highlight-end ), } ); ``` ::: ## Additional considerations ### Multiple vector embeddings (named vectors) Collections can have multiple [named vectors](../reference-configuration/collections.md#multiple-vector-embeddings-named-vectors). The vectors in a collection can have their own configurations, and compression must be enabled independently for each vector. Every vector is independent and can use [PQ](compression-pq-compression.md), [BQ](compression-bq-compression.md), [RQ](compression-rq-compression.md), [SQ](compression-sq-compression.md), or no compression. ### Multi-vector embeddings (ColBERT, ColPali, etc.) :::callout{intent="info" title="Added in `v1.30`"} ::: Multi-vector embeddings (implemented through models like ColBERT, ColPali, or ColQwen) represent each object or query using multiple vectors instead of a single vector. Just like with single vectors, multi-vectors support [PQ](compression-pq-compression.md), [BQ](compression-bq-compression.md), [RQ](compression-rq-compression.md), [SQ](compression-sq-compression.md), or no compression. During the initial search phase, compressed vectors are used for efficiency. However, when computing the `MaxSim` operation, uncompressed vectors are utilized to ensure more precise similarity calculations. This approach balances the benefits of compression for search efficiency with the accuracy of uncompressed vectors during final scoring. ## Further resources - [Starter guides: Compression](../starter-guides/managing-resources-compression.md) - [Reference: Vector index](../reference-configuration/indexing-vector-index.md) - [Concepts: Vector quantization](../concepts/vector-quantization.md) - [Concepts: Vector index](../indexing/vector-index.md) ## Questions and feedback Have a question or feedback? Here's how to reach us. ::::card-grid :::card{title="Community Forum" href="https://forum.weaviate.io/c/support" icon="messages-square"} Ask questions and connect with other developers on our **Community forum**. ::: :::card{title="Support" href="/guides/support-overview" icon="life-buoy"} Weaviate Cloud user or customer? Find the right channel on the **Support page**. ::: :::: ## Related pages - [Agents](./agents-index.md) - [AI-assisted Weaviate code generation](./ai-assisted-vibe-coding-index.md) - [APIs](./apis-index.md) - [Authorization and authentication](./authorization-and-authentication-index.md) - [Benchmarks](./benchmarks-index.md) - [Best practices](./best-practices-index.md) - [Client libraries](./clients-index.md) - [Client Libraries / SDKs](./client-libraries-index.md) - [Cloud](./cloud-index.md) - [Cloud account management](./cloud-account-management-index.md) # Agent Instructions This portal answers questions programmatically. To receive a synthesized, source-cited answer instead of crawling page by page, append the `?ask=` query parameter to any page URL on this site: /guides/quickstart?ask=how+do+I+authenticate Optional parameters: - `&goal=` steers the answer toward your objective (e.g. `&goal=write+a+python+client`). - `&version=