An **inverted index** is a data structure in Weaviate that enables efficient text search and filtering operations. :::accordion{title="Additional information"} In Weaviate, the inverted index supports search capabilities such as keyword search, filtering, and range queries. An inverted index maps from terms (_tokens_) back to the objects that contain them. This mapping allows Weaviate to quickly identify which objects contain specific terms or match certain criteria during search queries. You can [enable inverted indexes](#enable-inverted-index-for-keyword-searches-and-filtering) on properties and adjust various parameters that control [indexing behavior](#set-inverted-index-parameters) and [tokenization strategies](#set-tokenization-type-for-property). Proper configuration of these parameters is crucial for optimizing both search performance and storage efficiency. ::: ## Enable inverted index for keyword searches and filtering [Inverted index parameters](../reference-configuration/indexing-inverted-index.md#inverted-index-parameters) control how individual properties are indexed for search and filtering operations. These parameters determine whether specific properties can be searched, filtered, or used in range queries. :::accordion{title="Enabling inverted index"} The inverted index in Weaviate can be enabled through parameters at the property level. The names below are the REST and camelCase client spellings; the Python client uses the snake\_case equivalent, so `indexFilterable` is `index_filterable`, `indexSearchable` is `index_searchable` and `indexRangeFilters` is `index_range_filters`. **`indexFilterable`** - Controls whether a property can be used in where filters. When set to `true`, the property values are indexed for efficient filtering operations. Disable this for properties that don't need filtering to save storage space. **`indexSearchable`** - Determines whether a property participates in keyword search queries. When `true`, the property's text content is tokenized and indexed for search. Set to `false` for properties that shouldn't be searchable to improve performance. **`indexRangeFilters`** - Enables range filtering capabilities (greater than, less than, etc.) for numerical and date properties. When enabled, additional indexing structures are created to support efficient range queries. ::: :::code-group{sync="languages"} ```python title="Python" {10-11,16-17,22} from weaviate.classes.config import Configure, Property, DataType client.collections.create( "Article", # Additional settings not shown properties=[ Property( name="title", data_type=DataType.TEXT, index_filterable=True, index_searchable=True, ), Property( name="chunk", data_type=DataType.TEXT, index_filterable=True, index_searchable=True, ), Property( name="chunk_number", data_type=DataType.INT, index_range_filters=True, ), ], ) ``` ```typescript title="JavaScript/TypeScript" {7-8,13-14,19} await client.collections.create({ name: 'Article', properties: [ { name: 'title', dataType: dataType.TEXT, indexFilterable: true, indexSearchable: true, }, { name: 'chunk', dataType: dataType.TEXT, indexFilterable: true, indexSearchable: true, }, { name: 'chunk_number', dataType: dataType.INT, indexRangeFilters: true, }, ], }) ``` ```go title="Go" vTrue := true vFalse := false articleClass := &models.Class{ Class: "Article", Description: "Collection of articles", Properties: []*models.Property{ { Name: "title", DataType: schema.DataTypeText.PropString(), Tokenization: "lowercase", IndexFilterable: &vTrue, IndexSearchable: &vFalse, }, { Name: "chunk", DataType: schema.DataTypeText.PropString(), Tokenization: "word", IndexFilterable: &vTrue, IndexSearchable: &vTrue, }, { Name: "chunk_no", DataType: schema.DataTypeInt.PropString(), IndexRangeFilters: &vTrue, }, }, InvertedIndexConfig: &models.InvertedIndexConfig{ Bm25: &models.BM25Config{ B: 0.7, K1: 1.25, }, IndexNullState: true, IndexPropertyLength: true, IndexTimestamps: true, }, } ``` ```java title="Java" client.collections.create("Article", col -> col .properties( Property.text("title", p -> p.indexFilterable(true).indexSearchable(true)), Property.text("chunk", p -> p.indexFilterable(true).indexSearchable(true)), Property.integer("chunk_number", p -> p.indexRangeFilters(true))) .invertedIndex(idx -> idx.bm25(b -> b.b(1).k1(2)) .indexNulls(true) .indexPropertyLength(true) .indexTimestamps(true))); ``` ```csharp title="C#" await client.Collections.Create( new CollectionCreateParams { Name = "Article", Properties = [ Property.Text("title", indexFilterable: true, indexSearchable: true), Property.Text("chunk", indexFilterable: true, indexSearchable: true), Property.Int("chunk_number", indexRangeFilters: true), ], InvertedIndexConfig = new InvertedIndexConfig { Bm25 = new BM25Config { B = 1, K1 = 2 }, IndexNullState = true, IndexPropertyLength = true, IndexTimestamps = true, }, } ); ``` ::: ## Set inverted index parameters [Inverted index parameters](../reference-configuration/indexing-inverted-index.md#inverted-index-parameters) control the overall behavior of the inverted index for an entire collection. These parameters affect ranking algorithms, null value handling, and timestamp indexing across all properties in the collection. :::accordion{title="Inverted index parameters"} The inverted index in Weaviate can be configured through various parameters at the collection level. The names below are the REST and camelCase client spellings. In REST, `b` and `k1` are members of the `bm25` object. The Python client uses the snake\_case equivalent, so they are `bm25_b`, `bm25_k1`, `index_null_state`, `index_property_length` and `index_timestamps`. **`bm25`: `b`** - Controls the degree of normalization by document length in the BM25 ranking algorithm. Values range from 0 to 1, where 0 means no length normalization and 1 means full normalization. Higher values favor shorter documents. **`bm25`: `k1`** - Controls term frequency saturation in BM25. Higher values make term frequency more important, while lower values reduce the impact of term frequency on scoring. **`indexNullState`** - Determines whether null values are indexed. When enabled, you can filter for objects that have null values in specific properties. **`indexPropertyLength`** - Controls whether the length of text properties is indexed. When enabled, allows filtering based on text length and can improve certain ranking algorithms. **`indexTimestamps`** - Enables indexing of creation and update timestamps for objects, allowing filtering and sorting operations. ::: :::code-group{sync="languages"} ```python title="Python" {6-12} from weaviate.classes.config import Configure, Property, DataType client.collections.create( "Article", # Additional settings not shown inverted_index_config=Configure.inverted_index( bm25_b=0.7, bm25_k1=1.25, index_null_state=True, index_property_length=True, index_timestamps=True, ), ) ``` ```typescript title="JavaScript/TypeScript" import { dataType } from 'weaviate-client'; ``` ```go title="Go" vTrue := true vFalse := false articleClass := &models.Class{ Class: "Article", Description: "Collection of articles", Properties: []*models.Property{ { Name: "title", DataType: schema.DataTypeText.PropString(), Tokenization: "lowercase", IndexFilterable: &vTrue, IndexSearchable: &vFalse, }, { Name: "chunk", DataType: schema.DataTypeText.PropString(), Tokenization: "word", IndexFilterable: &vTrue, IndexSearchable: &vTrue, }, { Name: "chunk_no", DataType: schema.DataTypeInt.PropString(), IndexRangeFilters: &vTrue, }, }, InvertedIndexConfig: &models.InvertedIndexConfig{ Bm25: &models.BM25Config{ B: 0.7, K1: 1.25, }, IndexNullState: true, IndexPropertyLength: true, IndexTimestamps: true, }, } ``` ```java title="Java" client.collections.create("Article", col -> col .properties( Property.text("title", p -> p.indexFilterable(true).indexSearchable(true)), Property.text("chunk", p -> p.indexFilterable(true).indexSearchable(true)), Property.integer("chunk_number", p -> p.indexRangeFilters(true))) .invertedIndex(idx -> idx.bm25(b -> b.b(1).k1(2)) .indexNulls(true) .indexPropertyLength(true) .indexTimestamps(true))); ``` ```csharp title="C#" await client.Collections.Create( new CollectionCreateParams { Name = "Article", Properties = [ Property.Text("title", indexFilterable: true, indexSearchable: true), Property.Text("chunk", indexFilterable: true, indexSearchable: true), Property.Int("chunk_number", indexRangeFilters: true), ], InvertedIndexConfig = new InvertedIndexConfig { Bm25 = new BM25Config { B = 1, K1 = 2 }, IndexNullState = true, IndexPropertyLength = true, IndexTimestamps = true, }, } ); ``` ::: ## Drop an inverted index Drop (delete) an inverted index from a property. This is a destructive operation: the index data is removed from disk. To use the index again, it must be regenerated. The following index types can be dropped: `searchable`, `filterable`, `rangeFilters`. :::code-group{sync="languages"} ```python title="Python" {3-10} collection = client.collections.get("Article") # Drop the searchable inverted index from the "title" property collection.config.delete_property_index("title", "searchable") # Drop the filterable inverted index from the "title" property collection.config.delete_property_index("title", "filterable") # Drop the range filter index from the "chunk_number" property collection.config.delete_property_index("chunk_number", "rangeFilters") ``` ```typescript title="JavaScript/TypeScript" {3-10} const article = client.collections.use('Article') // Drop the searchable inverted index from the "title" property await article.config.dropInvertedIndex('title', 'searchable') // Drop the filterable inverted index from the "title" property await article.config.dropInvertedIndex('title', 'filterable') // Drop the range filter index from the "chunk_number" property await article.config.dropInvertedIndex('chunk_number', 'rangeFilters') ``` ```go title="Go" {3-22} collection := client.Schema() // Drop the searchable inverted index from the "title" property err = collection.PropertyIndexDeleter(). WithClassName("Article"). WithPropertyName("title"). WithSearchable(). Do(ctx) // Drop the filterable inverted index from the "title" property err = collection.PropertyIndexDeleter(). WithClassName("Article"). WithPropertyName("title"). WithFilterable(). Do(ctx) // Drop the range filter index from the "chunk_no" property err = collection.PropertyIndexDeleter(). WithClassName("Article"). WithPropertyName("chunk_no"). WithRangeFilters(). Do(ctx) ``` ```java title="Java" {3-10} var collection = client.collections.use("Article"); // Drop the searchable inverted index from the "title" property collection.config.dropPropertyIndex("title", PropertyIndexType.SEARCHABLE); // Drop the filterable inverted index from the "title" property collection.config.dropPropertyIndex("title", PropertyIndexType.FILTERABLE); // Drop the range filter index from the "chunk_number" property collection.config.dropPropertyIndex("chunk_number", PropertyIndexType.RANGE_FILTERS); ``` ::: ## Set tokenization type for property Configure a [tokenization method](../reference-configuration/collections.md#tokenization) for each property individually. :::accordion{title="Tokenization methods"} Tokenization determines how text content is broken down into individual terms that can be indexed and searched. Weaviate supports several tokenization strategies: **`word`** - The default tokenization that splits text on whitespace and punctuation, converting to lowercase. Best for general text search where you want to match individual words. **`lowercase`** - Splits text on whitespace only, then lowercases each token. Preserves symbols (like `&`, `@`, `_`) that `word` tokenization would strip. Good for case-insensitive matching where punctuation is meaningful, such as code snippets or email addresses. **`whitespace`** - Splits text only on whitespace characters, preserving punctuation and case. Good when punctuation is meaningful for search. **`field`** - Treats the entire property value as a single token without any processing. Use for exact matching of complete field values like IDs, email addresses, or URLs. **`trigram`** - Breaks text into overlapping 3-character sequences. Enables fuzzy matching and is useful for handling typos or partial matches. **`gse`** - Language-aware tokenization for Chinese and Japanese text. Disabled by default. Enable with the `ENABLE_TOKENIZER_GSE` environment variable. For Korean text, see the `kagome_kr` option. For the full list of supported tokenizers (including `kagome_ja`, `kagome_kr`, and the per-property text-analyzer options), see the [tokenization reference](../reference-configuration/collections.md#tokenization). ::: :::code-group{sync="languages"} ```python title="Python" {10-11,16} from weaviate.classes.config import Configure, Property, DataType, Tokenization client.collections.create( "Article", vector_config=Configure.Vectors.text2vec_cohere(), properties=[ Property( name="title", data_type=DataType.TEXT, tokenization=Tokenization.LOWERCASE, # Use "lowercase" tokenization description="The title of the article.", # Optional description ), Property( name="body", data_type=DataType.TEXT, tokenization=Tokenization.WHITESPACE, # Use "whitespace" tokenization ), ], ) ``` ```typescript title="JavaScript/TypeScript" {8,13} const newCollection = await client.collections.create({ name: 'Article', vectorizers: vectors.text2VecHuggingFace(), properties: [ { name: 'title', dataType: dataType.TEXT, tokenization: tokenization.LOWERCASE }, { name: 'body', dataType: dataType.TEXT, tokenization: tokenization.WHITESPACE }, ], }) ``` ```go title="Go" vTrue := true vFalse := false articleClass := &models.Class{ Class: "Article", Description: "Collection of articles", Properties: []*models.Property{ { Name: "title", DataType: schema.DataTypeText.PropString(), Tokenization: "lowercase", IndexFilterable: &vTrue, IndexSearchable: &vFalse, ModuleConfig: map[string]interface{}{ "text2vec-cohere": map[string]interface{}{ "vectorizePropertyName": true, }, }, }, { Name: "body", DataType: schema.DataTypeText.PropString(), Tokenization: "whitespace", IndexFilterable: &vTrue, IndexSearchable: &vTrue, ModuleConfig: map[string]interface{}{ "text2vec-cohere": map[string]interface{}{ "vectorizePropertyName": false, }, }, }, }, Vectorizer: "text2vec-cohere", } ``` ```java title="Java" client.collections.create("Article", col -> col.properties( Property.text("title", p -> p.description("The title of the article.") .tokenization(Tokenization.LOWERCASE) .vectorizePropertyName(false)), Property.text("body", p -> p.skipVectorization(true) .tokenization(Tokenization.WHITESPACE)))); ``` ```csharp title="C#" await client.Collections.Create( new CollectionCreateParams { Name = "Article", Properties = [ Property.Text("title", tokenization: PropertyTokenization.Lowercase), Property.Text("body", tokenization: PropertyTokenization.Whitespace), ], } ); ``` ::: ## Further resources - [References: Collection definition](../reference-configuration/collections.md) - [Concepts: Inverted index](../reference-configuration/indexing-inverted-index.md) ## Questions and feedback Have a question or feedback? Here's how to reach us. ::::card-grid :::card{title="Community Forum" href="https://forum.weaviate.io/c/support" icon="messages-square"} Ask questions and connect with other developers on our **Community forum**. ::: :::card{title="Support" href="/guides/support-overview" icon="life-buoy"} Weaviate Cloud user or customer? Find the right channel on the **Support page**. ::: :::: ## Related pages - [Agents](./agents-index.md) - [AI-assisted Weaviate code generation](./ai-assisted-vibe-coding-index.md) - [APIs](./apis-index.md) - [Authorization and authentication](./authorization-and-authentication-index.md) - [Benchmarks](./benchmarks-index.md) - [Best practices](./best-practices-index.md) - [Client libraries](./clients-index.md) - [Client Libraries / SDKs](./client-libraries-index.md) - [Cloud](./cloud-index.md) - [Cloud account management](./cloud-account-management-index.md) # Agent Instructions This portal answers questions programmatically. To receive a synthesized, source-cited answer instead of crawling page by page, append the `?ask=` query parameter to any page URL on this site: /guides/quickstart?ask=how+do+I+authenticate Optional parameters: - `&goal=` steers the answer toward your objective (e.g. `&goal=write+a+python+client`). - `&version=