## In short - The Question and Answer (Q\&A) module is a Weaviate module for answer extraction from data. - The module depends on a text vectorization module that should be running with Weaviate. - The module adds an `ask {}` operator to the GraphQL `Get {}` queries - The module returns a max. of 1 answer in the GraphQL `_additional {}` field. - The answer with the highest `certainty` (confidence level) will be returned. ## Introduction The Question and Answer (Q\&A) module is a Weaviate module for answer extraction from data. It uses BERT-related models for finding and extracting answers. This module can be used in GraphQL `Get{...}` queries, as a search operator. The `qna-transformers` module tries to find an answer in the data objects of the specified class. If an answer is found within the given `certainty` range, it will be returned in the GraphQL `_additional { answer { ... } }` field. There will be a maximum of 1 answer returned, if this is above the optionally set `certainty`. The answer with the highest `certainty` (confidence level) will be returned. There are currently five different Question Answering models available (source: [Hugging Face Model Hub](https://huggingface.co/models)): [`distilbert-base-uncased-distilled-squad (uncased)`](https://huggingface.co/distilbert-base-uncased-distilled-squad), [`bert-large-uncased-whole-word-masking-finetuned-squad (uncased)`](https://huggingface.co/bert-large-uncased-whole-word-masking-finetuned-squad), [`distilbert-base-cased-distilled-squad (cased)`](https://huggingface.co/distilbert-base-cased-distilled-squad), [`deepset/roberta-base-squad2`](https://huggingface.co/deepset/roberta-base-squad2), and [`deepset/bert-large-uncased-whole-word-masking-squad2 (uncased)`](https://huggingface.co/deepset/bert-large-uncased-whole-word-masking-squad2). Note that not all models perform well on every dataset and use case. We recommend to use `bert-large-uncased-whole-word-masking-finetuned-squad (uncased)`, which performs best on most datasets (although it's quite heavyweighted). Starting with `v1.10.0`, the answer score can be used as a reranking factor for the search results. ## How to enable (module configuration) ### Docker Compose The Q\&A module can be added as a service to the Docker Compose file. You must have a text vectorizer like `text2vec-contextionary` or `text2vec-transformers` running. An example Docker Compose file for using the `qna-transformers` module (`bert-large-uncased-whole-word-masking-finetuned-squad (uncased)`) in combination with the `text2vec-transformers`is as follows: ```yaml --- services: weaviate: command: - --host - 0.0.0.0 - --port - '8080' - --scheme - http image: cr.weaviate.io/semitechnologies/weaviate:1.38.2 ports: - 8080:8080 - 50051:50051 restart: on-failure:0 environment: TRANSFORMERS_INFERENCE_API: 'http://text2vec-transformers:8080' QNA_INFERENCE_API: "http://qna-transformers:8080" QUERY_DEFAULTS_LIMIT: 25 AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true' PERSISTENCE_DATA_PATH: '/var/lib/weaviate' ENABLE_MODULES: 'text2vec-transformers,qna-transformers' CLUSTER_HOSTNAME: 'node1' text2vec-transformers: image: cr.weaviate.io/semitechnologies/transformers-inference:sentence-transformers-msmarco-distilbert-base-v2 environment: ENABLE_CUDA: '1' NVIDIA_VISIBLE_DEVICES: all deploy: resources: reservations: devices: - capabilities: [gpu] qna-transformers: image: cr.weaviate.io/semitechnologies/qna-transformers:bert-large-uncased-whole-word-masking-finetuned-squad environment: ENABLE_CUDA: '1' NVIDIA_VISIBLE_DEVICES: all deploy: resources: reservations: devices: - capabilities: [gpu] ... ``` Variable explanations: - `QNA_INFERENCE_API`: where the qna module is running - `ENABLE_CUDA`: if set to 1 it uses GPU (if available on the host machine) _Note: at the moment, text vectorization modules cannot be combined in a single setup. This means that you can either enable the `text2vec-contextionary`, the `text2vec-transformers` or no text vectorization module._ ## How to use (GraphQL) ### GraphQL Ask search This module adds a search operator to GraphQL `Get{...}` queries: `ask{}`. This new operator takes the following arguments: | Field | Data Type | Required | Example value | Description | | ------------ | --------------- | -------- | --------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `question` | string | yes | `"What is the name of the Dutch king?"` | The question to be answered. | | `certainty` | float | no | `0.75` | Desired minimal certainty or confidence of answer to the question. The higher the value, the stricter the search becomes. The lower the value, the fuzzier the search becomes. If no certainty is set, any answer that could be extracted will be returned | | `properties` | list of strings | no | `["summary"]` | The properties of the queries Class which contains text. If no properties are set, all are considered. | | `rerank` | bool | no | `true` | If enabled, the qna module will rerank the result based on the answer score. For example, if the 3rd result - as determined by the previous (semantic) search contained the most likely answer, result 3 will be pushed to position 1, etc. _Not supported prior to v1.10.0_ | Notes: - The GraphQL `Explore { }` function does support the `ask` searcher, but the result is only a beacon to the object containing the answer. It is thus not any different from performing a nearText semantic search with the question. No extraction is happening. - You cannot use the `'ask'` operator along with a `'nearXXX'` operator! ### Example query :::code-group{sync="languages"} ```python title="Python" # QnA module use is not yet supported by the V4 client. Please use a raw GraphQL query instead. response = client.graphql_raw_query( """ { Get { Article( ask: { question: "Who is the king of the Netherlands?", properties: ["summary"], }, limit: 1 ) { title _additional { answer { hasAnswer property result startPosition endPosition } } } } } """ ``` ```go title="Go" package main import ( "context" "fmt" "github.com/weaviate/weaviate-go-client/v5/weaviate" "github.com/weaviate/weaviate-go-client/v5/weaviate/graphql" ) func main() { cfg := weaviate.Config{ Host: "localhost:8080", Scheme: "http", } client, err := weaviate.NewClient(cfg) if err != nil { panic(err) } className := "Article" fields := []graphql.Field{ {Name: "title"}, {Name: "_additional", Fields: []graphql.Field{ {Name: "answer", Fields: []graphql.Field{ {Name: "hasAnswer"}, {Name: "certainty"}, {Name: "property"}, {Name: "result"}, {Name: "startPosition"}, {Name: "endPosition"}, }}, }}, } ask := client.GraphQL().AskArgBuilder(). WithQuestion("Who is the king of the Netherlands?"). WithProperties([]string{"summary"}) ctx := context.Background() result, err := client.GraphQL().Get(). WithClassName(className). WithFields(fields...). WithAsk(ask). WithLimit(1). Do(ctx) if err != nil { panic(err) } fmt.Printf("%v", result) } ``` ```bash title="Curl" echo '{ "query": "{ Get { Article( ask: { question: \"Who is the king of the Netherlands?\", properties: [\"summary\"] }, limit: 1 ) { title _additional { answer { hasAnswer property result startPosition endPosition } } } } } " }' | curl \ -X POST \ -H 'Content-Type: application/json' \ -H 'Authorization: Bearer learn-weaviate' \ -H "X-OpenAI-Api-Key: $OPENAI_API_KEY" \ -d @- \ https://edu-demo.weaviate.network/v1/graphql ``` ```graphql title="GraphQL" { Get { Article( ask: { question: "Who is the king of the Netherlands?", properties: ["summary"], }, limit: 1 ) { title _additional { answer { hasAnswer property result startPosition endPosition } } } } } ``` ::: ### GraphQL response The answer is contained in a new GraphQL `_additional` property called `answer`. It contains the following fields: - `hasAnswer` (`boolean`): could an answer be found? - `result` (nullable `string`): An answer if one could be found. `null` if `hasAnswer==false` - `certainty` (nullable `float`): The certainty of the answer returned. `null` if `hasAnswer==false` - `property` (nullable `string`): The property which contains the answer. `null` if `hasAnswer==false` - `startPosition` (`int`): The character offset where the answer starts. `0` if `hasAnswer==false` - `endPosition` (`int`): The character offset where the answer ends `0` if `hasAnswer==false` Note: `startPosition`, `endPosition` and `property` in the response are not guaranteed to be present. They are calculated by a case-insensitive string matching function against the input text. If the transformer model formats the output differently (e.g. by introducing spaces between tokens which were not present in the original input), the calculation of the position and determining the property fails. ### Example response ```json { "data": { "Get": { "Article": [ { "_additional": { "answer": { "certainty": 0.73, "endPosition": 26, "hasAnswer": true, "property": "summary", "result": "king willem - alexander", "startPosition": 48 } }, "title": "Bruised Oranges - The Dutch royals are botching covid-19 etiquette" } ] } }, "errors": null } ``` ## Custom Q\&A Transformer module You can use the same approach as for `text2vec-transformers`, see [here](../model-provider-integrations/transformers-embeddings-custom-image.md), i.e. either pick one of the pre-built containers or build your own container from your own model using the `semitechnologies/qna-transformers:custom` base image. Make sure that your model is compatible with Hugging Face's `transformers.AutoModelForQuestionAnswering`. ## How it works (under the hood) Under the hood, the model uses a two-step approach. First it performs a semantic search to find the documents (e.g. a Sentence, Paragraph, Article, etc.) most likely to contain the answer. In a second step, a BERT-style answer extraction is performed on all `text` and `string` properties of the document. There are now three possible outcomes: 1. No answer was found because the question can not be answered, 2. An answer was found, but did not meet the user-specified minimum certainty, so it was discarded (typically the case when the document is on topic, but does not contain an actual answer to the question), and 3. An answer was found that matches the desired certainty. It is returned to the user. The module performs a semantic search under the hood, so a `text2vec-...` module is required. It does not need to be of the same type as the `qna-...` module. For example, you can use a `text2vec-contextionary` module to perform the semantic search, and a `qna-transformers` module to extract the answer. ### Automatic sliding window for long documents If a text value in a data object is longer than 512 tokens, the Q\&A Transformer module automatically splits the text into smaller texts. The module uses a sliding window, i.e. overlapping pieces of text, to avoid a scenario that an answer cannot be found if it lies on a boundary. If an answer lies on the boundary, the Q\&A module returns the result (answer) with the highest score (as the sliding mechanism could lead to duplicates). ## Model license(s) The `qna-transformers` module is compatible with various models, each with their own license. For detailed information, see the license of the model you are using in the [Hugging Face Hub](https://huggingface.co/models). It is your responsibility to evaluate whether the terms of its license(s), if any, are appropriate for your intended use. ## Questions and feedback Have a question or feedback? Here's how to reach us. ::::card-grid :::card{title="Community Forum" href="https://forum.weaviate.io/c/support" icon="messages-square"} Ask questions and connect with other developers on our **Community forum**. ::: :::card{title="Support" href="/guides/support-overview" icon="life-buoy"} Weaviate Cloud user or customer? Find the right channel on the **Support page**. ::: :::: ## Related pages - [Agents](./agents-index.md) - [AI-assisted Weaviate code generation](./ai-assisted-vibe-coding-index.md) - [APIs](./apis-index.md) - [Authorization and authentication](./authorization-and-authentication-index.md) - [Benchmarks](./benchmarks-index.md) - [Best practices](./best-practices-index.md) - [Client libraries](./clients-index.md) - [Client Libraries / SDKs](./client-libraries-index.md) - [Cloud](./cloud-index.md) - [Cloud account management](./cloud-account-management-index.md) # Agent Instructions This portal answers questions programmatically. To receive a synthesized, source-cited answer instead of crawling page by page, append the `?ask=` query parameter to any page URL on this site: /guides/quickstart?ask=how+do+I+authenticate Optional parameters: - `&goal=` steers the answer toward your objective (e.g. `&goal=write+a+python+client`). - `&version=