# Retrieval augmented generation (RAG)

:::callout{intent="info" title="Related pages"}
- [Which Weaviate is right for me?](which-weaviate.md)
- [How-to: Retrieval augmented generation](../how-to-query-search/generative.md)
:::

This pages introduces you to retrieval augmented generation (RAG) using Weaviate. It covers:

- What RAG is.
- How to configure Weaviate for RAG.
- How to perform RAG.
- Importing data with RAG in mind.

### Prerequisites

This guide assumes some familiarity with Weaviate, but it is not required. If you are new to Weaviate, we suggest starting with the [Weaviate Quickstart guide](../quickstart/index.md).

## Background

### What is retrieval augmented generation?

Retrieval augmented generation is a powerful technique that retrieves relevant data to provide to large language models (LLMs) as context, along with the task prompt. It is also called RAG, generative search, or in-context learning in some cases.

### Why use RAG?

LLM are incredibly powerful, but can suffer from two important limitations. These limitation are that:

- They can confidently produce incorrect, or outdated, information (also called 'hallucination'); and
- They might simply not be trained on the information you need.

RAG remedies this problem with a two-step process.

The first step is to retrieve relevant data through a query. Then, in the second step, the LLM is prompted with a combination of the retrieve data with a user-provided query.

This provides in-context learning for the LLM, which causes it to use the relevant and up-to-date data rather than rely on recall from its training, or even worse, hallucinated outputs.

### Weaviate and retrieval augmented generation

Weaviate incorporates key functionalities to make RAG easier and faster.

For one, Weaviate's search capabilities make it easier to find relevant information. You can use any of similarity, keyword and hybrid searches, along with filtering capabilities to find the information you need.

Additionally, Weaviate has integrated RAG capabilities, so that the retrieval and generation steps are combined into a single query. This means that you can use Weaviate's search capabilities to retrieve the data you need, and then in the same query, prompt the LLM with the same data.

This makes it easier, faster and more efficient to implement RAG workflows in your application.

## Examples of RAG

Let's begin by viewing examples of RAG in action. We will then explore how to configure Weaviate for RAG.

We have run this demo with an OpenAI language model and a cloud instance of Weaviate. But you can run it with any [deployment method](which-weaviate.md) and with any generative AI [model integration](../model-provider-integrations/index.md).

Connect to the instance like so, remembering to replace the API key for the LLM used (OpenAI in this case) with your own API key:

:::code-group{sync="languages"}
```python title="Python"
import weaviate
from weaviate.classes.init import Auth
import os

client = weaviate.connect_to_weaviate_cloud(
    cluster_url=os.getenv("WEAVIATE_URL"),
    auth_credentials=Auth.api_key(api_key=os.getenv("WEAVIATE_API_KEY")),
    headers={
        "X-OpenAI-Api-Key": os.getenv("OPENAI_API_KEY")
    }
)
```

```typescript title="JavaScript/TypeScript"
import weaviate, { WeaviateClient } from 'weaviate-client';

const client: WeaviateClient = await weaviate.connectToWeaviateCloud(
  'https://WEAVIATE_INSTANCE_URL',  // Replace with your Weaviate endpoint
 {
   authCredentials: new weaviate.ApiKey('YOUR-WEAVIATE-API-KEY'),  // Replace with your Weaviate instance API key
   headers: {
     'X-OpenAI-Api-Key': process.env.OPENAI_API_KEY || '',  // Replace with your inference API key
   }
 }
)
```

```java title="Java"
// Pass the API key for your LLM provider, OpenAI in this case, as a header
client = WeaviateClient.connectToLocal(config -> config.setHeaders(
    Map.of("X-OpenAI-Api-Key", System.getenv("OPENAI_API_KEY"))));
```
:::

### Data retrieval

Let's take an illustrative example with passages from a book. Here, the Weaviate instance contains a collection of passages from the [Pro Git book](https://git-scm.com/book/en/v2).

Before we can generate text, we need to retrieve relevant data. Let's retrieve the three most similar passages to the meaning of `history of git` with a semantic search.

:::code-group{sync="languages"}
```python title="Python"
collection_name = "GitBookChunk"

chunks = client.collections.use(collection_name)
response = chunks.query.near_text(query="history of git", limit=3)
```

```typescript title="JavaScript/TypeScript"
const myCollection = client.collections.use('GitBookChunk');

const dataRetrievalResult = await myCollection.query.nearText(['states in git'], {
  returnProperties: ['chunk', 'chapter_title', 'chunk_index'],
  limit: 2, })

console.log(JSON.stringify(dataRetrievalResult, null, 2));
```

```java title="Java"
var dataRetrievalResponse =
    chunks.query.nearText("history of git", q -> q.limit(3));
```
:::

This should return a set of results like the following (truncated for brevity):

```json
{
  "data": {
    "Get": {
      "GitBookChunk": [
        {
          "chapter_title": "01-introduction",
          "chunk": "=== A Short History of Git\n\nAs with many great things in life, Git began with a bit of creative ...",
          "chunk_index": 0
        },
        {
          "chapter_title": "01-introduction",
          "chunk": "== Nearly Every Operation Is Local\n\nMost operations in Git need only local files and resources ...",
          "chunk_index": 2
        },
        {
          "chapter_title": "02-git-basics",
          "chunk": "==\nYou can specify more than one instance of both the `--author` and `--grep` search criteria...",
          "chunk_index": 2
        }
      ]
    }
  }
}
```

### Transform result sets

We can transform this result set into new text using RAG with just a minor modification of the code. First, let's use a `grouped task` prompt to summarize this information.

Run the following code snippet, and inspect the results:

:::code-group{sync="languages"}
```python title="Python" {7}
collection_name = "GitBookChunk"

chunks = client.collections.use(collection_name)
response = chunks.generate.near_text(
    query="history of git",
    limit=3,
    grouped_task="Summarize the key information here in bullet points"
)

print(response.generative.text)
```

```typescript title="JavaScript/TypeScript"
const groupedTaskResponse = await myCollection.generate.nearText("history of git",{
  singlePrompt: `Summarize the key information here in bullet points`
},
{
  returnProperties: ['chunk', 'chapter_title', 'chunk_index'],
  limit: 2,
})

console.log(groupedTaskResponse.generated);
```

```java title="Java" {3-4}
var transformResponse =
    chunks.generate.nearText("history of git", q -> q.limit(3),
        g -> g.groupedTask(
            "Summarize the key information here in bullet points")
    );

System.out.println("\n--- TransformResultSets Result ---");
System.out.println(transformResponse.generative().text());
```
:::

Here is our generated text:

```
  - Git began as a replacement for the proprietary DVCS called BitKeeper, which was used by the Linux kernel project.
  - The relationship between the Linux development community and BitKeeper broke down in 2005, leading to the development of Git by Linus Torvalds.
  - Git was designed with goals such as speed, simple design, strong support for non-linear development, and the ability to handle large projects efficiently.
  - Most operations in Git only require local files and resources, making them fast and efficient.
  - Git allows browsing project history instantly and can calculate differences between file versions locally.
  - Git allows offline work and does not require a network connection for most operations.
  - This book was written using Git version 2, but most commands should work in older versions as well.
```

In a `grouped task` RAG query, Weaviate:

- Retrieves the three most similar passages to the meaning of `history of git`.
- Then prompts the LLM with a combination of:
  - Text from all of the search results, and
  - The user-provided prompt, `Summarize the key information here in bullet points`.

Note that the user-provided prompt did not contain any information about the subject matter. But because Weaviate retrieved the relevant data about the history of git, it was able to summarize the information relating to this subject matter using verifiable data.

That's how easy it is to perform RAG queries in Weaviate.

:::callout{intent="note" title="Your results may vary"}
There will be variability in the actual text that has been generated. This due to the randomness in LLMs' behaviors, and variability across models. This is perfectly normal.
:::

### Transform individual objects

In this example, we will take a look at how to transform individual objects. This is useful when you want to generate text for each object individually, rather than for the entire result set.

Here we prompt the model to translate individual wine reviews into French, using emojis. The reviews is a subset from a [publicly available dataset of wine reviews](https://www.kaggle.com/zynicide/wine-reviews).

Note that in this query, we apply a `single prompt` parameter. This means that the LLM is prompted with each object individually, rather than with the entire result set.

:::code-group{sync="languages"}
```python title="Python" {7-10}
collection_name = "WineReview"

reviews = client.collections.use(collection_name)
response = reviews.generate.near_text(
    query="fruity white wine",
    limit=3,
    single_prompt="""
        Translate this review into French, using emojis:
        ===== Country of origin: {country}, Title: {title}, Review body: {review_body}
    """
)
```

```typescript title="JavaScript/TypeScript"
const myWineCollection = client.collections.use('WineReview');

const singlePromptresult = await myWineCollection.generate.nearText("fruity white wine",{
  singlePrompt: `Translate this review into French, using emojis:
  ===== Country of origin: {country}, Title: {title}, Review body: {review_body}`
},{
  returnProperties: ['review_body','title','country','points'],
  limit: 5,
})

console.log(JSON.stringify(singlePromptresult.objects, null, 2));
```

```java title="Java" {3-5}
var wineResponse = wineReview.generate.nearText("fruity white wine",
    q -> q.limit(3),
    g -> g
        .singlePrompt("Translate this review into French, using emojis: "
            + "===== Country of origin: {country}, Title: {title}, Review body: {review_body}")
);
```
:::

As the query was run with a limit of 5, you should see 5 objects returned, including generated texts.

Here is our generated text for the first object, and the source text:

```
===== Gener =====
🇺🇸🍷🌿🍑🌼🍯🍊🍮🍽️🌟

Origine : États-Unis
Titre : Schmitz 24 Brix 2012 Sauvignon Blanc (Sierra Foothills)
Corps de la critique : Pas du tout un Sauvignon Blanc typique, il sent l'abricot et le chèvrefeuille et a le goût de la marmelade. Il est sec, mais a le goût d'un vin de dessert tardif. Attendez-vous à une petite aventure gustative ici.

===== Original review =====
Country: US,
Title: Schmitz 24 Brix 2012 Sauvignon Blanc (Sierra Foothills)
Review body Not at all a typical Sauvignon Blanc, this smells like apricot and honeysuckle and tastes like marmalade. It is dry, yet tastes like a late-harvest dessert wine. Expect a little taste adventure here.

```

Here, Weaviate has:

- Retrieved five most similar wine reviews to the meaning of `fruity white wine`.
- For each result, prompted the LLM with:
  - The user-provided prompt, replacing `{country}`, `{title}`, and `{review_body}` with the corresponding text.

In both examples, you saw Weaviate return new text that is original, but grounded in the retrieved data. This is what makes RAG powerful, by combining the best of data retrieval and language generation.

## RAG, end-to-end

Now, let's go through an end-to-end example for using Weaviate for RAG.

### Your own Weaviate instance

For this example, you will need access to a Weaviate instance that you can write to. You can use any Weaviate instance, such as a local Docker instance, or a WCD instance.

### Configure Weaviate for RAG

:::callout{intent="info" title="Generative model integration mutability"}
A collection's `generative` model integration configuration is mutable from `v1.25.23`, `v1.26.8` and `v1.27.1`. See [this section](../how-to-manage-collections/generative-reranker-models.md#update-the-generative-model-integration) for details on how to update the collection configuration.
:::

To use RAG, the appropriate `generative-xxx` module must be:

- Enabled in Weaviate, and
- Specified in the collection definition.

Each module is tied to a specific group of LLMs, such as `generative-cohere` for Cohere models, `generative-openai` for OpenAI models and `generative-google` for Google models.

If you are using WCD, you will not need to do anything to enable modules.

::::accordion{title="How to list enabled modules"}
You can check which modules are enabled by viewing the `meta` information for your Weaviate instance, as shown below:

:::code-group{sync="languages"}
```python title="Python"
response = client.get_meta()
print(response)
```

```typescript title="JavaScript/TypeScript"
const metaResponse = await client.getMeta()
console.log(metaResponse)
```

```java title="Java"
var metaResponse = client.meta();
System.out.println("\n--- ListModules Result ---");
System.out.println(metaResponse);
```
:::

The response will include a list of modules. Check that your desired module is enabled.
::::

:::accordion{title="How to enable modules"}
For configurable deployments, you can specify enabled modules. For example, in a Docker deployment, you can do so by listing them on the `ENABLE_MODULES` environment variable, as shown below:

```yaml
services:
  weaviate:
    environment:
      ENABLE_MODULES: "text2vec-cohere,text2vec-huggingface,text2vec-openai,text2vec-google,generative-cohere,generative-openai,generative-google"
```

Check the specific documentation for your deployment method ([Docker](../installation/installation-guides-docker-installation.md), [Kubernetes](../installation/installation-guides-k8s-installation.md), [Embedded Weaviate](../installation/installation-guides-embedded.md)) for more information on how to configure it.
:::

:::accordion{title="How to configure the language model"}
Model parameters are exposed through the generative model provider configuration. You can set them when you create the collection, alongside the generative integration itself.

For example, the `generative-cohere` integration can be configured as follows:

```python
from weaviate.classes.config import Configure

client.collections.create(
    "DemoCollection",
    generative_config=Configure.Generative.cohere(
        # # These parameters are optional
        # model="command-a-03-2025",
        # temperature=0.7,
        # max_tokens=500,
        # k=5,
        # stop_sequences=["\n\n"],
    )
    # Additional parameters not shown
)
```

And the `generative-openai` integration can be configured as follows:

```python
from weaviate.classes.config import Configure

client.collections.create(
    "DemoCollection",
    generative_config=Configure.Generative.openai(
        # # These parameters are optional
        # model="gpt-5-mini",
        # temperature=0.7,
        # max_tokens=500,
        # frequency_penalty=0,
        # presence_penalty=0,
        # top_p=0.7,
    )
    # Additional parameters not shown
)
```

Each parameter is optional. If you do not set a parameter, Weaviate applies the server-defined default. For the available models, the default model and the full parameter list, see the model provider pages for [Cohere](../model-provider-integrations/cohere-generative.md) and [OpenAI](../model-provider-integrations/openai-generative.md).

See the [documentation](../model-provider-integrations/index.md) for various model provider integrations.
:::

### Populate database

Adding data to Weaviate for RAG is similar to adding data for other purposes. However, there are some important considerations to keep in mind, such as chunking and data structure.

You can read further discussions in the [Best practices & tips](#best-practices--tips) section. Here, we will use a chunk length of 150 words and a 25-word overlap. We will also include the title of the book, the chapter it is from, and the chunk number. This will allow us to search through the chunks, as well as filter it.

#### Download & chunk

In the following snippet, we download a chapter of the `Pro Git` book, clean it and chunk it.

:::code-group{sync="languages"}
```python title="Python"
from typing import List


def download_and_chunk(src_url: str, chunk_size: int, overlap_size: int) -> List[str]:
    import requests
    import re

    response = requests.get(src_url)  # Retrieve source text
    source_text = re.sub(r"\s+", " ", response.text)  # Remove multiple whitespaces
    text_words = re.split(r"\s", source_text)  # Split text by single whitespace

    chunks = []
    for i in range(0, len(text_words), chunk_size):  # Iterate through & chunk data
        chunk = " ".join(text_words[max(i - overlap_size, 0): i + chunk_size])  # Join a set of words into a string
        chunks.append(chunk)
    return chunks


pro_git_chapter_url = "https://raw.githubusercontent.com/progit/progit2/main/book/01-introduction/sections/what-is-git.asc"
chunked_text = download_and_chunk(pro_git_chapter_url, 150, 25)
```

```typescript title="JavaScript/TypeScript"
async function downloadAndChunk(srcUrl: string, chunkSize: number, overlapSize: number) {
  const response = await fetch(srcUrl);
  const sourceText = await response.text();
  const textWords = sourceText.replace(/\s+/g, ' ').split(' ');

  let chunks = [];
  for (let i = 0; i < textWords.length; i += chunkSize) {
      let chunk = textWords.slice(Math.max(i - overlapSize, 0), i + chunkSize).join(' ');
      chunks.push(chunk);
  }
  return chunks;
}

const proGitChapterUrl = 'https://raw.githubusercontent.com/progit/progit2/main/book/01-introduction/sections/what-is-git.asc';
const chunks = await downloadAndChunk(proGitChapterUrl, 150, 25)
```

```javaraw title="Java"
private List<String> downloadAndChunk(String srcUrl, int chunkSize,
    int overlapSize) throws Exception {
  // Retrieve source text
  URL url = URI.create(srcUrl).toURL();
  HttpURLConnection conn = (HttpURLConnection) url.openConnection();
  conn.setRequestMethod("GET");
  String sourceText;
  try (BufferedReader reader = new BufferedReader(new InputStreamReader(conn.getInputStream()))) {
    sourceText = reader.lines().reduce("", String::concat);
  }

  // Remove multiple whitespaces
  sourceText = sourceText.replaceAll("\\s+", " ");
  // Split text by single whitespace
  String[] textWords = sourceText.split("\\s");

  List<String> chunks = new ArrayList<>();
  for (int i = 0; i < textWords.length; i += chunkSize) { // Iterate through & chunk data
    // Join a set of words into a string
    String[] chunkWords =
        Arrays.copyOfRange(textWords, Math.max(i - overlapSize, 0),
            Math.min(i + chunkSize, textWords.length));
    String chunk = String.join(" ", chunkWords);
    chunks.add(chunk);
  }
  return chunks;
}
```
:::

This will download the text from the chapter, and return a list/array of strings of 150 word chunks, with a 25-word overlap added in front.

#### Create collection definitions

We can now create a collection definition for the chunks. To use RAG, your desired generative module must be specified at the collection level as shown below.

The below collection definition for the `GitBookChunk` collection specifies `text2vec-openai` as the vectorizer and `generative-openai` as the generative module. Note that the `generative-openai` parameter can have an empty dictionary/object as its value, which will use the default parameters.

:::code-group{sync="languages"}
```python title="Python" {25-26}
import weaviate.classes as wvc


collection_name = "GitBookChunk"

if client.collections.exists(collection_name):  # In case we've created this collection before
    client.collections.delete(collection_name)  # THIS WILL DELETE ALL DATA IN THE COLLECTION

chunks = client.collections.create(
    name=collection_name,
    properties=[
        wvc.config.Property(
            name="chunk",
            data_type=wvc.config.DataType.TEXT
        ),
        wvc.config.Property(
            name="chapter_title",
            data_type=wvc.config.DataType.TEXT
        ),
        wvc.config.Property(
            name="chunk_index",
            data_type=wvc.config.DataType.INT
        ),
    ],
    vector_config=wvc.config.Configure.Vectors.text2vec_openai(),  # Use `text2vec-openai` as the vectorizer
    generative_config=wvc.config.Configure.Generative.openai(),  # Use `generative-openai` with default parameters
)
```

```typescript title="JavaScript/TypeScript" {17-18}
const schemaDefinition = {
  name: 'GitBookChunk',
  properties: [
    {
      name: 'Chunk',
      dataType: 'text' as const,
    },
    {
      name: 'chapter_title',
      dataType: 'text' as const,
    },
    {
      name: 'chunk_index',
      dataType: 'int' as const,
    }
  ],
  vectorizers: weaviate.configure.vectors.text2VecOpenAI(),
  generative: weaviate.configure.generative.openAI()
}
```

```java title="Java" {8-9}
if (client.collections.exists(gitBookCollectionName)) {
  client.collections.delete(gitBookCollectionName);
}

client.collections.create(gitBookCollectionName,
    col -> col.properties(Property.text("chunk"),
        Property.text("chapter_title"), Property.integer("chunk_index"))
        .vectorConfig(VectorConfig.text2vecOpenAi()) // Use `text2vec-openai` as the vectorizer
        .generativeModule(Generative.openai()) // Use `generative-openai` with default parameters
);
```
:::

#### Import data

Now, we can import the data into Weaviate.

:::code-group{sync="languages"}
```python title="Python"
chunks_list = list()
for i, chunk in enumerate(chunked_text):
    data_properties = {
        "chapter_title": "What is Git",
        "chunk": chunk,
        "chunk_index": i
    }
    data_object = wvc.data.DataObject(properties=data_properties)
    chunks_list.append(data_object)
chunks.data.insert_many(chunks_list)
```

```typescript title="JavaScript/TypeScript"
const gitCollection = client.collections.use('GitBookChunkTest');

async function importData(chunkData: Array<string>) {
  const list:Array<any> = [];

  for (const [index, chunk] of chunkData.entries()) {
    const obj = {
      properties: {
        chunk: chunk,
        chunk_index: index,
        chapter_title: 'What is Git',
      },
    };

    list.push(obj);
    }
  const result = await gitCollection.data.insertMany(list)
  console.log('just bulk inserted',result);
};

await importData(chunks);
```

```java title="Java"
List<WeaviateObject<Map<String, Object>>> chunksList =
    new ArrayList<>();
for (int i = 0; i < chunkedText.size(); i++) {
  Map<String, Object> dataProperties = new HashMap<>();
  dataProperties.put("chapter_title", "What is Git");
  dataProperties.put("chunk", chunkedText.get(i));
  dataProperties.put("chunk_index", (long) i); // Use long for integer

  chunksList.add(WeaviateObject.of(b -> b.properties(dataProperties)));
}
chunks.data.insertMany(chunksList);
```
:::

Once this is done, you should have imported a collection of chunks from the chapter into Weaviate. You can check this by running a simple aggregation query:

:::code-group{sync="languages"}
```python title="Python"
response = chunks.aggregate.over_all(total_count=True)
print(response.total_count)
```

```typescript title="JavaScript/TypeScript"
const objectCount =  await gitCollection.aggregate.overAll()
console.log(JSON.stringify(objectCount.totalCount));
```

```java title="Java"
var countResponse =
    chunks.aggregate.overAll(a -> a.includeTotalCount(true));
System.out
    .println("GitBookChunk total count: " + countResponse.totalCount());
```
:::

Which should indicate that there are `10` chunks in the database.

### Generative queries

Now that we have configured Weaviate and populated it with data, we can perform generative queries as you saw in the examples above.

#### Single (per-object) prompts

Single prompts tell Weaviate to generate text based on each retrieved object and the user-provided prompt. In this example, we retrieve two objects and prompt the language model to write a haiku based on the text of each chunk.

:::code-group{sync="languages"}
```python title="Python"
response = chunks.generate.fetch_objects(
    limit=2,
    single_prompt="Write the following as a haiku: ===== {chunk} "
)

for o in response.objects:
    print(f"\n===== Object index: [{o.properties['chunk_index']}] =====")
    print(o.generated)
```

```typescript title="JavaScript/TypeScript"
const haikuResponse = await gitCollection.generate.fetchObjects({
  singlePrompt: `Write the following as a haiku: ===== {chunk}`
},{
  returnProperties: ['chunk','chunk_index'],
  limit: 2,
})

if (haikuResponse) {
  for (const result of haikuResponse.objects) {
    console.log(`\n===== Object index: [${result.properties['chunk_index']}] =====`)
    console.log(result.generated)
  }
}
```

```java title="Java"
var singlePromptResponse =
    chunks.generate.fetchObjects(q -> q.limit(2), g -> g
        .singlePrompt("Write the following as a haiku: ===== {chunk} "));

System.out.println("\n--- SinglePrompt Results ---");
for (var o : singlePromptResponse.objects()) {
  System.out.printf("\n===== Object index: [%s] =====\n",
      o.properties().get("chunk_index"));
  System.out.println(o.generative().text());
}
```
:::

It should return haiku-like text, such as:

```
===== Object index: [1] =====
Git's data stored
As snapshots of files, not changes
Efficient and unique

===== Object index: [6] =====
Git has three states:
Untracked, modified, staged.
Commit to save changes.
```

#### Grouped tasks

A grouped task is a prompt that is applied to a group of objects. This allows you to prompt the language model with the entire set of search results, such as source documents or relevant passages.

In this example, we prompt the language model to write a trivia tweet based on the result.

:::code-group{sync="languages"}
```python title="Python"
response = chunks.generate.fetch_objects(
    limit=2,
    grouped_task="Write a trivia tweet based on this text. Use emojis and make it succinct and cute."
)

print(response.generative.text)
```

```typescript title="JavaScript/TypeScript"
const triviaResponse = await gitCollection.generate.fetchObjects({
  groupedTask: `Write a trivia tweet based on this text. Use emojis and make it succinct and cute.`
},{
  limit: 2,
})

console.log(triviaResponse.generated)
```

```java title="Java"
var groupedTaskResponse =
    chunks.generate.fetchObjects(q -> q.limit(2), g -> g.groupedTask(
        "Write a trivia tweet based on this text. Use emojis and make it succinct and cute."));

System.out.println("\n--- GroupedTask Result ---");
System.out.println(groupedTaskResponse.generative().text());
```
:::

It should return a factoid written for social media, such as:

```
Did you know? 🤔 Git thinks of its data as snapshots, not just changes to files.
📸 Every time you commit, Git takes a picture of all your files and stores a reference to that snapshot.
📂🔗 #GitTrivia
```

#### Pairing with search

RAG in Weaviate is a two-step process under the hood, involving retrieval of objects and then generation of text. This means that you can use the full power of Weaviate's search capabilities to retrieve the objects you want to use for generation.

In this example, we search the chapter for passages that relate to the states of git before generating a tweet as before.

:::code-group{sync="languages"}
```python title="Python"
response = chunks.generate.near_text(
    query="states of git",
    limit=2,
    grouped_task="Write a trivia tweet based on this text. Use emojis and make it succinct and cute."
)

print(response.generative.text)
```

```typescript title="JavaScript/TypeScript"
const searchResponse = await gitCollection.generate.nearText("states of git",{
  groupedTask: "Write a trivia tweet based on this text. Use emojis and make it succinct and cute."
},{
  limit: 2,
})

console.log('concept',JSON.stringify(searchResponse.generated, null, 2));
```

```java title="Java"
var nearTextResponse1 = chunks.generate.nearText("states of git",
    q -> q.limit(2), g -> g.groupedTask(
        "Write a trivia tweet based on this text. Use emojis and make it succinct and cute."));

System.out
    .println("\n--- NearTextGroupedTask (states of git) Result ---");
System.out.println(nearTextResponse1.generative().text());
```
:::

This should return text like:

```
📝 Did you know? Git has three main states for files: modified, staged, and committed.
🌳📦📂 Learn more about these states and how they affect your Git project!
#GitBasics #Trivia
```

Now, simply by changing the search query, we can generate similar content about different topics.

:::code-group{sync="languages"}
```python title="Python"
response = chunks.generate.near_text(
    query="how git saves data",
    limit=2,
    grouped_task="Write a trivia tweet based on this text. Use emojis and make it succinct and cute."
)

print(response.generative.text)
```

```typescript title="JavaScript/TypeScript"
const anotherSearchResponse = await gitCollection.generate.nearText("how git saves data",{
  groupedTask: "Write a trivia tweet based on this text. Use emojis and make it succinct and cute."
},{
  limit: 2,
})

console.log('concept',JSON.stringify(anotherSearchResponse.generated, null, 2));
```

```java title="Java"
var nearTextResponse2 = chunks.generate.nearText("how git saves data",
    q -> q.limit(2), g -> g.groupedTask(
        "Write a trivia tweet based on this text. Use emojis and make it succinct and cute."));

System.out.println(
    "\n--- SecondNearTextGroupedTask (how git saves data) Result ---");
System.out.println(nearTextResponse2.generative().text());
```
:::

In this case, the result should be something like:

```
Did you know? 🤔 Git stores everything by the hash value of its contents, not by file name!
📁🔍 It's hard to lose data in Git, making it a joy to use!
😄🔒 Git thinks of its data as a stream of snapshots, making it more than just a VCS!
📸🌟 Most Git operations are local, so no need for network latency!
🌐💨 #GitTrivia
```

As you can see, Weaviate allows you to use the full power of search to retrieve the objects you want to use for generation. This allows you to ground the language model in the context of up-to-date information, which you can retrieve with the full power of Weaviate's search capabilities.

## Best practices & tips

### Chunking

In the context of language processing, "chunking" refers to the process of splitting texts into smaller pieces of texts, i.e. "chunks".

For RAG, chunking affects both the information retrieval and the amount of contextual information provided.

While there is no one-size-fits all chunking strategy that we can recommend, we can provide some general guidelines. Chunking by semantic markers, or text length may both be viable strategies.

#### Chunking by semantic markers

Using semantic markers, such as paragraphs, or sections can be a good strategy that will allows you to retain related information in each chunk. Some potential risks are that chunk lengths may vary significantly, and outlier conditions may occur common (e.g. chunks with headers that are not particularly meaningful).

#### Chunking by text length

Using text length, such as 100-150 words, can be a robust baseline strategy. This will allow you to retrieve relevant information without having to worry about the exact length of the text. One potential risk is that chunks may be cut off where they are not semantically meaningful, cutting off important contextual information.

You could use a sliding window approach to mitigate this risk, by overlapping chunks. The length of each chunk can be adjusted to your needs, and based on any unit, such as words, tokens, or even characters.

A baseline strategy could involve using chunks created with a 100-200 word sliding window and a 50-word overlap.

#### Mixed-strategy chunking

Another, slightly more complicated strategy may be using paragraph-based chunks with a maximum and a minimum length, say of 200 words and 50 words respectively.

### Data structure

Another important consideration is the data structure. For example, your chunk object could also contain any additional source-level data, such as the title of the book, the chapter it is from, and the chunk number.

This will allow you to search through the chunks, as well as filter it. Then, you could use this information to control the generation process, such as by prompting the LLM with contextual data (chunks) in the order that they appear in the source document.

Additionally, you could link the chunks to the source document, allowing you to retrieve the source document, or even the entire source document, if needed.

### Complex prompts

While the field of prompting is relatively new, it has seen significant advancements already.

As one example, a technique called "[chain-of-thought prompting](https://arxiv.org/abs/2201.11903)" can be an effective technique. It suggests that the prompt can be used to nudge the model towards producing intermediate reasoning steps, which improves the quality of the answer.

We recommend keeping up to date with the latest developments in the field, and experimenting with different techniques.

Our own [Connor Shorten's podcast](https://weaviate.io/podcast) is a great resource for keeping up with the research, as are resources such as [Arxiv](https://arxiv.org/).

## Wrap-up

We've explored the dynamic capabilities of RAG in Weaviate, showcasing how it enhances large language models through retrieval-augmented generation.

To learn more about specific search capabilities, check out the [How-to: search guide](../how-to-query-search/index.md). And to learn more about individual modules, check out the [Model provider integrations](../model-provider-integrations/index.md).

## Questions and feedback

Have a question or feedback? Here's how to reach us.

::::card-grid
:::card{title="Community Forum" href="https://forum.weaviate.io/c/support" icon="messages-square"}
Ask questions and connect with other developers on our **Community forum**.
:::

:::card{title="Support" href="/guides/support-overview" icon="life-buoy"}
Weaviate Cloud user or customer? Find the right channel on the **Support page**.
:::
::::

## Related pages

- [Agents](./agents-index.md)
- [AI-assisted Weaviate code generation](./ai-assisted-vibe-coding-index.md)
- [APIs](./apis-index.md)
- [Authorization and authentication](./authorization-and-authentication-index.md)
- [Benchmarks](./benchmarks-index.md)
- [Best practices](./best-practices-index.md)
- [Client libraries](./clients-index.md)
- [Client Libraries / SDKs](./client-libraries-index.md)
- [Cloud](./cloud-index.md)
- [Cloud account management](./cloud-account-management-index.md)

# Agent Instructions

This portal answers questions programmatically. To receive a synthesized,
source-cited answer instead of crawling page by page, append the `?ask=`
query parameter to any page URL on this site:

    /guides/quickstart?ask=how+do+I+authenticate

Optional parameters:

- `&goal=<what-you-are-trying-to-do>` steers the answer toward your
  objective (e.g. `&goal=write+a+python+client`).
- `&version=<label>` scopes the answer to a mounted version when the
  portal publishes more than one.

The response is `text/markdown`: the answer followed by a `# Sources` list
of the portal pages it was grounded in. Status codes are the contract:

- `200` — the answer; `402` — the portal owner’s plan or answer credits are
  exhausted (surface this to your operator; do NOT retry); `429` — you are
  rate-limited; back off for the `Retry-After` seconds; `503` — the answer
  lane is temporarily unavailable; fall back to crawling the `.md` pages.

For the full corpus map read `llms.txt` at the site root; for the tool
surface (search + page fetch as MCP tools) see `/mcp`.