Weaviate's Backup feature is designed to work natively with cloud technology. Most notably, it allows: - Seamless integration with widely-used cloud blob storage, such as AWS S3, GCS, or Azure Storage - Backup and Restore between different storage providers - Single-command backup and restore - Choice of backing up an entire instance, or selected collections only - [Incremental backups](#incremental-backups) that only store changed data, reducing backup and speeding up backup times - Easy migration to new environments :::callout{intent="warning" title="Important backup considerations"} * **Version Requirements**: If you are running Weaviate `v1.23.12` or older, you must [update](../deploy/migration.md) to `v1.23.13` or higher before restoring a backup to prevent data corruption. * **[Multi-tenancy](../concepts/data.md#multi-tenancy) limitations**: Backups include both `active` (HOT) and `inactive` (COLD) tenants. Inactive tenants are backed up directly from disk without activation. `Offloaded` (FROZEN) tenants are still skipped since they have no local data. Inactive tenant support was added in `v1.37.0`, and backported to `v1.35.17` and `v1.36.10`. In earlier releases only active tenants are included, so be sure to [activate](../how-to-manage-collections/multi-tenancy.md#manage-tenant-states) any required tenants before creating a backup. ::: ## Backup Quickstart This quickstart demonstrates using backups in Weaviate using the local filesystem as a backup provider, which is suitable for development and testing environments. ### 1. Configure Weaviate Add these environment variables to your Weaviate configuration (e.g. Docker or Kubernetes configuration file): ```yaml # Enable the filesystem backup module ENABLE_MODULES=backup-filesystem # Set backup location (e.g. within a Docker container or on a Kubernetes pod) BACKUP_FILESYSTEM_PATH=/var/lib/weaviate/backups ``` ### 2. Start a backup Restart Weaviate to apply the new configuration. Then, you are ready to start a backup: :::code-group{sync="languages"} ```python title="Python" from weaviate.classes.backup import BackupLocation ``` ```typescript title="JavaScript/TypeScript" import weaviate from 'weaviate-client'; const client = await weaviate.connectToLocal(); let result = await client.backup.create({ backupId: 'my-very-first-backup', backend: 'filesystem', // Optional parameters // waitForCompletion: true, // includeCollections: ['Article', 'Publication'], // excludeCollections: ['Author'], }) console.log(result.status); ``` ```go title="Go" createResponse, err := client.Backup().Creator(). WithIncludeClassNames("Article", "Publication"). WithBackend(backup.BACKEND_FILESYSTEM). WithBackupID("my-very-first-backup"). WithWaitForCompletion(true). Do(context.Background()) ``` ```bash title="curl" curl \ -X POST \ -H "Content-Type: application/json" \ -d '{ "id": "my-very-first-backup", "include": ["Article", "Publication"] }' \ http://localhost:8080/v1/backups/filesystem ``` ::: That's all there is to getting started with backups in Weaviate. The backup will be stored in the specified location on the local filesystem. You can also: - [Restore the backup](#restore-backup) to a Weaviate instance - [Check the status](#asynchronous-status-checking) of the backup (if you did not wait for completion) - [Cancel the backup](#cancel-backup) if needed Note that local backups are not suitable for production environments. For production, use a cloud provider like S3, GCS, or Azure Storage. The following sections provide more details on how to configure and use backups in Weaviate. ## Configuration Weaviate supports four backup storage options: | Provider | Module Name | Best For | Multi-Node Support | | -------------------- | ------------------- | ------------------------------------------ | ------------------ | | AWS S3 | `backup-s3` | Production deployments, AWS environments | Yes | | Google Cloud Storage | `backup-gcs` | Production deployments, GCP environments | Yes | | Azure Storage | `backup-azure` | Production deployments, Azure environments | Yes | | Local Filesystem | `backup-filesystem` | Development, testing, single-node setups | No | To use any provider: 1. Enable the module - Add the module name to the `ENABLE_MODULES` environment variable - On Weaviate Cloud instances, a relevant default module is enabled 2. Configure the required modules - Option 1: Set the necessary environment variables - Option 2 (Kubernetes): Configure the [Helm chart values](#kubernetes-configuration) Note multiple providers can be enabled simultaneously. ### S3 (AWS or S3-compatible) - Works with Amazon S3 and S3-compatible object stores (for example, MinIO) - Supports multi-node deployments - Recommended for production use To configure `backup-s3`, you need to enable the module and provide the necessary configuration. #### Enable module Add `backup-s3` to the `ENABLE_MODULES` environment variable. For example, to enable the S3 module along with the `text2vec-cohere` module, set: ``` ENABLE_MODULES=backup-s3,text2vec-cohere ``` #### S3 configuration (vendor-agnostic) This configuration applies to any S3-compatible backend. | Environment variable | Required | Description | | -------------------------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `BACKUP_S3_BUCKET` | yes | The name of the S3 bucket for all backups. | | `BACKUP_S3_PATH` | no | The root path inside your bucket that all your backups will be copied into and retrieved from.

Optional, defaults to `""` which means that the backups will be stored in the bucket root instead of a sub-folder. | | `BACKUP_S3_ENDPOINT` | no | The S3 endpoint host to use, optionally including a port. Do not include an `http://` or `https://` scheme; TLS is controlled by `BACKUP_S3_USE_SSL`. If the value includes a scheme, the `backup-s3` module fails to initialize and Weaviate does not start.

Optional, defaults to `"s3.amazonaws.com"`. | | `BACKUP_S3_USE_SSL` | no | Whether the connection should be secured with SSL/TLS.

Only the exact value `false` (in any capitalization) disables TLS. Any other value — including `0`, `off`, `no`, or a typo — leaves TLS enabled.

Optional, defaults to `"true"`. | | `BACKUP_SKIP_ACCESS_CHECK` | no | Skip the write-and-delete probe that Weaviate runs against the bucket before each backup. Useful for least-privilege credentials that can write objects but cannot delete them.

Optional, defaults to `false`. See [Skip the storage access check](#skip-the-storage-access-check). | #### S3 authentication For Amazon S3, provide Weaviate with authentication details. You can choose between AWS IAM/ARN-based authentication or access-key authentication. For S3-compatible object stores, use access-key authentication. The `backup-s3` module resolves credentials with its own logic rather than the AWS shared configuration chain. Profiles in `~/.aws/credentials` and the `AWS_PROFILE` variable are not read. Credentials are resolved in this order: 1. An external authentication broker, if `BACKUP_S3_AUTH_PROXY_ENDPOINT` is set. This advanced option takes precedence over everything else, including access keys. 2. The access key and secret access key, if both are set in Weaviate's environment. 3. AWS IAM (an IRSA or EC2 instance role), used only when no access key and secret access key are set. :::callout{intent="warning" title="Access keys shadow an attached IAM role"} If both `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` are present in Weaviate's environment, Weaviate uses them and never contacts IAM — even when an instance role is correctly attached. Stale or leftover keys therefore take priority over the role, silently. Unset both variables to authenticate with IAM. ::: ##### Option 1: With AWS IAM and ARN roles Weaviate uses AWS IAM only when no access key and secret access key are set in its environment. No additional variables are required; the module uses the IRSA or EC2 instance role attached to the workload. ##### Option 2: With access key and secret access key These environment variables are named for AWS, but they also apply to S3-compatible object stores that issue access-key credentials. Both the key and the secret must be set; if only one is present, Weaviate falls back to IAM. | Environment variable | Description | | ----------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `AWS_ACCESS_KEY_ID` | The id of the AWS access key for the desired account. The legacy name `AWS_ACCESS_KEY` is also accepted. | | `AWS_SECRET_ACCESS_KEY` | The secret AWS access key for the desired account. The legacy name `AWS_SECRET_KEY` is also accepted. | | `AWS_REGION` | Optional for Amazon S3. Set it for S3-compatible object stores, some of which require a specific region value. If not provided, the module will try to parse `AWS_DEFAULT_REGION`. | #### S3-compatible endpoints To use an S3-compatible object store, set `BACKUP_S3_ENDPOINT` to the provider's S3 endpoint host, optionally including a port, and authenticate with `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`. Set `AWS_REGION` if your provider requires one. ```bash BACKUP_S3_BUCKET=weaviate-backups BACKUP_S3_ENDPOINT=your-s3-endpoint.example.com # host[:port], no scheme BACKUP_S3_USE_SSL=true AWS_ACCESS_KEY_ID= AWS_SECRET_ACCESS_KEY= # Optional: set if your provider requires a region. # AWS_REGION= ``` ### GCS (Google Cloud Storage) - Works with Google Cloud Storage - Supports multi-node deployments - Recommended for production use To configure `backup-gcs`, you need to enable the module and provide the necessary configuration. #### Enable module Add `backup-gcs` to the `ENABLE_MODULES` environment variable. For example, to enable the S3 module along with the `text2vec-cohere` module, set: ``` ENABLE_MODULES=backup-gcs,text2vec-cohere ``` #### GCS bucket-related variables | Environment variable | Required | Description | | --------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `BACKUP_GCS_BUCKET` | yes | The name of the GCS bucket for all backups. | | `BACKUP_GCS_USE_AUTH` | no | Whether or not credentials will be used for authentication. Defaults to `true`. A case for `false` would be for use with a local GCS emulator. | | `BACKUP_GCS_PATH` | no | The root path inside your bucket that all your backups will be copied into and retrieved from.

Optional, defaults to `""` which means that the backups will be stored in the bucket root instead of a sub-folder. | #### Google Application Default Credentials The `backup-gcs` module follows the Google [Application Default Credentials](https://cloud.google.com/docs/authentication/application-default-credentials) best-practices. This means that credentials can be discovered through the environment, through a local Google Cloud CLI setup, or through an attached service account. This makes it easy to use the same module in different setups. For example, you can use the environment-based approach in production, and the CLI-based approach on your local machine. This way you can easily pull a backup that was created in a remote environment to your local system. This can be helpful in debugging an issue, for example. #### Environment-based Configuration | Environment variable | Example value | Description | | -------------------------------- | ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `GOOGLE_APPLICATION_CREDENTIALS` | `/your/google/credentials.json` | The path to the secret GCP service account or workload identity file. | | `GCP_PROJECT` | `my-gcp-project` | Optional. If you use a service account with `GOOGLE_APPLICATION_CREDENTIALS` the service account will already contain a Google project. You can use this variable to explicitly set a project if you are using user credentials which may have access to more than one project. | ### Azure Storage - Works with Microsoft Azure Storage - Supports multi-node deployments - Recommended for production use To configure `backup-azure`, you need to enable the module and provide the necessary configuration. #### Enable module Add `backup-azure` to the `ENABLE_MODULES` environment variable. For example, to enable the Azure module along with the `text2vec-cohere` module, set: ``` ENABLE_MODULES=backup-azure,text2vec-cohere ``` In addition to enabling the module, you need to configure it using environment variables. There are container-related variables, as well as credential-related variables. #### Azure container-related variables | Environment variable | Required | Description | | ------------------------ | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `BACKUP_AZURE_CONTAINER` | yes | The name of the Azure container for all backups. | | `BACKUP_AZURE_PATH` | no | The root path inside your container that all your backups will be copied into and retrieved from.

Optional, defaults to `""` which means that the backups will be stored in the container root instead of a sub-folder. | #### Azure Credentials There are two different ways to authenticate against Azure with `backup-azure`. You can use either: 1. An Azure Storage connection string, or 2. An Azure Storage account name and key. Both options can be implemented using environment variables as follows: | Environment variable | Required | Description | | --------------------------------- | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `AZURE_STORAGE_CONNECTION_STRING` | yes (\*see note) | A string that includes the authorization information required ([Azure documentation](https://learn.microsoft.com/en-us/azure/storage/common/storage-configure-connection-string)).

This variable is checked and used first before `AZURE_STORAGE_ACCOUNT`. | | `AZURE_STORAGE_ACCOUNT` | yes (\*see note) | The name of your Azure Storage account. | | `AZURE_STORAGE_KEY` | no | An access key for your Azure Storage account.

For anonymous access, specify `""`. | If both of `AZURE_STORAGE_CONNECTION_STRING` and `AZURE_STORAGE_ACCOUNT` are provided, Weaviate will use `AZURE_STORAGE_CONNECTION_STRING` to authenticate. :::callout{intent="note" title="At least one credential option is required"} At least one of `AZURE_STORAGE_CONNECTION_STRING` or `AZURE_STORAGE_ACCOUNT` must be present. ::: #### Azure block size and Concurrency | Environment variable | Required | Default | Description | | -------------------- | -------- | ----------------- | ------------------------------- | | `AZURE_BLOCK_SIZE` | no | `41943040` (40MB) | The Azure block size (in bytes) | | `AZURE_CONCURRENCY` | no | `1` | Upload concurrency | :::callout{intent="note"} You can also use `X-Azure-Block-Size` and `X-Azure-Concurrency` as a client header parameter. If provided, they will override any environment variables. ::: ### Filesystem - Works with the local filesystem and cloud providers - Supports single-node deployments only - Not recommended for production use To configure `backup-filesystem`, you need to enable the module and provide the necessary configuration. #### Enable module Add `backup-filesystem` to the `ENABLE_MODULES` environment variable. For example, to enable the S3 module along with the `text2vec-cohere` module, set: ``` ENABLE_MODULES=backup-filesystem,text2vec-cohere ``` #### Backup Configuration In addition to enabling the module, you need to configure it using environment variables: | Environment variable | Required | Description | | ------------------------ | -------- | -------------------------------------------------------------------------- | | `BACKUP_FILESYSTEM_PATH` | yes | The root path that all your backups will be copied into and retrieved from | ### Other Backup Backends If you are missing your desired backup module, you can open a feature request on the [Weaviate GitHub repository](https://github.com/weaviate/weaviate/issues). We are also open to community contributions for new backup modules. ## API For REST API documentation, see the [Backups section](/weaviate/api/rest#tag/backups). ### Create Backup Once the modules are enabled and the configuration is provided, you can start a backup on any running instance with a single request. You can choose to include or exclude specific collections in the backup. If you do not specify any collections, all collections are included by default. The `include` and `exclude` options are mutually exclusive. You can set none or exactly one of those. #### Wildcard matching :::callout{intent="info" title="Added in `v1.36.0`"} ::: The `include` and `exclude` options support wildcard patterns to match multiple collections at once. Wildcard matching is **case sensitive**. The `*` character matches any sequence of characters. For example, `Article*` matches `Article`, `ArticleV1`, and `ArticleV2`, but not `article` or `Publication`. ##### Available `config` object properties | name | type | required | default | description | | ---------------------------- | ------ | -------- | -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `CPUPercentage` | number | no | `50%` | An optional integer to set the desired CPU core utilization ranging from 1%-80%. | | `ChunkSize` | number | no | - | **Deprecated. This option has no effect.** Weaviate ignores any value sent here, so it neither sets nor caps the chunk size. Chunk sizing is now controlled by the [`BACKUP_CHUNK_TARGET_SIZE`](#chunking-and-file-splitting) environment variable, which replaced it. | | `CompressionLevel` | string | no | `DefaultCompression` | An optional [compression level](#compression-levels) to be used. | | `Path` | string | no | `""` | An optional string to manually set the backup location. If not provided, the backup will be stored in the default location. Introduced in Weaviate `v1.27.2`. | | `incremental_base_backup_id` | string | no | `None` | The ID of a previous backup to use as the base for an [incremental backup](#incremental-backups). Files unchanged since the base backup are stored as references rather than copied. Introduced in Weaviate `v1.37`. | :::code-group{sync="languages"} ```python title="Python" from weaviate.classes.backup import BackupLocation ``` ```typescript title="JavaScript/TypeScript" import weaviate from 'weaviate-client'; const client = await weaviate.connectToLocal(); let result = await client.backup.create({ backupId: 'my-very-first-backup', backend: 'filesystem', // Optional parameters // waitForCompletion: true, // includeCollections: ['Article', 'Publication'], // excludeCollections: ['Author'], }) console.log(result.status); ``` ```go title="Go" createResponse, err := client.Backup().Creator(). WithIncludeClassNames("Article", "Publication"). WithBackend(backup.BACKEND_FILESYSTEM). WithBackupID("my-very-first-backup"). WithWaitForCompletion(true). Do(context.Background()) ``` ```java title="Java" var createResult = client.backup .create(backupId, backend, backup -> backup.includeCollections("Article", "Publication")) .waitForCompletion(client); // Replicates wait_for_completion=True System.out.println(createResult); ``` ```bash title="curl" curl \ -X POST \ -H "Content-Type: application/json" \ -d '{ "id": "my-very-first-backup", "include": ["Article", "Publication"] }' \ http://localhost:8080/v1/backups/filesystem ``` ::: While you are waiting for a backup to complete, [Weaviate stays available](#read--write-requests-while-a-backup-is-running). #### Compression levels :::callout{intent="info" title="`zstd` compression availability"} `zstd` compression is only available in Weaviate in `v1.35.0`, `v1.34.1`, `v1.33.6` and `v1.32.18` or higher. ::: Where [`zstd` compression](https://github.com/facebook/zstd) is available, choose one of: `ZstdDefaultCompression`, `ZstdBestSpeed` and `ZstdBestCompression`. Otherwise, choose one of the standard [gzip compression](https://pkg.go.dev/compress/gzip#pkg-constants) options: `DefaultCompression`, `BestSpeed` and `BestCompression`. You can also explicitly disable compression by setting the level to `NoCompression`. #### Asynchronous Status Checking All client implementations have a "wait for completion" option which will poll the backup status in the background and only return once the backup has completed (successfully or unsuccessfully). If you set the "wait for completion" option to false, you can also check the status yourself using the Backup Creation Status API. ```js GET /v1/backups/{backend}/{backup_id} ``` #### Parameters ##### URL Parameters | Name | Type | Required | Description | | ----------- | ------ | -------- | -------------------------------------------------------------------------------------------------------------- | | `backend` | string | yes | The name of the backup provider module without the `backup-` prefix, for example `s3`, `gcs`, or `filesystem`. | | `backup_id` | string | yes | The user-provided backup identifier that was used when sending the request to create the backup. | The response contains a `"status"` field. If the status is `SUCCESS`, the backup is complete. If the status is `FAILED`, an additional error is provided. :::code-group{sync="languages"} ```python title="Python" from weaviate.classes.backup import BackupLocation ``` ```typescript title="JavaScript/TypeScript" let backupStatus = await client.backup.getCreateStatus({ backupId: 'my-very-first-backup', backend: 'filesystem', }) console.log(backupStatus); ``` ```go title="Go" statusCreateResponse, err := client.Backup().CreateStatusGetter(). WithBackend(backup.BACKEND_FILESYSTEM). WithBackupID("my-very-first-backup"). Do(context.Background()) ``` ```java title="Java" Optional createStatus = client.backup.getCreateStatus(backupId, backend); System.out.println(createStatus.orElse(null)); ``` ```bash title="curl" curl http://localhost:8080/v1/backups/filesystem/my-very-first-backup ``` ::: ### Incremental Backups :::callout{intent="info" title="Added in `v1.37`"} ::: Incremental backups reduce backup size and duration by only storing data that has changed since a previous backup. Instead of copying all files again, an incremental backup references unchanged files from a base backup. This can result in dramatically smaller backups and much faster backup times. #### How it works When creating a backup, Weaviate packs a shard's files into chunks. During an incremental backup, Weaviate compares each file against the base backup. Files that haven't changed are stored as pointers to the base backup rather than being copied again. On restore, Weaviate automatically fetches the referenced files from the base backup. The base backup can itself be an incremental backup, so you can build a [chain of incremental backups](#chained-incremental-backups) that ends at a full backup. Weaviate walks the whole chain to find unchanged files, so every backup in the chain must remain available. Only a file large enough to get a chunk of its own can be referenced individually, so the way Weaviate groups files into chunks determines how much an incremental backup can reuse. For how Weaviate decides which files get their own chunk, and the environment variables that control chunking, see [Chunking and file splitting](#chunking-and-file-splitting). #### Create a full (base) backup First, create a regular backup that will serve as the base: ```python result = client.backup.create( backup_id="base-backup", backend="filesystem", include_collections=["Article", "Publication"], wait_for_completion=True, ) print(result) ``` #### Create an incremental backup To create an incremental backup, pass the `incremental_base_backup_id` parameter with the ID of the base backup: ```python {6} result = client.backup.create( backup_id="incremental-backup-1", backend="filesystem", include_collections=["Article", "Publication"], wait_for_completion=True, incremental_base_backup_id="base-backup", ) print(result) ``` #### Chained incremental backups You can chain incremental backups by using a previous incremental backup as the base. Weaviate will walk the chain back to the original full backup to find unchanged files. ```python {6} result = client.backup.create( backup_id="incremental-backup-2", backend="filesystem", include_collections=["Article", "Publication"], wait_for_completion=True, incremental_base_backup_id="incremental-backup-1", ) print(result) ``` #### Restore an incremental backup Restoring an incremental backup works the same as restoring any other backup. Weaviate automatically resolves the chain and fetches files from previous backups as needed. ```python result = client.backup.restore( backup_id="incremental-backup-2", backend="filesystem", wait_for_completion=True, ) print(result) ``` :::callout{intent="warning" title="Keep base backups available"} Base backups (and any intermediate incremental backups in a chain) must remain available for as long as you need to restore from any incremental backup that depends on them. ::: ### List Backups You can list the backups that are stored in a backup backend. The listing reports the status of each backup, the collections it holds, its size, and, for an incremental backup, the backup it was built on. This is how you inspect an existing [chain of incremental backups](#chained-incremental-backups) and confirm that every backup the chain depends on is still present. ```js GET /v1/backups/{backend} ``` #### Parameters ##### URL Parameters | Name | Type | Required | Description | | --------- | ------ | -------- | -------------------------------------------------------------------------------------------------------------- | | `backend` | string | yes | The name of the backup provider module without the `backup-` prefix, for example `s3`, `gcs`, or `filesystem`. | ##### Query Parameters | Name | Type | Required | Default | Description | | ------- | ------ | -------- | ------- | ---------------------------------------------------------------------------------------------- | | `order` | string | no | `desc` | Sort the returned backups by start time, either `asc` (oldest first) or `desc` (newest first). | ##### Response fields | name | type | description | | ---------------------------- | ------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `id` | string | The identifier of the backup. | | `classes` | array | The collections the backup contains. | | `status` | string | The status of the backup, such as `SUCCESS` or `FAILED`. | | `startedAt` | date | When the backup started. | | `completedAt` | date | When the backup finished, successfully or not. | | `size` | number | The size of the backup in GiB, measured before compression. | | `incremental_base_backup_id` | string | The identifier of the backup that this [incremental backup](#incremental-backups) was built on. Empty when the backup is a full backup. Only returned to root users, see below. Introduced in Weaviate `v1.37.6`. | The listing only includes backups whose collections you are authorized to read. Backups you have no read access to are left out rather than causing an error. :::callout{intent="warning" title="One name, two different things"} `incremental_base_backup_id` appears on both sides of the backup API, and the two are not interchangeable: - On the **create** side it is an input that you supply. It names the backup that the new backup should build on, as described under [Create an incremental backup](#create-an-incremental-backup). - On the **list** side it is a read-only output. It reports the backup that an already-created backup was built on, which is what lets you walk a chain back to its full base backup. You cannot choose a base backup through the list API. The value it returns reflects a decision that was made when that backup was created. ::: :::callout{intent="info" title="The base backup identifier is only returned to root users"} The list-side `incremental_base_backup_id` is treated as sensitive. Weaviate only fills it in when it has confirmed that the caller is a [root user](../authorization-and-authentication/deploy-configuration-configuring-rbac.md). Any other caller receives an empty value for this field, even one with full backup permissions and even when the backup really is incremental. If you are auditing a backup chain and every entry comes back empty, check the identity you are connecting with before concluding that no incremental backups exist. ::: ```python backups = client.backup.list_backups( backend="filesystem", sort_by_starting_time_asc=True, ) for backup in backups: print(backup.backup_id, backup.status, backup.incremental_base_backup_id) ``` :::accordion{title="Code output"} ``` base-backup BackupStatus.SUCCESS None incremental-backup-1 BackupStatus.SUCCESS base-backup incremental-backup-2 BackupStatus.SUCCESS incremental-backup-1 ``` ::: ### Cancel Backup An ongoing backup can be cancelled at any time. The backup process will be stopped, and the backup will be marked as `CANCELLED`. :::code-group{sync="languages"} ```python title="Python" from weaviate.classes.backup import BackupLocation ``` ```typescript title="JavaScript/TypeScript" let cancelStatus = await client.backup.cancel({ backupId: 'my-very-first-backup', backend: 'filesystem' }) console.log(cancelStatus); ``` ```java title="Java" // Note: The cancel() method is called on the Backup object backupToCancel.cancel(client); ``` ```go title="Go" err = client.Backup().Canceler(). WithBackend(backup.BACKEND_FILESYSTEM). WithBackupID("some-unwanted-backup"). Do(context.Background()) ``` ::: This operation is particularly useful if you have started a backup by accident, or if you would like to stop a backup that is taking too long. ### Restore Backup You can restore any backup to any machine as long as the name and number of nodes between source and target are identical. The backup does not need to be created on the same instance. Once a backup backend is configured, you can restore a backup with a single request. As with backup creation, the `include` and `exclude` options are mutually exclusive. You can set none or exactly one of those. In a restore operation, `include` and `exclude` are relative to the collections contained in the backup. The restore process is not aware of collections that existed on the source machine if they were not part of the backup. Note that a restore fails if any of the collections already exist on this instance. :::callout{intent="warning" title="Restoring backups from `v1.23.12` and older"} If you are running Weaviate `v1.23.12` or older, first **[update Weaviate](../deploy/migration.md) to version 1.23.13** or higher before restoring a backup. Versions prior to `v1.23.13` had a bug that could lead to data not being stored correctly from a backup of your data. ::: ##### Available `config` object properties | name | type | required | default | description | | --------------- | ------ | ------------------------------------ | ------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `cpuPercentage` | number | no | `50%` | An optional integer to set the desired CPU core utilization ranging from 1%-80%. | | `Path` | string | Required if created at a custom path | `""` | An optional string to manually set the backup location. If not provided, the backup will be restored from the default location. Introduced in Weaviate `v1.27.2`. | | `rolesOptions` | string | no | `"noRestore"` | An optional string to manually set if RBAC roles will be backed up and restored. Can be either `"noRestore"` for not backing up roles and permissions or `"all"` to include all of them. Introduced in Weaviate `v1.32.0`. | | `usersOptions` | string | no | `"noRestore"` | An optional string to manually set if RBAC users will be backed up. Can be either `"noRestore"` for not backing up users or `"all"` to include all of them. Introduced in Weaviate `v1.32.0`. | :::code-group{sync="languages"} ```python title="Python" from weaviate.classes.backup import BackupLocation ``` ```typescript title="JavaScript/TypeScript" import weaviate from 'weaviate-client'; const client = await weaviate.connectToLocal(); let result = await client.backup.restore({ backupId: 'my-very-first-backup', backend: 'filesystem', // Optional parameters // waitForCompletion: true, // includeCollections: ['Article', 'Publication'], // excludeCollections: ['Author'], }) console.log(result.status); ``` ```go title="Go" restoreResponse, err := client.Backup().Restorer(). WithExcludeClassNames("Article"). WithBackend(backup.BACKEND_FILESYSTEM). WithBackupID("my-very-first-backup"). WithWaitForCompletion(true). // The following two methods can be used to restore roles and users separately // .WithRBACRoles(rbac.RBACAll). // .WithRBACUsers(rbac.UserAll). WithRBACAndUsers(). // This is a convenience method to restore both roles and users Do(context.Background()) ``` ```java title="Java" var restoreResult = client.backup.restore(backupId, backend, restore -> restore.excludeCollections("Article") // Note: roles_restore and users_restore syntax not provided ).waitForCompletion(client); // Replicates wait_for_completion=True System.out.println(restoreResult); ``` ```bash title="curl" curl \ -X POST \ -H "Content-Type: application/json" \ -d '{ "id": "my-very-first-backup", "exclude": ["Article"] }' \ http://localhost:8080/v1/backups/filesystem/my-very-first-backup/restore ``` ::: #### Asynchronous Status Checking All client implementations have a "wait for completion" option which will poll the restore status in the background and only return once the restore has completed (successfully or unsuccessfully). If you set the "wait for completion" option to false, you can also check the status yourself using the Backup Restore Status API. The response contains a `"status"` field. If the status is `SUCCESS`, the restore is complete. If the status is `FAILED`, an additional error is provided. :::code-group{sync="languages"} ```python title="Python" from weaviate.classes.backup import BackupLocation ``` ```typescript title="JavaScript/TypeScript" let restoreStatus = await client.backup.getRestoreStatus({ backupId: 'my-very-first-backup', backend: 'filesystem', }) console.log(restoreStatus); ``` ```go title="Go" statusRestoreResponse, err := client.Backup().RestoreStatusGetter(). WithBackend(backup.BACKEND_FILESYSTEM). WithBackupID("my-very-first-backup"). Do(context.Background()) ``` ```java title="Java" Optional restoreStatus = client.backup.getRestoreStatus(backupId, backend); System.out.println(restoreStatus.orElse(null)); ``` ```bash title="curl" curl http://localhost:8080/v1/backups/filesystem/my-very-first-backup/restore ``` ::: ### Cancel Restore :::callout{intent="info" title="Added in `v1.36`"} ::: An ongoing restore operation can be cancelled before it reaches the `FINALIZING` phase. Cancellation is not possible once schema changes are being applied via Raft, as this could leave the cluster in an inconsistent state. A restore goes through the following phases: | Status | Meaning | Cancellable | | -------------- | ------------------------------------------- | ----------- | | `STARTED` | Restore initiated, preparing to stage files | Yes | | `TRANSFERRING` | Files being staged from object storage | Yes | | `TRANSFERRED` | File staging complete on all nodes | Yes | | `FINALIZING` | Schema changes in progress (Raft commits) | No | | `SUCCESS` | Restore complete | N/A | | `CANCELLING` | Cancellation claimed, aborting nodes | N/A | | `CANCELED` | Restore was cancelled | N/A | :::code-group{sync="languages"} ```python title="Python" from weaviate.classes.backup import BackupLocation ``` ```typescript title="JavaScript/TypeScript" let cancelRestoreStatus = await client.backup.cancel({ backupId: 'my-very-first-backup', backend: 'filesystem', operation: 'restore', }) console.log(cancelRestoreStatus); ``` ```java title="Java" client.backup.cancelRestore(backupId, backend); ``` ```bash title="curl" curl -X DELETE http://localhost:8080/v1/backups/filesystem/my-very-first-backup/restore ``` ::: ## Kubernetes configuration When running Weaviate on Kubernetes, you can configure the backup provider using Helm chart values. These values are available under the `backups` key in the `values.yaml` file. Refer to the inline documentation in the `values.yaml` file for more information. ## Technical Considerations ### Read & Write requests while a backup is running The backup process is designed to be minimally invasive to a running setup. Even on very large setups, where terabytes of data need to be copied, Weaviate stays available during backup. It even accepts write requests while a backup process is running. This sections explains how backups work under the hood and why Weaviate can safely accept writes while a backup is copied. Weaviate uses a custom [LSM Store](../concepts/storage.md#object-and-inverted-index-store) for its object store and inverted index. LSM stores are a hybrid of immutable disk segments and an in-memory structure called a memtable that accepts all writes (including updates and deletes). Most of the time, files on disk are immutable, there are only three situations where files are changed: 1. Anytime a memtable is flushed. This creates a new segment. Existing segments are not changed. 2. Any write into the memtable is also written into a Write-Ahead-Log (WAL). The WAL is only needed for disaster-recovery. Once a segment has been orderly flushed, the WAL can be discarded. 3. There is an async background process called Compaction that optimizes existing segments. It can merge two small segments into a single larger segment and remove redundant data as part of the process. Weaviate's Backup implementation makes use of the above properties in the following ways: 1. Weaviate first flushes all active memtables to disk. This process takes in the 10s or 100s of milliseconds. Any pending write requests simply waits for a new memtable to be created without any failing requests or substantial delays. 2. Now that the memtables are flushed, there is a guarantee: All data that should be part of the backup is present in the existing disk segments. Any data that will be imported after the backup request ends up in new disk segments. The backup references a list of immutable files. 3. To prevent a compaction process from changing the files on disk while they are being copied, compactions are temporarily paused until all files have been copied. They are automatically resumed right after. This way the backup process can guarantee that the files that are transferred to the remote backend are immutable (and thus safe to copy) even with new writes coming in. Even if it takes minutes or hours to backup a very large setup, Weaviate stays available without any user impact while the backup process is running. It is not just safe - but even recommended - to create backups on live production instances while they are serving user requests. ### Async nature of the Backup API The backup API is built in a way that no long-running network requests are required. The request to create a new backup returns immediately. It does some basic validation, then returns to the user. The backup is now in status `STARTED`. To get the status of a running backup you can poll the [status endpoint](#asynchronous-status-checking). This makes the backup itself resilient to network or client failures. If you would like your application to wait for the background backup process to complete, you can use the "wait for completion" feature that is present in all language clients. The clients will poll the status endpoint in the background and block until the status is either `SUCCESS` or `FAILED`. This makes it easy to write simple synchronous backup scripts, even with the async nature of the API. ### Chunking and file splitting :::callout{intent="info" title="Added in `v1.36.0`"} Chunking and `BACKUP_CHUNK_TARGET_SIZE` were backported to `v1.33.14`, `v1.34.11`, and `v1.35.4`. The other variables were added later. See the table below. ::: On every backup, full or incremental, Weaviate packs each [shard's](../concepts/storage.md#logical-storage-units-indexes-shards-stores) files into chunks: - Each of the shard's biggest files gets a **chunk of its own**. - A file larger than `BACKUP_SPLIT_FILE_SIZE` is **split into parts**, and each part gets a chunk of its own that holds nothing else. - The remaining smaller files are packed together into **shared chunks** of roughly `BACKUP_CHUNK_TARGET_SIZE`. Chunking matters because it determines how much a later [incremental backup](#incremental-backups) can reuse: - Only a file with a chunk of its own can be referenced from the base backup instead of copied again, and only if Weaviate also treats the file as immutable. - A big file that Weaviate keeps rewriting has its own chunk but is still re-uploaded on every backup. - Shared chunks are re-uploaded on every incremental backup, even if nothing in them changed. A file gets its own chunk when it reaches the **qualifying size**: the larger of `BACKUP_MIN_CHUNK_SIZE` and the size of the shard's Nth largest file, where N is `BACKUP_MAX_INDIVIDUAL_FILES`, reduced on an incremental backup by the number of files already reused from the base backup. This budget is shared across a whole backup chain rather than renewed for each backup in it. Because the larger value wins, `BACKUP_MIN_CHUNK_SIZE` is only a floor. Lowering it never reduces how many files qualify. Which knob qualifies more files depends on the shard. With N or more files above the floor, raise `BACKUP_MAX_INDIVIDUAL_FILES`. With fewer, lower `BACKUP_MIN_CHUNK_SIZE`. :::accordion{title="Diagram: how a file becomes a chunk"} ```mermaid flowchart TD File["Shard file"] Qualifies{"Reaches the
qualifying size?"} OverSplit{"Larger than
the split size?"} Own["Own chunk"] Parts["Split into parts,
one chunk per part"] Shared["Packed into a
shared chunk"] Immutable{"Immutable
file?"} Reusable["Reusable by later
incremental backups"] Reuploaded["Re-uploaded on
every backup"] File --> Qualifies Qualifies -->|"No"| Shared Qualifies -->|"Yes"| OverSplit OverSplit -->|"No"| Own OverSplit -->|"Yes"| Parts Own --> Immutable Parts --> Immutable Immutable -->|"Yes"| Reusable Immutable -->|"No"| Reuploaded Shared --> Reuploaded style File fill:#ffffff,stroke:#B9C8DF,color:#130C49 style Qualifies fill:#ffffff,stroke:#B9C8DF,color:#130C49 style OverSplit fill:#ffffff,stroke:#B9C8DF,color:#130C49 style Own fill:#ffffff,stroke:#B9C8DF,color:#130C49 style Parts fill:#ffffff,stroke:#B9C8DF,color:#130C49 style Shared fill:#ffffff,stroke:#B9C8DF,color:#130C49 style Immutable fill:#ffffff,stroke:#B9C8DF,color:#130C49 style Reusable fill:#ffffff,stroke:#B9C8DF,color:#130C49 style Reuploaded fill:#ffffff,stroke:#B9C8DF,color:#130C49 ``` ::: The parts of a split file are sized by `BACKUP_SPLIT_FILE_SIZE`, not by the chunk target: Weaviate divides the file into roughly equal parts that are each at least half and at most the full split size. At the defaults, a chunk carrying a split part is therefore up to `50GiB`. That is far larger than the `10MiB` chunk target, not smaller. The three size variables accept a plain number of bytes or a number with a case-sensitive unit suffix (`B`, `KB`, `MB`, `GB`, `TB`, `KiB`, `MiB`, `GiB`, `TiB`), for example `4MiB`. Decimal and binary suffixes are distinct: `MB` is 1,000,000 bytes while `MiB` is 1,048,576 bytes. They also accept `unlimited` or `nolimit`, which is how you disable file splitting through `BACKUP_SPLIT_FILE_SIZE`. All three are read at startup, so changing them requires a restart. | Environment variable | Required | Description | | ----------------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `BACKUP_MIN_CHUNK_SIZE` | no | A floor on the qualifying size a file must reach to get a chunk of its own. Defaults to `1MiB`.

Added in `v1.36.3` (backported to `v1.34.18`, `v1.35.13`). | | `BACKUP_CHUNK_TARGET_SIZE` | no | The size Weaviate aims for when packing several smaller files into a shared chunk. Defaults to `10MiB`.

Added in `v1.36.0` (backported to `v1.33.14`, `v1.34.11`, `v1.35.4`). | | `BACKUP_SPLIT_FILE_SIZE` | no | The size above which a file is split into parts. Set it to `unlimited` to disable splitting. Defaults to `50GiB`.

Added in `v1.36.5` (backported to `v1.35.15`). | | `BACKUP_MAX_INDIVIDUAL_FILES` | no | How many of a shard's biggest files Weaviate aims to give a chunk of their own. This is a target rather than a hard cap. The value is a count, not a size, and it must be greater than `0`. Defaults to `100`. Settable without a restart through the `backup_max_individual_files` [runtime configuration](../database-configuration/runtime-config.md) key.

Added in `v1.37.14` and `v1.38.7`. | :::callout{intent="note" title="How Weaviate adjusts these values"} These settings are lower bounds rather than exact values: - `BACKUP_CHUNK_TARGET_SIZE` and `BACKUP_SPLIT_FILE_SIZE` are raised to the qualifying size if you set them lower. - If a shard holds fewer files than the `BACKUP_MAX_INDIVIDUAL_FILES` budget (after subtracting files already reused on an incremental backup), the qualifying size falls back to the size of the shard's smallest file, still raised to `BACKUP_MIN_CHUNK_SIZE` if that is larger. ::: ### Skip the storage access check When a cloud backup backend (`backup-s3`, `backup-gcs`, or `backup-azure`) initializes, Weaviate verifies that the configured credentials can write to and delete from the target bucket. It does this by writing a temporary `access-check` object and then removing it. This probe fails on immutable (write-once / WORM) buckets, or with least-privilege credentials that are not permitted to delete objects. Set `BACKUP_SKIP_ACCESS_CHECK=true` to skip this probe. The variable applies to all cloud backup backends, defaults to `false`, and is applied at startup (a restart is required to change it). | Environment variable | Required | Description | | -------------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `BACKUP_SKIP_ACCESS_CHECK` | no | Skip the write-and-delete access check performed when a backup backend initializes. Set to `true` for immutable (write-once / WORM) buckets or least-privilege credentials that cannot delete objects. Defaults to `false`.

Added in `v1.37.8`. | ## Other Use cases ### Migrating to another environment The flexibility around backup providers opens up new use cases. Besides using the backup & restore feature for disaster recovery, you can also use it for duplicating environments or migrating between clusters. For example, consider the following situation: You would like to do a load test on production data. If you would do the load test in production it might affect users. An easy way to get meaningful results without affecting uses it to duplicate your entire environment. Once the new production-like "loadtest" environment is up, create a backup from your production environment and restore it into your "loadtest" environment. This even works if the production environment is running on a completely different cloud provider than the new environment. ## Troubleshooting and notes - Single node backup is available starting in Weaviate `v1.15`. Multi-node backups is available starting in `v1.16`. - In some cases, backups can take a long time, or get "stuck", causing Weaviate to be unresponsive. If this happens, you can [cancel the backup](#cancel-backup) and try again. - If a backup module is misconfigured, such as having an invalid backup path, it can cause Weaviate to not start. Review the system logs for any errors. - RBAC roles and users are not restored by default. You need to enable them manually through the configuration properties when [restoring a backup](#restore-backup). ## Related pages - [References: REST API: Backups](/weaviate/api/rest#tag/backups) ## Questions and feedback Have a question or feedback? Here's how to reach us. ::::card-grid :::card{title="Community Forum" href="https://forum.weaviate.io/c/support" icon="messages-square"} Ask questions and connect with other developers on our **Community forum**. ::: :::card{title="Support" href="/guides/support-overview" icon="life-buoy"} Weaviate Cloud user or customer? Find the right channel on the **Support page**. ::: :::: ## Related pages - [Agents](./agents-index.md) - [AI-assisted Weaviate code generation](./ai-assisted-vibe-coding-index.md) - [APIs](./apis-index.md) - [Authorization and authentication](./authorization-and-authentication-index.md) - [Benchmarks](./benchmarks-index.md) - [Best practices](./best-practices-index.md) - [Client libraries](./clients-index.md) - [Client Libraries / SDKs](./client-libraries-index.md) - [Cloud](./cloud-index.md) - [Cloud account management](./cloud-account-management-index.md) # Agent Instructions This portal answers questions programmatically. To receive a synthesized, source-cited answer instead of crawling page by page, append the `?ask=` query parameter to any page URL on this site: /guides/quickstart?ask=how+do+I+authenticate Optional parameters: - `&goal=` steers the answer toward your objective (e.g. `&goal=write+a+python+client`). - `&version=