Usage Module
What it does:
Section titled “What it does:”- Periodically collecting usage data.
- Uploading JSON reports to either S3 or GCS.
- Supports runtime configuration overrides.
- Includes metrics and logging.
- Verifies storage permissions before uploading.
Configuration
Section titled “Configuration”Example configuration with backup and usage
Section titled “Example configuration with backup and usage”
# environment variables
ENABLE_MODULES="backup-gcs,usage-gcs"
BACKUP_GCS_BUCKET=weaviate-backups
USAGE_GCS_BUCKET=weaviate-usage
USAGE_GCS_PREFIX=billing-usage
TRACK_VECTOR_DIMENSIONS=true # won't be needed from 1.32.1
RUNTIME_OVERRIDES_ENABLED=true
RUNTIME_OVERRIDES_PATH="${PWD}/tools/dev/config.runtime-overrides.yaml"
RUNTIME_OVERRIDES_LOAD_INTERVAL=30s
# in tools/dev/config.runtime-overrides.yaml
usage_scrape_interval: 1h
usage_shard_jitter_interval: 100ms #(optional)
usage_gcs_bucket: weaviate-usage
usage_gcs_prefix: billing
Environment variables
Section titled “Environment variables”# Enable the usage modules (required)
ENABLE_MODULES=usage-gcs # if you want gcs
ENABLE_MODULES=usage-s3 # if you want s3
ENABLE_MODULES=usage-s3,usage-gcs # or both
Runtime overrides
Section titled “Runtime overrides”RUNTIME_OVERRIDES_ENABLED=true
RUNTIME_OVERRIDES_PATH="${PWD}/tools/dev/config.runtime-overrides.yaml"
RUNTIME_OVERRIDES_LOAD_INTERVAL=30s
# Required: Enable vector dimension tracking metrics
TRACK_VECTOR_DIMENSIONS=true # won't be needed from 1.32.1
# Collection interval (default: 1h)
USAGE_SCRAPE_INTERVAL=2h
# (optional) Shard loop jitter (default: 100ms)
USAGE_SHARD_JITTER_INTERVAL=50ms
# (optional) Policy version (default: 2025-06-01)
USAGE_POLICY_VERSION=2025-06-01
# (optional) verify the bucket permission on start (default:false)
USAGE_VERIFY_PERMISSIONS=trueExample runtime-overrides.yaml
Section titled “Example runtime-overrides.yaml”usage_scrape_interval: 1s
usage_shard_jitter_interval: 100ms # (optional)
usage_verify_permissions: true/false # (optional)
# usage-gcs config
usage_gcs_bucket: weaviate-usage
usage_gcs_prefix: billing
# usage-s3 config
usage_s3_bucket: weaviate-usage
usage_s3_prefix: billingAWS S3 variables
Section titled “AWS S3 variables”# Required: S3 bucket name
USAGE_S3_BUCKET=my-weaviate-usage-bucket
# Optional: Object prefix (default: empty)
USAGE_S3_PREFIX=usage-reports
GCP GCS variables
Section titled “GCP GCS variables”# Required: GCS bucket name
USAGE_GCS_BUCKET=my-weaviate-usage-bucket
# Optional: Object prefix (default: empty)
USAGE_GCS_PREFIX=usage-reportsMonitoring
Section titled “Monitoring”The modules provide Prometheus metrics:
weaviate_usage_{gcs|s3}_operations_total: Total number of operations for module labels (operation:collect/upload, status: success, error).weaviate_usage_{gcs|s3}_operation_latency_seconds: Latency of usage operations in seconds labels (operation:collect/upload).weaviate_usage_{gcs|s3}_resource_count: Number of resources tracked by module, labels (resource_type:collections/shards/backups).weaviate_usage_{gcs|s3}_uploaded_file_size_bytes: Size of the uploaded usage file in bytes.
Debug logs
Section titled “Debug logs”See detailed module activity by enabling debugging.
LOG_LEVEL=debugTesting
Section titled “Testing”If using Minio, set the environment variable AWS_ENDPOINT, AWS_REGION
AWS_REGION=us-east-1
AWS_ENDPOINT=http://localhost:9000AWS_REGION=us-east-1
AWS_ENDPOINT=minio.weaviate.svc.cluster.local:9000 Further resources
Section titled “Further resources”Questions and feedback
Section titled “Questions and feedback”Have a question or feedback? Here's how to reach us.