AWS: Hardening your EKS deployment

You've got a Weaviate deployment running on EKS. Awesome! Now it's time to make it production-ready and secure.

While Weaviate is a powerful vector database, like any self-hosted service, it needs proper security hardening. Your first deployment focused on getting things running; for production, we need to tighten things up.

This guide walks through practical steps to secure your deployment.

Control who can access Weaviate and what actions they can perform.

Anonymous access is convenient for testing, but in production it's like leaving your front door wide open.

Action: Disable anonymous access immediately.

📚 How to disable anonymous access

Use strong authentication with properly managed credentials.

Best practices:

• Generate strong, random keys (32+ characters minimum)
• Store keys in Kubernetes secrets
• Rotate keys regularly
• Map each key to a specific user or service for accountability

📚 API Key authentication setup

If you're using an enterprise identity provider (Okta, Azure AD, Auth0, etc.), integrate with it for single sign-on.

Benefits:

• Centralized user management
• Single sign-on experience

📚 OIDC integration guide

Choose the authorization model that fits your needs:

Best for teams needing granular access control.

Features:

• Custom role creation
• Collection-level access control
• Perfect for teams with different responsibilities

📚 RBAC configuration

Simpler model for small teams or basic setups.

Features:

• admin users: full access
• read-only users: query access only

📚 Admin list setup

Instead of static AWS credentials, use IAM Roles for Service Accounts (IRSA) for temporary, secure credentials.

Benefits:

• No static credentials to manage
• Automatic credential rotation
• Fine-grained IAM permissions

Set up CloudTrail to log all IAM API calls and configure alerts for suspicious activity (repeated access denied errors, unusual access patterns).

This is a must-have, configure TLS properly.

Requirements:

• Use cert-manager for automatic certificate management
• Configure HTTP-to-HTTPS redirects
• Ensure data never travels in plaintext

Rule: Your data should never travel unencrypted.

Deploy all worker nodes in private subnets only.

Architecture:

• Worker nodes: private subnets only
• Outbound internet: NAT gateways
• AWS services: VPC endpoints
• Never expose Weaviate directly to the internet

Lock down pod-to-pod communication with Kubernetes network policies.

Approach:

1. Start with default-deny policy
2. Explicitly allow only required traffic
3. Apply principle of least privilege to security groups

Enable the "restricted" pod security standard on your Weaviate namespace.

Prevents:

• Running as root
• Privileged containers
• Host namespace access
• Other dangerous configurations

Secure your container runtime.

Configuration checklist:

• ✅ Run as non-root user
• ✅ Drop unnecessary capabilities
• ✅ Prevent privilege escalation
• ✅ Enable read-only filesystems where possible

Prevent resource starvation and ensure predictable performance by setting CPU and memory requests/limits.

Create a dedicated service account for Weaviate with minimal RBAC permissions.

If budget permits, use dedicated node groups for Weaviate workloads.

Benefits:

• Better resource isolation
• Predictable performance
• Easier capacity planning

Protect data confidentiality and availability through encryption and backups.

Configuration:

• Use encrypted EBS volumes
• Use customer-managed KMS keys
• Enable automatic key rotation
• Restrict key access with IAM policies

Set up automated, encrypted backups to S3.

Checklist:

• ✅ Enable server-side encryption
• ✅ Enable bucket versioning
• ✅ Set up lifecycle policies for cost management
• ✅ Block all public access

📚 S3 backup configuration

Turn on Prometheus metrics for system visibility.

📚 Monitoring setup guide

Set up alerts for issues that matter:

• 🚨 Pods going down
• 🚨 High error rates
• 🚨 Resource exhaustion
• 🚨 Backup failures

Import or create Grafana dashboards to visualize key metrics.

Track:

• Memory usage
• Query latency
• Error rates
• Disk usage

📚 Sample dashboards

Turn on EKS control plane logging, especially audit logs.

Why:

• Required for compliance
• Essential for incident investigation
• Security team requirements

• Authentication
• Authorization
• OIDC Configuration
• RBAC
• Backups
• Monitoring
• AWS network security - Best practices

Have a question or feedback? Here's how to reach us.