Cluster resource messages

Messages on this page mean that a Weaviate node asked the operating system for a resource and was refused. The data is intact, but the node cannot open the files it needs until the limit is raised or the load is reduced. If your message is not here, the message index lists the other groups.

Not enough memory mappings


Ids	`core-mem001`
Raised by	Weaviate Database
Severity	error
Impact	A shard refused to open, so its collection or tenant is unavailable for reads and writes. Nothing is corrupted.
Fix	Raise vm.max_map_count on the host and restart the node.

What you see

In the node's logs, an error ending in not enough memory mappings:

text

memory pressure: cannot init shard: not enough memory mappings

In the response your client gets when the failure happened inside a request you made (REST shown; gRPC and GraphQL carry the same suffix):

JSON

{"error":[{"message":"updating db: TYPE_UPDATE_TENANT: memory pressure: cannot init shard: not enough memory mappings (see https://docs.weaviate.io/e/core-mem001)"}]}

Which log entry carries it depends on the operation that hit the limit. <tenant> and <error> stand for the tenant name and the appended error text:

Log message	`action` field	Level
`failed to load shard, loading the rest anyway: <error>`	`load_shard`	error
`loading shard "<tenant>" failed: <error>`	`tenant_activation_lazy_load_shard`	error
`failed to reload local index`	(none)	error
`error updating tenants`	`update_tenants`	error
`error updating tenants: <error>`	`update_tenants_process`	error
`apply command`	(none)	error
`failure while loading shard: <error>` (while a replica is moved or copied)	`replication_engine`	error
`drop-vector: load lazy shard: <error>` (while a vector index is dropped)	(none)	warning

The exact line varies by operation and version, so search the logs for not enough memory mappings: it appears in the message or in a sibling error field, depending on the entry. On versions that include this feature, every entry carrying the phrase also carries a docs_url field pointing at this page.

The failure often shows up next to replication errors such as broadcast: cannot reach enough replicas, because the shard those requests needed is the one that did not load.

Why it happens

Every shard memory-maps several files, and the Linux kernel caps how many mappings one process may hold: vm.max_map_count. Weaviate reads that cap at startup, budgets 70% of it to leave room for other processes, and refuses to open a shard that would take it past the budget. Refusing early is deliberate: exhausting the kernel limit outright fails unpredictably somewhere else in the process.

The usual trigger is scale, not a bug. Distribution defaults were chosen for ordinary workloads, and a node holding thousands of active tenants or collections passes them.

How to fix it

The limit belongs to the host kernel, so where you set it depends on how you run Weaviate.

Read the current limit on the node that logged the error. Most distributions ship 65530:
Bash
```
sysctl vm.max_map_count
```
Raise it. The value is a count, not an amount of memory, so a high value costs nothing on its own. 262144 is the floor other memory-mapped databases ask for; a node holding thousands of active tenants needs a value in the millions, and 4194304 leaves room to grow:
Bash
```
sysctl -w vm.max_map_count=4194304
```

Make it survive a reboot:

Bash

echo "vm.max_map_count=4194304" >> /etc/sysctl.conf

Restart the affected node so Weaviate reads the new limit.

On Linux, containers share the host kernel, so the limit is the host's to raise: follow the Linux host steps on the machine that runs Docker, then restart the container. Setting it per container does not work — docker run --sysctl vm.max_map_count=... is refused, because Docker permits only namespaced sysctls and this one belongs to the whole host.

Docker Desktop on Mac and Windows runs containers inside a Linux VM, so the Linux steps on your own machine change the wrong kernel. Set the limit inside the VM instead.

On Windows, with the WSL 2 backend:

Bash

wsl -d docker-desktop -u root sysctl -w vm.max_map_count=4194304

To make it survive a restart, add this to %USERPROFILE%\.wslconfig:

ini

[wsl2]
kernelCommandLine = "sysctl.vm.max_map_count=4194304"

On macOS, apply it from a privileged container. The setting lasts until Docker Desktop restarts:

Bash

docker run --rm --privileged --pid=host alpine nsenter -t 1 -m -u -n -i sysctl -w vm.max_map_count=4194304

Then restart the Weaviate container so Weaviate reads the new limit.

The setting belongs to the host kernel, not to the container, so setting it inside an unprivileged pod has no effect. Apply it on every node that can schedule a Weaviate pod, through one of:

the official Helm chart, which already ships a privileged init container, enabled by default, that runs sysctl -w vm.max_map_count on the pod's node before Weaviate starts. Its default of 524288 (initContainers.sysctlVmMaxMapCount) is well below the value this page recommends, so set initContainers.sysctlVmMaxMapCount: 4194304 in your values rather than adding infrastructure
your node configuration or image, with vm.max_map_count=4194304 in /etc/sysctl.conf
a DaemonSet whose privileged container runs sysctl -w vm.max_map_count=4194304 on each node
a privileged init container on the Weaviate pod that sets it before Weaviate starts

Then restart the affected pod so Weaviate reads the new limit.

The host configuration is not yours to change. Open a support ticket with the message text and your cluster URL.

If the limit is already high and the error keeps coming back, the node holds more shards than it was sized for: deactivate or offload tenants that are not in use, or add nodes and rebalance.

Learn more

Known issue: shard init failure

Resource planning

Tenant states and offloading

Deployment troubleshooting

Improve your cluster

Questions and feedback

Have a question or feedback? Here's how to reach us.

Community Forum

Ask questions and connect with other developers on our Community forum.

Support

Weaviate Cloud user or customer? Find the right channel on the Support page.