Cluster high memory pressure
Symptom
A CustomerClusterHighMemoryPressure alert is firing. It indicates a node in your cluster is under high memory pressure.
Observations
Here are the details on the alerts:
Alert: CustomerClusterHighMemoryPressure
| Property | Value |
|---|---|
| Expression | instance:node_memory_available:ratio * 100 < 5 and rate(node_vmstat_pgmajfault[2m]) > 1000 |
| Delay | 15 minutes |
| Labels | team: customer |
Annotations:
summarydescriptiontenantcluster_idcluster_nameinstance
Determine the node with high memory pressure
The alert contains:
- the cluster name/id (the
k8s_cluster_nameandk8s_cluster_idlabels) - the cluster node (the instance label)
- the device (the device label)
- the filesystem (the mountpoint label)
Solutions
This section provides solutions that should help resolve the issue in most cases.
Increase cluster node instance type
The memory available to a cluster node is determined by the instance type. Cluster nodes are usually t3a-xlarge EC2 instances with 16GB memory. The next larger instance type, t3a.2xlarge, has 32GB memory. There are no instance types larger than t3a.2xlarge.
Add a node to the cluster
We recommend that each Magnolia instance and its database be run on a separate node in the cluster: the author plus two public instances can each be run on separate nodes and each Magnolia instance can have enough memory (10 - 12 GB for author instances and 8 - 10 GB for public instances).
If you want to run more than three Magnolia instances on the cluster, additional nodes can be added to the cluster.