Objective: Find appropriate CPU/memory settings to improve resource efficiency Estimated Time: 45 minutes
Scope of This GuideCovers: Measuring resource usage, determining appropriate requests/limits values, resolving throttling/OOM
Does not cover: Auto-scaling (see Scaling), Pod startup issues (see Pod Troubleshooting)
Before You Begin#
Verify the following prerequisites.
1. Verify kubectl Installation and Version#
kubectl version --clientSuccess output:
Client Version: v1.28.02. Verify Metrics Server Installation#
kubectl top nodesSuccess output:
NAME CPU(cores) CPU% MEMORY(bytes) MEMORY%
node-1 250m 12% 1024Mi 50%If you get an error, install Metrics Server.
minikube addons enable metrics-serverkubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yamlWait 1-2 minutes after installation, then verify again.
3. Verify Target Pods#
Verify the Pods you want to optimize are in Running status.
kubectl get pods -l app=<your-app>Success output:
NAME READY STATUS RESTARTS AGE
my-app-xxx-yyy 1/1 Running 0 5mStep 1: Measure Current Usage#
Measurement Duration
Measure for at least 1 hour for accurate analysis. Including peak hours is recommended.
Check Pod Resource Usage#
kubectl top podsExpected output:
NAME CPU(cores) MEMORY(bytes)
my-app-xxx-yyy 50m 256Mi
my-app-xxx-zzz 45m 248MiTo check by container, run:
kubectl top pods --containersSuccess check: CPU(cores) and MEMORY(bytes) values are displayed.
Track Usage Over Time#
Start real-time monitoring that refreshes every 2 seconds.
watch -n 2 kubectl top podsTip
Record both peak time maximum values and normal operation values.
Recording example:
| Time Period | CPU | Memory |
|---|---|---|
| Normal | 50m | 256Mi |
| Peak | 200m | 400Mi |
Step 2: Check Current Settings#
Check the current resource settings of your Deployment.
kubectl get deployment <deployment-name> -o jsonpath='{.spec.template.spec.containers[0].resources}'Expected output:
{"limits":{"cpu":"1000m","memory":"2Gi"},"requests":{"cpu":"500m","memory":"1Gi"}}Problem Diagnosis Criteria#
| Current State | Problem | Action |
|---|---|---|
| requests > 5x actual usage | Resource waste | Decrease requests |
| requests < actual usage | Throttling risk | Increase requests |
| limits < peak usage | OOM/throttling occurring | Increase limits |
Success check: You can view current settings and compare with actual usage.
Step 3: Calculate Appropriate Values#
Recommended Calculation Formula#
CPU requests = Normal usage × 1.2 (20% buffer)
CPU limits = Peak usage × 1.5 or unset
Memory requests = Normal usage × 1.2
Memory limits = requests × 1.5Calculation Example#
Measurements:
- Normal CPU: 50m, Peak CPU: 200m
- Normal Memory: 256Mi, Peak Memory: 400Mi
Calculations:
CPU requests = 50m × 1.2 = 60m → 100m (rounded)
CPU limits = 200m × 1.5 = 300m → 500m
Memory requests = 256Mi × 1.2 = 307Mi → 320Mi
Memory limits = 320Mi × 1.5 = 480Mi → 512MiUsing VPA Recommender (Optional)#
Installing VPA automatically calculates recommended values.
# vpa.yaml
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
name: my-app-vpa
spec:
targetRef:
apiVersion: apps/v1
kind: Deployment
name: my-app
updatePolicy:
updateMode: "Off" # Only show recommendations, don't auto-applykubectl apply -f vpa.yaml
kubectl describe vpa my-app-vpaSuccess check: Recommended values are displayed in the Recommendation section.
Step 4: Apply Changes#
Caution
In production environments, avoid making large changes at once. Adjust incrementally.
Modify Deployment#
kubectl edit deployment <deployment-name>Or use the patch command.
kubectl patch deployment <deployment-name> -p '
{
"spec": {
"template": {
"spec": {
"containers": [{
"name": "<container-name>",
"resources": {
"requests": {
"memory": "320Mi",
"cpu": "100m"
},
"limits": {
"memory": "512Mi",
"cpu": "500m"
}
}
}]
}
}
}
}'Success check: New Pod is created and reaches Running status.
kubectl rollout status deployment <deployment-name>Step 5: Validate#
Verify the following after making changes.
Check for Throttling#
Check if CPU throttling is occurring.
kubectl exec <pod-name> -- cat /sys/fs/cgroup/cpu/cpu.statCheck output:
nr_throttled 0 # 0 means no throttling
throttled_time 0 # 0 means normalcgroup v2 EnvironmentsIn cgroup v2 environments, use the following command:
kubectl exec <pod-name> -- cat /sys/fs/cgroup/cpu.stat
Check for OOM#
Verify no OOM events are occurring.
kubectl get events --field-selector reason=OOMKillingSuccess check: No events returned.
Check QoS Class#
kubectl get pod <pod-name> -o jsonpath='{.status.qosClass}'| QoS Class | Meaning | Recommended Scenario |
|---|---|---|
| Guaranteed | requests = limits | Critical workloads |
| Burstable | requests < limits | General workloads |
| BestEffort | No resources set | Test environments only |
Success check: Intended QoS class is applied.
Common Errors#
“OOMKilled” Repeatedly Occurring#
Cause: Memory limits are too low.
Solution:
# Check current limits
kubectl describe pod <pod-name> | grep -A 2 "Limits"
# Increase limits (e.g., 512Mi → 1Gi)
kubectl patch deployment <name> -p '{"spec":{"template":{"spec":{"containers":[{"name":"<container>","resources":{"limits":{"memory":"1Gi"}}}]}}}}'Response Delay Due to CPU Throttling#
Cause: CPU limits are too low.
Solution:
- Increase CPU limits, or
- Remove CPU limits (set only requests)
resources:
requests:
cpu: "100m"
# limits.cpu omitted - can use node CPU
limits:
memory: "512Mi"“0/N nodes are available: Insufficient cpu/memory”#
Cause: requests exceed available node resources.
Solution:
# Check node available resources
kubectl describe nodes | grep -A 5 "Allocatable"
# Adjust requests to be within available resourcesMetrics Server “error: Metrics API not available”#
Cause: Metrics Server is not installed or not ready.
Solution:
# Check Metrics Server status
kubectl get pods -n kube-system | grep metrics-server
# Check logs
kubectl logs -n kube-system deployment/metrics-serverJava Application Specific Settings#
JVM heap memory configuration is critical for Java applications.
Caution
Setting JVM heap equal to container memory will cause OOM. JVM uses additional memory beyond heap (metaspace, thread stacks, etc.).
Recommended Settings#
resources:
requests:
memory: "512Mi"
limits:
memory: "1Gi"
env:
- name: JAVA_OPTS
value: "-XX:MaxRAMPercentage=75.0 -XX:+UseG1GC"Heap Ratio by Container Memory#
| Container Memory | Recommended Heap Ratio | Reason |
|---|---|---|
| < 512Mi | 50-60% | Non-heap memory ratio is relatively higher |
| 512Mi - 2Gi | 65-75% | Typical settings |
| > 2Gi | 75-80% | Non-heap ratio decreases for large heaps |
Success check: Pod runs stably without OOM.
Checklist#
Measurement#
- Is Metrics Server installed?
- Have you measured usage for at least 1 hour?
- Did you check peak time usage?
Configuration#
- Are requests 100-120% of normal usage?
- Can limits accommodate peak usage?
- For Java apps, is JVM heap appropriately configured?
Validation#
- Is the Pod in Running status after changes?
- Is there no CPU throttling?
- Is there no OOM occurring?
- Is the intended QoS class applied?
Next Steps#
| Goal | Recommended Document |
|---|---|
| Configure auto-scaling | Scaling |
| Resolve Pod issues | Pod Troubleshooting |
| Resource management concepts | Resource Management |