Change IP pool block size
Big picture
Change the IP pool block size to efficiently manage IP pool addresses.
Value
Changing IP pool block size after installation requires ordered steps to minimize pod connectivity disruption.
Concepts
About IP pools
By default, Calico Enterprise uses an IPAM block size of 64 addresses – /26 for IPv4, and /122 for IPv6. However, the block size can be changed depending on the IP pool address family.
- IPv4: 20-32, inclusive
- IPv6: 116-128, inclusive
You can have only one default IP pool for per protocol in your installation manifest. In this example, there is one IP pool for IPv4 (/26), and one IP pool for IPv6 (/122).
apiVersion: operator.tigera.io/v1
kind: Installation
metadata:
name: default
spec:
# Configures Calico networking.
calicoNetwork:
# Note: The ipPools section cannot be modified post-install.
ipPools:
- blockSize: 26
cidr: 10.48.0.0/21
encapsulation: IPIP
natOutgoing: Enabled
nodeSelector: all()
- blockSize: 122
cidr: 2001::00/64
encapsulation: None
natOutgoing: Enabled
nodeSelector: all()
However, the following is invalid because it has two IP pools for IPv4.
apiVersion: operator.tigera.io/v1
kind: Installation
metadata:
name: default
spec:
# Configures Calico networking.
calicoNetwork:
# Note: The ipPools section cannot be modified post-install.
ipPools:
- blockSize: 26
cidr: 10.48.0.0/21
encapsulation: IPIP
natOutgoing: Enabled
nodeSelector: all()
- blockSize: 31
cidr: 10.48.8.0/21
encapsulation: IPIP
natOutgoing: Enabled
nodeSelector: all()
Expand or shrink IP pool block sizes
By default, the Calico Enterprise IPAM block size for an IP pool is /26. To expand from the default size /26, lower the blockSize (for example, /24). To shrink the blockSize from the default /26, raise the number (for example, /28).
Best practice: change IP pool block size before installation
Because the blockSize field cannot be edited directly after Calico Enterprise installation, it is best to change the IP pool block size before installation to minimize disruptions to pod connectivity.
Before you begin...
Required
Verify that you are using Calico Enterprise IPAM.
If you are not sure which IPAM your cluster is using, the way to tell depends on install method.
The IPAM plugin can be queried on the default Installation resource.
kubectl get installation default -o go-template --template {{.spec.cni.ipam.type}}
If your cluster is using Calico IPAM, the above command should return a result of Calico.
How to
Follow the steps to minimize pod connectivity disruption. Pods may lose connectivity when they are redeployed, and may lose external connectivity while in the temporary pool. Also, when pods are deleted, applications may be temporarily unavailable (depending on the type of application). Plan your changes accordingly.
The high-level steps to follow are:
- Create a temporary IP pool Note: The temporary IP pool must not overlap with the existing one.
- Disable the existing IP pool Note: When you disable an IP pool, only new IP address allocations are prevented; networking of existing pods are not affected.
- Delete pods from the existing IP pool This includes any new pods that may have been created with the existing IP pool prior to disabling the pool. Verify that new pods get an address from the temporary IP pool.
- Delete the existing IP pool
- Create a new IP pool with the desired block size
- Disable the temporary IP pool
- Delete pods from the temporary IP pool
- Delete the temporary IP pool
Tutorial
In the following steps, our Kubernetes cluster has a default CIDR block size of /26. We want to shrink the block size to /28 to use the pool more efficiently.
Create a temporary IP pool
We add a new IPPool with the CIDR range, 10.0.0.0/16.
Create a temporary-pool.yaml.
apiVersion: projectcalico.org/v3
kind: IPPool
metadata:
name: temporary-pool
spec:
cidr: 10.0.0.0/16
ipipMode: Always
natOutgoing: true
Apply the changes.
kubectl apply -f temporary-pool.yaml
Let’s verify the temporary IP pool.
calicoctl get ippool -o wide
NAME CIDR NAT IPIPMODE DISABLED
default-ipv4-ippool 192.168.0.0/16 true Always false
temporary-pool 10.0.0.0/16 true Always false
Disable the existing IP pool
Disable allocations in the default pool.
kubectl patch ippool default-ipv4-ippool -p '{"spec": {"disabled": "true"}}'
Verify the changes.
calicoctl get ippool -o wide
NAME CIDR NAT IPIPMODE DISABLED
default-ipv4-ippool 192.168.0.0/16 true Always true
temporary-pool 10.0.0.0/16 true Always false
Delete pods from the existing IP pool
In our example, coredns is our only pod; for multiple pods you would trigger a deletion for all pods in the cluster.
kubectl delete pod -n kube-system coredns-6f4fd4bdf-8q7zp
Restart all pods with just one command.
The following command is disruptive and may take several minutes depending on the number of pods deployed.
kubectl delete pod -A --all
Delete the existing IP pool
Now that you’ve verified that pods are getting IPs from the new range, you can safely delete the existing pool.
kubectl delete ippool default-ipv4-ippool
Create a new IP pool with the desired block size
In this step, we update the IPPool with the new block size of (/28).
apiVersion: projectcalico.org/v3
kind: IPPool
metadata:
name: default-ipv4-ippool
spec:
blockSize: 28
cidr: 192.0.0.0/16
ipipMode: Always
natOutgoing: true
Apply the changes.
kubectl apply -f pool.yaml
Disable the temporary IP pool
kubectl patch ippool temporary-pool -p '{"spec": {"disabled": "true"}}'
Delete pods from the temporary IP pool
In our example, coredns is our only pod; for multiple pods you would trigger a deletion for all pods in the cluster.
kubectl delete pod -n kube-system coredns-6f4fd4bdf-8q7zp
Restart all pods with just one command.
The following command is disruptive and may take several minutes depending on the number of pods deployed.
kubectl delete pod -A --all
Validate your pods and block size are correct by running the following commands:
kubectl get pods --all-namespaces -o wide
calicoctl ipam show --show-blocks
Delete the temporary IP pool
Clean up the IP pools by deleting the temporary IP pool.
kubectl delete pool temporary-pool