Skip to main content
Calico Enterprise 3.19 (latest) documentation

Install using Operator

Big picture​

Install Calico Enterprise for Windows on your Kubernetes cluster using the Tigera Operator.

Concepts​

Calico Enterprise for Windows is a hybrid implementation that requires a Linux cluster for Calico Enterprise components and Linux workloads, and Windows nodes for Windows workloads.

Before you begin​

Review the Linux requirements and the Calico Enterprise for Windows requirements.

Before beginning, setup a Calico Enterprise cluster on Linux nodes and provision Windows machines.

How to​

Configure strict affinity for clusters using Calico Enterprise networking​

For Linux control nodes using Calico Enterprise networking, strict affinity must be set to true. This is required to prevent Linux nodes from borrowing IP addresses from Windows nodes:

kubectl patch ipamconfigurations default --type merge --patch='{"spec": {"strictAffinity": true}}'

Install Calico Enterprise for Windows using the operator​

With Kubernetes v1.22, there is a new Windows container type called HostProcess containers (HPC) that can run directly on the host with access to the host network namespace, storage, and devices. With this feature, Calico Enterprise for Windows can now be installed and managed using Kubernetes resources such as Daemonsets and ConfigMaps, instead of needing to configure and install Calico Enterprise for Windows manually on each node. Using this installation method, the Calico Enterprise for Windows services are no longer registered on the host. Instead, the services are run directly within HostProcess containers.

HPC requirements​

In addition to the Calico Enterprise for Windows requirements, this installation method has additional requirements:

  • Kubernetes v1.22+
  • Enable HostProcess containers support. For Kubernetes v1.22, see here. For Kubernetes v1.23+, HostProcess containers are enabled by default.
  • containerd v1.6+
  • The Windows nodes have joined the cluster.

To install containerd on the Windows node and configure the containerd service:

Invoke-WebRequest https://raw.githubusercontent.com/kubernetes-sigs/sig-windows-tools/master/hostprocess/Install-Containerd.ps1 -OutFile c:\Install-Containerd.ps1
c:\Install-Containerd.ps1 -ContainerDVersion 1.6.22 -skipHypervisorSupportCheck -CNIConfigPath "c:/etc/cni/net.d" -CNIBinPath "c:/opt/cni/bin"

If you have an existing Calico Enterprise for Windows installation using the manual method, your Windows nodes may have already joined the cluster.

To join a Windows node to a cluster provisioned with kubeadm:

  • Install kubeadm and kubelet binaries and install the kubelet service
Invoke-WebRequest https://raw.githubusercontent.com/kubernetes-sigs/sig-windows-tools/master/hostprocess/PrepareNode.ps1 -OutFile c:\PrepareNode.ps1
c:\PrepareNode.ps1 -KubernetesVersion v1.26.6
  • Run kubeadm on a control plane host and copy the join command
kubeadm token create --print-join-command
  • Edit the join command by updating the kubeadm.exe path to c:\k\kubeadm.exe.
  • If using a Kubernetes version earlier than v1.25, also append --cri-socket "npipe:////./pipe/containerd-containerd" to the join command. An example join command:
c:\k\kubeadm.exe join 172.16.101.139:6443 --token v8w2jt.jmc45acn85dbll1e --discovery-token-ca-cert-hash sha256:d0b7040a704d8deb805ba1f29f56bbc7cea8af6aafa78137a9338a62831739b4 --cri-socket "npipe:////./pipe/containerd-containerd"
  • Run the join command on the Windows node. Shortly after it completes successfully, the Windows node will appear in kubectl get nodes. The new node's status will be NotReady since the Calico CNI has not yet been installed.

Migrate from Calico Enterprise for Windows manual installation to operator installation​

The HPC installation in this guide is operator-based and will automatically migrate your Windows nodes that have Calico Enterprise for Windows installed using the manual installation method. This installation process will uninstall any existing Calico Enterprise for Windows services and overwrite the Calico Enterprise for Windows installation files with those included in the tigera/cni-windows and tigera/cnx-node-windows images. If kubelet was installed using C:\TigeraCalico\kubernetes\install-kube-services.ps1, this service will remain installed. If the kube-proxy service was also installed using C:\TigeraCalico\kubernetes\install-kube-services.ps1, it will be stopped and uninstalled by the calico-node-windows HPC initContainers. It is recommended to run kube-proxy-windows as hostprocess container images using the manifest provided in the kubernetes-sigs sig-windows-tools repository.

note

Before proceeding, take note of the configuration parameters in C:\TigeraCalico\config.ps1. These configuration parameters will be needed during the install.

Operator installation​

  1. Ensure that BGP is disabled.
    • If you installed Calico Enterprise using the manifest, BGP is already disabled.
    • If you installed Calico Enterprise using the operator, run this command:

      • kubectl patch installation default --type=merge -p '{"spec": {"calicoNetwork": {"bgp": "Disabled"}}}'

  2. Get the cluster's Kubernetes API server host and port, which will be used to update the Calico Enterprise for Windows config map. The API server host and port is required so that the Calico Enterprise for Windows installation script can create a kubeconfig file for Calico Enterprise services. If your Windows nodes already have Calico Enterprise for Windows installed manually, skip this step. The installation script will use the API server host and port from your node's existing kubeconfig file if the KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT variables are not provided in the calico-windows-config ConfigMap.

    First, make a note of the address of the API server:

    • If you have a single API server with a static IP address, you can use its IP address and port. The IP can be found by running:

      kubectl get endpoints kubernetes -o wide

      The output should look like the following, with a single IP address and port under "ENDPOINTS":

      NAME         ENDPOINTS             AGE
      kubernetes   172.16.101.157:6443   40m

      If there are multiple entries under "ENDPOINTS", then your cluster must have more than one API server. In this case, use the appropriate load balancing option below for your cluster.

    • If using DNS load balancing (as used by kops), use the FQDN and port of the API server api.internal.<clustername>.

    • If you have multiple API servers with a load balancer in front, you should use the IP and port of the load balancer.

      • tip

      If your cluster uses a ConfigMap to configure kube-proxy you can find the "right" way to reach the API server by examining the config map. For example:

      kubectl get configmap -n kube-system kube-proxy -o yaml | grep server
      server: https://d881b853ae312e00302a84f1e346a77.gr7.us-west-2.eks.amazonaws.com

      In this case, the server is d881b853aea312e00302a84f1e346a77.gr7.us-west-2.eks.amazonaws.com and the port is 443 (the standard HTTPS port).

  3. Create the kubernetes-services-endpoint ConfigMap with the Kubernetes API server host and port (discovered in the previous step) used to create a kubeconfig file for Calico Enterprise services.

    kubectl apply -f - << EOF
    kind: ConfigMap
    apiVersion: v1
    metadata:
      name: kubernetes-services-endpoint
      namespace: tigera-operator
    data:
      KUBERNETES_SERVICE_HOST: "${APISERVER_ADDR}"
      KUBERNETES_SERVICE_PORT: "${APISERVER_PORT}"
    EOF

  4. Get the Kubernetes service clusterIP range configured in your cluster. This must match the service-cluster-ip-range used by kube-apiserver.

  5. Add the Kubernetes service CIDR (discovered in the previous step) enable Calico Enterprise for Windows on the Tigera operator installation resource.

    For example, with a Kubernetes service clusterIP range of 10.96.0.0/12:

    kubectl patch installation default --type merge --patch='{"spec": {"serviceCIDRs": ["10.96.0.0/12"], "calicoNetwork": {"windowsDataplane": "HNS"}}}'

  6. Install kube-proxy on Windows nodes.

    Depending on your platform, you may already have kube-proxy running on your Windows nodes. If kube-proxy is already running on your Windows nodes, skip this step. If kube-proxy is not running, you must install and run kube-proxy on each of the Windows nodes in your cluster. Note: The manifest provided in the kubernetes-sigs sig-windows-tools repository depends on the kubeconfig provided by the kube-proxy ConfigMap in the kube-system namespace.

    You must replace KUBE_PROXY_VERSION with your cluster's Kubernetes version in kube-proxy.yml to ensure the daemonset uses a kube-proxy Windows image that is compatible with your Kubernetes cluster. Use a command like the following to retrieve the YAML file, replace the version, and apply it:

    curl -L  https://raw.githubusercontent.com/kubernetes-sigs/sig-windows-tools/master/hostprocess/calico/kube-proxy/kube-proxy.yml | sed "s/KUBE_PROXY_VERSION/<YOUR_KUBERNETES_VERSION>/g" | kubectl apply -f -

  7. Monitor the installation.

    The Calico Enterprise for Windows HPC installation has 2 initContainers: uninstall-calico, which deals with removing previous manually installed Calico Enterprise for Windows services, if any and install-cni, which installs needed CNI binaries and configuration, when using Calico CNI.
    kubectl logs -f -n calico-system -l k8s-app=calico-node-windows -c uninstall-calico
    kubectl logs -f -n calico-system -l k8s-app=calico-node-windows -c install-cni

    After these initContainers finish their execution, installation is complete. Next, theCalico Enterprise for Windows services are started in separate containers:

    kubectl logs -f -n calico-system -l k8s-app=calico-node-windows -c node
    kubectl logs -f -n calico-system -l k8s-app=calico-node-windows -c felix

    The calico-node-windows pods will be ready after their containers finish initializing.

Congratulations! You now have a Kubernetes cluster with Calico Enterprise for Windows and a Linux control node.

Next steps​

You can now use the Calico Enterprise Linux-based docs site for your documentation. Before you continue, review the Limitations and known issues to understand the features (and sections of documentation) that do not apply to Windows.