Install Calico Enterprise on a Red Hat Openshift on AWS (ROSA) cluster
Big picture
Install Calico Enterprise on a Red Hat Openshift on AWS (ROSA) cluster. Classic ROSA does not support installations with custom CNI plugin. ROSA with HCP clusters allows installing Calico Enterprise.
Value
Provides steps to install Calico Enterprise in Red Hat Openshift on AWS (ROSA) with HCP clusters.
Before you begin
-
Ensure that your environment meets the Calico Enterprise system requirements.
-
Ensure that you have a RedHat account. A RedHat account is required to get the pull secret necessary to provision a ROSA cluster.
-
Ensure that you have:
- Configured an AWS account appropriate for OpenShift 4
- Set up your AWS credentials
- Enabled ROSA with HCP and HCP classic on AWS.
- Ensured you have met the resource quota requirements for ROSA on AWS.
- Linked AWS and Red Hat accounts.
- Generated a local SSH private key and added it to your ssh-agent
-
Install the ROSA CLI and fulfill all the other prerequisites from the ROSA documentation
-
Create a Virtual Private Cloud (VPC) for your ROSA cluster following the documentation.
How to
Create roles, policies
-
Create the account wide STS roles and policies needed to create a ROSA with HCP cluster by following the Red Hat documentation.
-
Create the OpenID Connect (OIDC) configuration prior to creating the ROSA with HCP cluster.
-
Create Operator IAM roles and policies needed for ROSA with HCP clusters.
Create ROSA cluster with HCP and no CNI plugin
Follow the steps from the Red Hat Openshift on AWS documentation to create a ROSA cluster with HCP and no CNI plugin.
Create a storage class
Calico Enterprise requires storage for logs and reports. Before finishing the installation, you must create a StorageClass for Calico Enterprise.
Download the Calico Enterprise install manifests
For OpenShift v4.16 or newer on AWS, configure AWS security groups to allow BGP, typha and IP-in-IP encapsulation traffic.
Custom TCP traffic
Port: 179
Description: BGP (Calico)
Custom TCP traffic
Port: 5473
Description: Typha (Calico)
Custom protocol
Protocol: 4
Description: IP-in-IP (Calico)
Now download and apply the Calico Enterprise manifests.
Download the Calico Enterprise manifests for OpenShift:
mkdir calico
wget -qO- https://downloads.tigera.io/ee/v3.21.1/manifests/ocp.tgz | tar xvz --strip-components=1 -C calico
Add an image pull secret
Update the contents of the secret with the image pull secret provided to you by a Tigera support representative.
For example, if the secret is located at ~/.docker/config.json, run the following commands.
SECRET=$(cat ~/.docker/config.json | tr -d '\n\r\t ' | base64 -w 0)
sed -i "s/SECRET/${SECRET}/" calico/02-pull-secret.yaml
Optionally provide additional configuration
You may want to provide Calico Enterprise with additional configuration at install-time. For example, BGP configuration or peers. You can use a Kubernetes ConfigMap with your desired Calico resources to set configuration as part of the installation. If you do not need to provide additional configuration, you can skip this section.
To include Calico resources during installation, edit calico/02-configmap-calico-resources.yaml
to add your own configuration.
If you have a directory with the Calico resources, you can create the file with the command:
oc create configmap -n tigera-operator calico-resources \
--from-file=<resource-directory> --dry-run -o yaml \
calico/02-configmap-calico-resources.yaml
With recent versions of oc it is necessary to have a kubeconfig configured or add --server='127.0.0.1:443'
even though it is not used.
If you have provided a calico-resources
configmap and the tigera-operator pod fails to come up with Init:CrashLoopBackOff
,
check the output of the init-container with oc logs -n tigera-operator -l k8s-app=tigera-operator -c create-initial-resources
.
Apply the Calico Enterprise install manifests
Apply the install manifests paying attention to the required order. First apply all necessary manifests to install the Tigera Operator:
cd calico/
ls 00* | xargs -n1 oc apply -f
ls 01* | xargs -n1 oc apply -f
ls 02* | xargs -n1 oc apply -f
Then wait until the Tigera Operator creates the necessary CustomResourceDefinitions (CRDs) before applying the CustomResources to install Calico Enterprise on your cluster:
timeout --foreground 600 bash -c "while ! kubectl get crd installations.operator.tigera.io; do sleep 5; done" # wait until CRDs are created by the operator
ls 03* | xargs -n1 oc apply -f
You can safely ignore any ls: cannot access '0X*': No such file or directory
errors if there's no yaml files with that prefix to be applied.
Install the Calico Enterprise license
In order to use Calico Enterprise, you must install the license provided to you by Tigera support representative. Before applying the license, wait until the Tigera API server is ready with the following command:
watch oc get tigerastatus
Wait until the apiserver
shows a status of Available
.
After the Tigera API server is ready, apply the license:
oc create -f </path/to/license.yaml>
calico-enterprise_versioned_docs/version-3.21-2/getting-started/install-on-clusters/openshift/rosa.mdx
Install Calico Enterprise resources
Apply the custom resources for enterprise features.
oc create -f https://downloads.tigera.io/ee/v3.21.1/manifests/ocp/tigera-enterprise-resources.yaml
Apply the Calico Enterprise manifests for the Prometheus operator.
oc create -f https://downloads.tigera.io/ee/v3.21.1/manifests/ocp/tigera-prometheus-operator.yaml
You can now monitor progress with the following command:
watch oc get tigerastatus
When it shows all components with status Available
, proceed to the next step.
(Optional) Apply the full CRDs including descriptions.
oc apply --server-side --force-conflicts -f https://downloads.tigera.io/ee/v3.21.1/manifests/operator-crds.yaml
Next steps
Recommended
- Configure access to the Calico Enterprise web console
- Authentication quickstart
- Configure your own identity provider
Recommended - Networking
- The default networking uses IP in IP encapsulation with BGP routing. For all networking options, see Determine best networking option.
Recommended - Security