Overview
Big picture
Use Calico Cloud log data for visibility and troubleshooting Kubernetes clusters.
Value
Workloads and policies are highly dynamic. To troubleshoot Kubernetes clusters, you need logs with workload identity and context. Calico Cloud deploys an Elasticsearch cluster and Kibana instance during installation with these features:
- Logs with workload context
- Centralized log collection for multiple clusters for Calico Cloud multi-cluster-management
- View Elasticsearch logs in Calico Cloud Manager UI (Kibana dashboard and Flow Visualizer), and the Elasticsearch API
- Standard Kubernetes RBAC for granular access control to logs
- Collect/archive logs or subset of logs
- Log aggregation for high-volume logs
- Configure data retention settings to manage cluster disk space
- Integration with third-party tools like Amazon S3, Syslog, Splunk
Concepts
Logs types
Elasticsearch logs provide the visibility and troubleshooting backend for Calico Cloud.
| Log type | Description | Log source | RBAC | Index |
|---|---|---|---|---|
| flow | Network flows for workloads: source and destination namespaces, pods, labels, and policies | Calico Cloud cnx-node (Felix) | flows | tigera_secure_ee_flows |
| audit | Audit logs for Calico Cloud resources | Calico Cloud apiserver | audit_ee | tigera_secure_ee_audit_ee |
| Audit logs for Kubernetes resources | Kubernetes apiserver | audit_kube | tigera_secure_ee_audit_kube | |
| Both audit logs above | audit* | tigera_secure_ee_audit* | ||
| bgp | Calico Cloud networking BGP peering and route propagation | Calico Cloud cnx-node (BIRD) | ee_bgp | tigera_secure_ee_bgp.* |
| dns | DNS lookups and responses from Calico Cloud domain-based policy | Calico Cloud cnx-node (Felix) | ee_dns | tigera_secure_ee_dns |
| ids | Calico Cloud intrusion detection events: suspicious IPs, suspicious domains, and global alerts | Calico Cloud intrusion-detection-controller | ee_events | tigera_secure_ee_events |
Because of their high-volume, flow and dns logs support aggregation.
Default log configuration and security
Calico Cloud automatically installs fluentd on all nodes and collects flow, audit, and DNS logs. You can configure additional destinations like Amazon S3, Syslog, Splunk.
Calico Cloud enables user authentication in Elasticsearch, and secures access to Elasticsearch and Kibana instances using network policy.
RBAC and log access
You control user access to logs using the standard Kubernetes RBAC cluster role and cluster role binding. For example:
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: bob-es-access
subjects:
- kind: User
name: bob
apiGroup: rbac.authorization.k8s.io
roleRef:
kind: ClusterRole
name: audit-ee-only
apiGroup: rbac.authorization.k8s.io
You configure Elasticsearch log access per cluster using RBAC and the Kubernetes API group, lma.tigera.io. For example:
apiGroups: ['lma.tigera.io']
resources: ['app-cluster']
resourceNames: ['flows', 'dns']
verbs: ['get']
Logs for compliance reporting
Calico Cloud compliance reports are based on archived flow logs and audit logs for these resources:
- Pods
- Host endpoints
- Service accounts
- Namespaces
- Kubernetes service endpoints
- Global network sets
- Calico Cloud and Kubernetes network policies
- Global network policies
- Network sets
Calico Cloud also supports archiving Cloudwatch for EKS audit logs.