Calico node status
A Calico node status resource (CalicoNodeStatus
) represents a collection of status information for a node that Calico reports back to the user for use during troubleshooting.
As of today, status of BGP agents, BGP sessions and routes exposed to BGP agents are collected from Linux nodes only. Windows nodes are not supported at this time. Calico node status resource is only valid when Calico BGP networking is in use.
Notes​
The updating of CalicoNodeStatus
will have a small performance impact on CPU/Memory usage of the node as well as adding load to kubernetes apiserver.
In our testing on a ten node, full mesh cluster, a CalicoNodeStatus
resource was created for each node where the update interval was set to ten seconds. On each node, this resulted in an increase in CPU use of 5% of a vCPU and an increase of 4MB of memory. The control plane node recorded an increase in CPU usage of 5% of a vCPU for these 10 nodes.
The implementation of CalicoNodeStatus
is designed to handle a small number of nodes (less than 10 is recommended) reporting back status in the same time. If CalicoNodeStatus
are created for a large number of nodes, and with short update interval,
the kubernetes apiserver may become slower and less responsive.
You should create CalicoNodeStatus
for the node you are interested in and for debugging purpose only. CalicoNodeStatus
resource should be deleted upon the completion of the debugging process.
Sample YAML​
To use this function, the user creates a CalicoNodeStatus object for the node, specifying the information to collect and the interval it should be collected at. This example collects information for node "my-kadm-node-0" with an update interval of 10 seconds.
kubectl apply -f -<<EOF
apiVersion: projectcalico.org/v3
kind: CalicoNodeStatus
metadata:
name: my-caliconodestatus-1
spec:
classes:
- Agent
- BGP
- Routes
node: my-kadm-node-0
updatePeriodSeconds: 10
EOF
The user then reads back the same resource using the command kubectl get caliconodestatus my-caliconodestatus-1 -o yaml
, and the information collected can be found in the status
field. For example:
apiVersion: projectcalico.org/v3
kind: CalicoNodeStatus
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"crd.projectcalico.org/v1","kind":"CalicoNodeStatus","metadata":{"annotations":{},"name":"my-caliconodestatus-1"},"spec":{"classes":["Agent","BGP","Routes"],"node":"my-kadm-node-0","updatePeriodSeconds":10}}
projectcalico.org/metadata: '{"uid":"6e3b676b-546a-41db-aeda-ad44f25ff73f","generation":2,"creationTimestamp":"2021-10-18T10:30:43Z","managedFields":[{"manager":"kubectl-client-side-apply","operation":"Update","apiVersion":"projectcalico.org/v3","time":"2021-10-18T10:30:43Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:kubectl.kubernetes.io/last-applied-configuration":{}}},"f:spec":{".":{},"f:node":{},"f:updatePeriodSeconds":{}}}},{"manager":"kubectl-edit","operation":"Update","apiVersion":"crd.projectcalico.org/v1","time":"2021-10-18T12:59:25Z","fieldsType":"FieldsV1","fieldsV1":{"f:spec":{"f:classes":{}}}}]}'
creationTimestamp: '2021-10-18T10:30:43Z'
generation: 139
name: my-caliconodestatus-1
resourceVersion: '203255'
uid: 6e3b676b-546a-41db-aeda-ad44f25ff73f
spec:
classes:
- Agent
- BGP
- Routes
node: my-kadm-node-0
updatePeriodSeconds: 10
status:
agent:
birdV4:
lastBootTime: '2021-10-18 09:23:19'
lastReconfigurationTime: '2021-10-18 09:23:19'
routerID: 10.128.0.240
state: Ready
version: v0.3.3+birdv1.6.8
birdV6:
lastBootTime: '2021-10-18 09:23:19'
lastReconfigurationTime: '2021-10-18 09:23:19'
routerID: 10.128.0.240
state: Ready
version: v0.3.3+birdv1.6.8
bgp:
numberEstablishedV4: 2
numberEstablishedV6: 0
numberNotEstablishedV4: 1
numberNotEstablishedV6: 0
peersV4:
- peerIP: 10.128.0.242
since: '09:23:22'
state: Established
type: NodeMesh
- peerIP: 10.128.0.200
since: '09:23:21'
state: Established
type: NodeMesh
- peerIP: 10.128.0.241
since: '09:23:22'
state: OpenSent
type: NodeMesh
lastUpdated: '2021-10-18T13:22:03Z'
routes:
routesV4:
- destination: 0.0.0.0/0
gateway: 172.18.0.1
interface: eth0
learnedFrom:
sourceType: Kernel
type: FIB
- destination: 192.168.82.0/26
gateway: 10.128.0.200
interface: eth0
learnedFrom:
peerIP: 10.128.0.200
sourceType: NodeMesh
type: FIB
- destination: 192.168.36.0/26
gateway: 10.128.0.242
interface: eth0
learnedFrom:
peerIP: 10.128.0.242
sourceType: NodeMesh
type: FIB
- destination: 10.128.0.5/16
gateway: N/A
interface: eth0
learnedFrom:
sourceType: Direct
type: FIB
Calico node status definition​
Metadata​
Field | Description | Accepted Values | Schema |
---|---|---|---|
name | The name of this CalicoNodeStatus resource. Required. | Alphanumeric string with optional . , _ , or - . | string |
Spec​
Field | Description | Accepted Values | Schema |
---|---|---|---|
classes | List of classes that declare the types of information to monitor for this node and allows for selective status reporting about certain subsets of information. Required. | List of Classes | list of string |
node | The node name identifies the Calico node instance to collect node status for. Required. | valid Kubernetes node name | string |
updatePeriodSeconds | Period in seconds at which CalicoNodeStatus should be updated. Set to 0 to disable CalicoNodeStatus refresh. Maximum update period is one day. Required. | 0 - 86400 | int |
Status​
Field | Description | Values |
---|---|---|
lastUpdated | Timestamp representing the server time when the CalicoNodeStatus object last updated. It is represented in RFC3339 form and is in UTC. | |
agent | List of BGP daemon status on the node. | 'birdV4' holds status of IPv4 bird daemon, 'birdV6' holds status of IPv6 bird daemon |
bgp | List of BGP status on the node. | 'peersV4' holds status of IPv4 BGP peers, 'peersV6' holds status of IPv6 BGP peers |
routes | List of Route status learned by BGP daemon on the node. | 'routesV4' holds status of IPv4 routes , 'routesV6' holds status of IPv6 routes |
Classes​
Accepted Values | Description |
---|---|
Agent | Status of BGP daemon agent |
BGP | Status of BGP sessions |
Routes | Status of routes exposed to BGP daemon |
BGPDaemonStatus​
Field | Description | Schema | Possible Values |
---|---|---|---|
state | State of BGP daemon. | string | Ready, NotReady |
version | Version of BGP daemon | string | |
routerID | Router ID used by BGP daemon | string | |
lastBootTime | Last boot time of BGP daemon | string | |
lastReconfigurationTime | Last reconfiguration time of BGP daemon | string |
BGPStatus​
Field | Description | Schema |
---|---|---|
numberEstablishedV4 | The total number of IPv4 established bgp sessions. | int |
numberNotEstablishedV4 | The total number of IPv4 non-established bgp sessions. | int |
numberEstablishedV6 | The total number of IPv6 established bgp sessions. | int |
numberNotEstablishedV6 | The total number of IPv6 non-established bgp sessions. | int |
peersV4 | IPv4 BGP peers status on the node. | List of BGP peer |
peersV6 | IPv6 BGP peers status on the node. | List of BGP peer |
RouteStatus​
Field | Description | Schema |
---|---|---|
routesV4 | IPv4 routes learned by BGP daemon on the node. | List of route |
routesV6 | IPv6 routes learned by BGP daemon on the node. | List of route |
BGPPeer​
Field | Description | Schema | Possible Values |
---|---|---|---|
peerIP | IP address of the peer whose condition we are reporting. | ip | |
type | Type indicates whether this peer is configured via the node-to-node mesh, or via en explicit global or per-node BGPPeer object. | string | NodeMesh, NodePeer, GlobalPeer |
state | BGP session state. Value of an empty string means BGP session state is unknown. | string | Idle, Connect, Active, OpenSent, OpenConfirm, Established, Close |
since | Timestamp of last BGP session state change. | time |
Route​
Field | Description | Schema | Possible Values |
---|---|---|---|
type | Type indicates if the route is being used for forwarding or not. | string | FIB, RIB |
destination | Destination IP of the route. | ip | |
gateway | Gateway IP of the route. If the route does not involve a gateway, for example a direct route, the value is N/A . | ip or N/A | |
interface | Network interface for the destination. For a blackhole route, the value is blackhole . For a unreachable route, the value is unreachable . | string | |
learnedFrom | Information regarding where this route originated. | learned from |
LearnedFrom​
Field | Description | Schema | Possible Values |
---|---|---|---|
sourceType | Type of the source where a route is learned from. | string | Kernel, Static, Direct, NodeMesh, BGPPeer |
peerIP | If sourceType is NodeMesh or BGPPeer, IP address of the router that sent us this route. | ip |