Skip to main content
Version: 3.18 (latest)

Felix configuration

A Felix configuration resource (FelixConfiguration) represents Felix configuration options for the cluster.

For kubectl commands, the following case-insensitive aliases may be used to specify the resource type on the CLI: felixconfiguration.projectcalico.org, felixconfigurations.projectcalico.org as well as abbreviations such as felixconfiguration.p and felixconfigurations.p.

See Configuring Felix for more details.

Sample YAML

apiVersion: projectcalico.org/v3
kind: FelixConfiguration
metadata:
name: default
spec:
ipv6Support: false
ipipMTU: 1400
chainInsertMode: Append

Felix configuration definition

Metadata

FieldDescriptionAccepted ValuesSchema
nameUnique name to describe this resource instance. Required.Alphanumeric string with optional ., _, or -.string
  • Calico Enterprise automatically creates a resource named default containing the global default configuration settings for Felix. You can use calicoctl to view and edit these settings
  • The resources with the name node.<nodename> contain the node-specific overrides, and will be applied to the node <nodename>. When deleting a node the FelixConfiguration resource associated with the node will also be deleted.

Spec

FieldDescriptionAccepted ValuesSchemaDefault
awsSrcDstCheckControls automatically setting source-destination-check on an AWS EC2 instance running Felix. Setting the value to Enable will set the check value in the instance description to true. For Disable, the check value will be false. Setting must be Disable if you want the EC2 instance to process traffic not matching the host interface IP address. For example, EKS cluster using Calico CNI with VXLANMode=CrossSubnet. Check IAM role and profile configuration for setting the necessary permission for this setting to work.DoNothing, Enable, DisablestringDoNothing
awsSecondaryIPSupportControls whether Felix will create secondary AWS ENIs for AWS-backed IP pools. This feature is documented in the egress gateways on AWS guide. Should only be enabled on AWS.Enabled, EnabledENIPerWorkload, DisabledstringDisabled
awsSecondaryIPRoutingRulePriorityControls the priority of the policy-based routing rules used to implement AWS-backed IP addresses. Should only be changed to avoid conflicts if your nodes have additional policy based routing rules.0-4294967295int101
awsRequestTimeoutTimeout used for communicating with the AWS API.5s, 10s, 1m etc.duration30s
dropActionOverrideControls what happens to each packet that is denied by the current Calico Enterprise policy. Normally the Drop or LogAndDrop value should be used. However when experimenting or debugging a scenario that is not behaving as you expect, the Accept and LogAndAccept values can be useful: then the packet will be still be allowed through. When one of the LogAnd... values is set, each denied packet is logged in syslog.*Drop, Accept, LogAndDrop, LogAndAcceptstringDrop
chainInsertModeControls whether Felix hooks the kernel's top-level iptables chains by inserting a rule at the top of the chain or by appending a rule at the bottom. Insert is the safe default since it prevents Calico Enterprise's rules from being bypassed. If you switch to Append mode, be sure that the other rules in the chains signal acceptance by falling through to the Calico Enterprise rules, otherwise the Calico Enterprise policy will be bypassed.Insert, AppendstringInsert
healthTimeoutOverridesA list of overrides for Felix's internal liveness/readiness timeouts.see belowList of HealthTimeoutOverride objects[]
dataplaneWatchdogTimeoutDeprecated, use healthTimeoutOverrides instead. Timeout before the main dataplane goroutine is determined to have hung and Felix will report non-live and non-ready. Can be increased if the liveness check incorrectly fails (for example if Felix is running slowly on a heavily loaded system).90s, 120s, 10m etc.duration90s
defaultEndpointToHostActionThis parameter controls what happens to traffic that goes from a workload endpoint to the host itself (after the traffic hits the endpoint egress policy). By default Calico Enterprise blocks traffic from workload endpoints to the host itself with an iptables "DROP" action. If you want to allow some or all traffic from endpoint to host, set this parameter to Return or Accept. Use Return if you have your own rules in the iptables "INPUT" chain; Calico Enterprise will insert its rules at the top of that chain, then Return packets to the "INPUT" chain once it has completed processing workload endpoint egress policy. Use Accept to unconditionally accept packets from workloads after processing workload endpoint egress policy.Drop, Return, AcceptstringDrop
deviceRouteSourceAddressIPv4 address to set as the source hint for routes programmed by Felix. When not set the source address for local traffic from host to workload will be determined by the kernel.IPv4string""
deviceRouteSourceAddressIPv6IPv6 address to set as the source hint for routes programmed by Felix. When not set the source address for local traffic from host to workload will be determined by the kernel.IPv6string""
deviceRouteProtocolThis defines the route protocol added to programmed device routes.ProtocolintRTPROT_BOOT
externalNodesCIDRListA comma-delimited list of CIDRs of external non-calico nodes that can source tunnel traffic for acceptance by calico-nodes.IPv4string""
failsafeInboundHostPortsUDP/TCP/SCTP protocol/cidr/port groupings that Felix will allow incoming traffic to host endpoints on irrespective of the security policy. This is useful to avoid accidentally cutting off a host with incorrect configuration. The default value allows SSH access, etcd, BGP, DHCP and the Kubernetes API.List of ProtoPort

- protocol: tcp
  port: 22
- protocol: udp
  port: 68
- protocol: tcp
  port: 179
- protocol: tcp
  port: 2379
- protocol: tcp
  port: 2380
- protocol: tcp
  port: 5473
- protocol: tcp
  port: 6443
- protocol: tcp
  port: 6666
- protocol: tcp
  port: 6667

failsafeOutboundHostPortsUDP/TCP/SCTP protocol/port groupings that Felix will allow outgoing traffic from host endpoints to irrespective of the security policy. This is useful to avoid accidentally cutting off a host with incorrect configuration. The default value opens etcd's standard ports to ensure that Felix does not get cut off from etcd as well as allowing DHCP, DNS, BGP and the Kubernetes API.List of ProtoPort

- protocol: udp
  port: 53
- protocol: udp
  port: 67
- protocol: tcp
  port: 179
- protocol: tcp
  port: 2379
- protocol: tcp
  port: 2380
- protocol: tcp
  port: 5473
- protocol: tcp
  port: 6443
- protocol: tcp
  port: 6666
- protocol: tcp
  port: 6667

featureDetectOverrideIs used to override the feature detection. Values are specified in a comma separated list with no spaces, example; "SNATFullyRandom=true,MASQFullyRandom=false,RestoreSupportsLock=". "true" or "false" will force the feature, empty or omitted values are auto-detected.stringstring""
genericXDPEnabledWhen enabled, Felix can fallback to the non-optimized generic XDP mode. This should only be used for testing since it doesn't improve performance over the non-XDP mode.true,falsebooleanfalse
interfaceExcludeA comma-separated list of interface names that should be excluded when Felix is resolving host endpoints. The default value ensures that Felix ignores Kubernetes' internal kube-ipvs0 device. If you want to exclude multiple interface names using a single value, the list supports regular expressions. For regular expressions you must wrap the value with /. For example having values /^kube/,veth1 will exclude all interfaces that begin with kube and also the interface veth1.stringstringkube-ipvs0
interfacePrefixThe interface name prefix that identifies workload endpoints and so distinguishes them from host endpoint interfaces. Note: in environments other than bare metal, the orchestrators configure this appropriately. For example our Kubernetes and Docker integrations set the 'cali' value, and our OpenStack integration sets the 'tap' value.stringstringcali
ipipEnabledOptional, you shouldn't need to change this setting as Felix calculates if IPIP should be enabled based on the existing IP Pools. When set, this overrides whether Felix should configure an IPinIP interface on the host. When explicitly disabled in FelixConfiguration, Felix will not clean up addresses from the tunl0 interface (use this if you need to add addresses to that interface and don't want to have them removed).true, false, unsetoptional booleanunset
ipipMTUThe MTU to set on the tunnel device. Zero value means auto-detect. See Configuring MTUintint0
ipsetsRefreshIntervalPeriod at which Felix re-checks the IP sets in the dataplane to ensure that no other process has accidentally broken Calico Enterprise's rules. Set to 0 to disable IP sets refresh.5s, 10s, 1m etc.duration10s
iptablesFilterAllowActionThis parameter controls what happens to traffic that is accepted by a Felix policy chain in the iptables filter table (i.e. a normal policy chain). The default will immediately Accept the traffic. Use Return to send the traffic back up to the system chains for further processing.Accept, ReturnstringAccept
iptablesBackendThis parameter controls which variant of iptables Felix uses. If using Felix on a system that uses the netfilter-backed iptables binaries, set this to nft.Legacy, nftstringautomatic detection
iptablesLockFilePathLocation of the iptables lock file. You may need to change this if the lock file is not in its standard location (for example if you have mapped it into Felix's container at a different path).stringstring/run/xtables.lock
iptablesLockProbeIntervalTime that Felix will wait between attempts to acquire the iptables lock if it is not available. Lower values make Felix more responsive when the lock is contended, but use more CPU.5s, 10s, 1m etc.duration50ms
iptablesLockTimeoutTime that Felix will wait for the iptables lock, or 0, to disable. To use this feature, Felix must share the iptables lock file with all other processes that also take the lock. When running Felix inside a container, this requires the /run directory of the host to be mounted into the cnx-node or calico/felix container.5s, 10s, 1m etc.duration0 (Disabled)
iptablesMangleAllowActionThis parameter controls what happens to traffic that is accepted by a Felix policy chain in the iptables mangle table (i.e. a pre-DNAT policy chain). The default will immediately Accept the traffic. Use Return to send the traffic back up to the system chains for further processing.Accept, ReturnstringAccept
iptablesMarkMaskMask that Felix selects its IPTables Mark bits from. Should be a 32 bit hexadecimal number with at least 8 bits set, none of which clash with any other mark bits in use on the system.netmasknetmask0xffff0000
iptablesNATOutgoingInterfaceFilterThis parameter can be used to limit the host interfaces on which Calico will apply SNAT to traffic leaving a Calico IPAM pool with "NAT outgoing" enabled. This can be useful if you have a main data interface, where traffic should be SNATted and a secondary device (such as the docker bridge) which is local to the host and doesn't require SNAT. This parameter uses the iptables interface matching syntax, which allows + as a wildcard. Most users will not need to set this. Example: if your data interfaces are eth0 and eth1 and you want to exclude the docker bridge, you could set this to eth+stringstring""
iptablesPostWriteCheckIntervalPeriod after Felix has done a write to the dataplane that it schedules an extra read back to check the write was not clobbered by another process. This should only occur if another application on the system doesn't respect the iptables lock.5s, 10s, 1m etc.duration1s
iptablesRefreshIntervalPeriod at which Felix re-checks all iptables state to ensure that no other process has accidentally broken Calico Enterprise's rules. Set to 0 to disable iptables refresh.5s, 10s, 1m etc.duration90s
ipv6SupportIPv6 support for Felixtrue, falsebooleantrue
logFilePathThe full path to the Felix log. Set to none to disable file logging.stringstring/var/log/calico/felix.log
logPrefixThe log prefix that Felix uses when rendering LOG rules.stringstringcalico-packet
logSeverityFileThe log severity above which logs are sent to the log file.Same as logSeveritySysstringInfo
logSeverityScreenThe log severity above which logs are sent to the stdout.Same as logSeveritySysstringInfo
logSeveritySysThe log severity above which logs are sent to the syslog. Set to none for no logging to syslog.Debug, Info, Warning, Error, FatalstringInfo
logDebugFilenameRegexcontrols which source code files have their Debug log output included in the logs. Only logs from files with names that match the given regular expression are included. The filter only applies to Debug level logs.regexstring""
maxIpsetSizeMaximum size for the ipsets used by Felix. Should be set to a number that is greater than the maximum number of IP addresses that are ever expected in a selector.intint1048576
metadataAddrThe IP address or domain name of the server that can answer VM queries for cloud-init metadata. In OpenStack, this corresponds to the machine running nova-api (or in Ubuntu, nova-api-metadata). A value of none (case insensitive) means that Felix should not set up any NAT rule for the metadata path.IPv4, hostname, nonestring127.0.0.1
metadataPortThe port of the metadata server. This, combined with global.MetadataAddr (if not 'None'), is used to set up a NAT rule, from 169.254.169.254:80 to MetadataAddr:MetadataPort. In most cases this should not need to be changed.intint8775
natOutgoingAddressThe source address to use for outgoing NAT. By default an iptables MASQUERADE rule determines the source address which will use the address on the host interface the traffic leaves on.IPV4string""
policySyncPathPrefixFile system path where Felix notifies services of policy changes over Unix domain sockets. This is required only if you're configuring L7 logs, or egress gateways. Set to "" to disable.stringstring""
prometheusGoMetricsEnabledSet to false to disable Go runtime metrics collection, which the Prometheus client does by default. This reduces the number of metrics reported, reducing Prometheus load.booleanbooleantrue
prometheusMetricsEnabledSet to true to enable the experimental Prometheus metrics server in Felix.booleanbooleanfalse
prometheusMetricsHostTCP network address that the Prometheus metrics server should bind to.IPv4, IPv6, Hostnamestring""
prometheusMetricsPortTCP port that the Prometheus metrics server should bind to.intint9091
prometheusProcessMetricsEnabledSet to false to disable process metrics collection, which the Prometheus client does by default. This reduces the number of metrics reported, reducing Prometheus load.booleanbooleantrue
prometheusReporterEnabledSet to true to enable configure Felix to keep count of recently denied packets and publish these as Prometheus metrics. Note that denied packet metrics are independent of the dropActionOverride setting. Specifically, if packets that would normally be denied are being allowed through by a setting of Accept or LogAndAccept, those packets still get counted as denied packets.true, falsebooleanfalse
prometheusReporterPortThe TCP port on which to report denied packet metrics, if prometheusReporterEnabled is set to true.9092
removeExternalRoutesWhether or not to remove device routes that have not been programmed by Felix. Disabling this will allow external applications to also add device routes.boolbooleantrue
reportingIntervalInterval at which Felix reports its status into the datastore. 0 means disabled and is correct for Kubernetes-only clusters. Must be non-zero in OpenStack deployments.5s, 10s, 1m etc.duration30s
reportingTTLTime-to-live setting for process-wide status reports.5s, 10s, 1m etc.duration90s
routeRefreshIntervalPeriod at which Felix re-checks the routes in the dataplane to ensure that no other process has accidentally broken Calico Enterprise's rules. Set to 0 to disable route refresh.5s, 10s, 1m etc.duration90s
ipsecModeControls which mode IPsec is operating on. The only supported value is PSK. An empty value means IPsec is not enabled.PSKstring""
ipsecAllowUnsecuredTrafficWhen set to false, only IPsec-protected traffic will be allowed on the packet paths where IPsec is supported. When set to true, IPsec will be used but non-IPsec traffic will be accepted. In general, setting this to true is less safe since it allows an attacker to inject packets. However, it is useful when transitioning from non-IPsec to IPsec since it allows traffic to flow while the cluster negotiates the IPsec mesh.true, falsebooleanfalse
ipsecIKEAlgorithmIPsec IKE algorithm. Default is NIST suite B recommendation.stringstringaes128gcm16-prfsha256-ecp256
ipsecESPAlgorithmIPsec ESP algorithm. Default is NIST suite B recommendation.stringstringaes128gcm16-ecp256
ipsecLogLevelControls log level for IPsec components. Set to None for no logging.None, Notice, Info, Debug, VerbosestringInfo
ipsecPSKFileThe path to the pre shared key file for IPsec.stringstring""
flowLogsFileEnabledSet to true, enables flow logs. If set to false no flow logging will occur. Flow logs are written to a file flows.log and sent to Elasticsearch. The location of this file can be configured using the flowLogsFileDirectory field. File rotation settings for this flows.log file can be configured using the fields flowLogsFileMaxFiles and flowLogsFileMaxFileSizeMB. Note that flow log exports to Elasticsearch are dependent on flow logs getting written to this file. Setting this parameter to false will disable flow logs.true, falsebooleanfalse
flowLogsFileDirectorySet the directory where flow logs files are stored on Linux nodes. This parameter only takes effect when flowLogsFileEnabled is set to true.stringstring/var/log/calico/flowlogs
flowLogsPositionFilePathSpecify the position of the external pipeline that reads flow logs on Linux nodes. This parameter only takes effect when FlowLogsDynamicAggregationEnabled is set to true.stringstring/var/log/calico/flows.log.pos
flowLogsFileMaxFilesSet the number of log files to keep. This parameter only takes effect when flowLogsFileEnabled is set to true.intint5
flowLogsFileMaxFileSizeMBSet the max size in MB of flow logs files before rotation. This parameter only takes effect when flowLogsFileEnabled is set to true.intint100
flowLogsFlushIntervalThe period, in seconds, at which Felix exports the flow logs.intint300s
flowLogsFileAggregationKindForAllowedHow much to aggregate the flow logs sent to Elasticsearch for allowed traffic. Bear in mind that changing this value may have a dramatic impact on the volume of flow logs sent to Elasticsearch.0-2AggregationKind2
flowLogsFileAggregationKindForDeniedHow much to aggregate the flow logs sent to Elasticsearch for denied traffic. Bear in mind that changing this value may have a dramatic impact on the volume of flow logs sent to Elasticsearch.0-2AggregationKind1
flowLogsFileIncludeServiceWhen set to true, include destination service information in the aggregated flow log. Note that service information will only be included when the flow can be explicitly determined to be bound to a service (e.g. pre-DNAT destination matches a service ClusterIP).true, falsebooleanfalse
flowLogsFileIncludeLabelsWhen set to true, include source and destination endpoint labels in the aggregated flow log. Note that only Kubernetes endpoints or network sets are included; arbitrary networks do not contain labels.true, falsebooleanfalse
flowLogsFileIncludePoliciesWhen set to true, include all policies in the aggregated flow logs that acted upon and matches the flow log traffic.true, falsebooleanfalse
flowLogsEnableNetworkSetsWhen set to true, include an arbitrary network set in the aggregated flow log that matches the IP address of the flow log endpoint.true, falsebooleanfalse
flowLogsCollectProcessInfoWhen set to true, Felix will load the kprobe BPF programs to collect process info.true, falsebooleanfalse
flowLogsCollectTcpStatsWhen set to true, Felix will collect the TCP socket stats.true, falsebooleantrue
flowLogsCollectProcessPathWhen set to true, along with flowLogsCollectProcessInfo, each flow log will include the full path of the executable and the arguments with which the executable was invoked.true, falsebooleanfalse
flowLogsFilePerFlowProcessLimitSpecify the maximum number of flow log entries with distinct process information beyond which process information will be aggregatedintint2
flowLogsFileNatOutgoingPortLimitSpecify the maximum number of distinct post SNAT ports that will appear in the flowLogsintint3
flowLogsFilePerFlowProcessArgsLimitSpecify the maximum number of unique arguments in the flowlogs beyond which process arguments will be aggregatedintint5
flowLogsFileDomainsLimitSpecify the maximum number of top-level domains to include in a flow log. This only applies to source reported flows to destinations external to the cluster.intint5
statsDumpFilePathSpecify the position of the file used for dumping flow log statistics on Linux nodes. Note this is an internal setting that users shouldn't need to modify.stringstring/var/log/calico/stats/dump
routeTableRangedeprecated in favor of RouteTableRanges Calico programs additional Linux route tables for various purposes. RouteTableRange specifies the indices of the route tables that Calico should use.RouteTableRanges""
routeTableRangesCalico programs additional Linux route tables for various purposes. RouteTableRanges specifies a set of table index ranges that Calico should use. Deprecates RouteTableRange, overrides RouteTableRangeRouteTableRanges[{"min": 1, "max": 250}]
routeSyncDisabledSet to true to disable Calico programming routes to local workloads.booleanbooleanfalse
serviceLoopPreventionWhen service IP advertisement is enabled, prevent routing loops to service IPs that are not in use, by dropping or rejecting packets that do not get DNAT'd by kube-proxy. Unless set to "Disabled", in which case such routing loops continue to be allowed.Drop, Reject, DisabledstringDrop
workloadSourceSpoofingControls whether pods can enable source IP address spoofing with the cni.projectcalico.org/allowedSourcePrefixes annotation. When set to Any, pods can use this annotation to send packets from any IP address.Any, DisabledstringDisabled
vxlanEnabledOptional, you shouldn't need to change this setting as Felix calculates if VXLAN should be enabled based on the existing IP Pools. When set, this overrides whether Felix should create the VXLAN tunnel device for VXLAN networking.true, false, unsetoptional booleanunset
vxlanMTUMTU to use for the IPv4 VXLAN tunnel device. Zero value means auto-detect. Also controls NodePort MTU when eBPF enabled.intint0
vxlanMTUV6MTU to use for the IPv6 VXLAN tunnel device. Zero value means auto-detect. Also controls NodePort MTU when eBPF enabled.intint0
vxlanPortPort to use for VXLAN traffic. A value of 0 means "use the kernel default".intint4789
vxlanVNIVirtual network ID to use for VXLAN traffic. A value of 0 means "use the kernel default".intint4096
allowVXLANPacketsFromWorkloadsSet to true to allow VXLAN encapsulated traffic from workloads.booleanbooleanfalse
allowIPIPPacketsFromWorkloadsSet to true to allow IPIP encapsulated traffic from workloads.booleanbooleanfalse
windowsFlowLogsFileDirectorySet the directory where flow logs files are stored on Windows nodes. This parameter only takes effect when flowLogsFileEnabled is set to true.stringstringc:\\TigeraCalico\\flowlogs
windowsFlowLogsPositionFilePathSpecify the position of the external pipeline that reads flow logs on Windows nodes. This parameter only takes effect when FlowLogsDynamicAggregationEnabled is set to true.stringstringc:\\TigeraCalico\\flowlogs\\flows.log.pos
windowsStatsDumpFilePathSpecify the position of the file used for dumping flow log statistics on Windows nodes. Note this is an internal setting that users shouldn't need to modify.stringstringc:\\TigeraCalico\\stats\\dump
windowsDNSCacheFileSpecify the name of the file that Calico uses to preserve learnt DNS information when restarting.stringstringc:\\TigeraCalico\\felix-dns-cache.txt
windowsDNSExtraTTLSpecify extra time in seconds to keep IPs and alias names that are learnt from DNS, in addition to each name or IP's advertised TTL.intint120
windowsManageFirewallRulesConfigure whether or not Felix will program Windows Firewall rules. (to allow inbound access to its own metrics ports) [Default: Disabled]Enabled, DisabledstringDisabled
wireguardEnabledEnable encryption on WireGuard supported nodes in cluster. When enabled, pod to pod traffic will be sent over encrypted tunnels between the nodes.true, falsebooleanfalse
wireguardEnabledV6Enable encryption for IPv6 on WireGuard supported nodes in cluster. When enabled, pod to pod traffic will be sent over encrypted tunnels between the nodes.true, falsebooleanfalse
wireguardInterfaceNameName of the IPv4 WireGuard interface created by Felix. If you change the name and want to clean up the previously configured interface names on each node, this is a manual process. Felix expects the name to end with either .cali or .calico suffix.stringstringwireguard.cali
wireguardInterfaceNameV6Name of the IPv6 WireGuard interface created by Felix. If you change the name and want to clean up the previously configured interface names on each node, this is a manual process. Felix expects the name to end with either .cali or .calico suffix.stringstringwg-v6.cali
wireguardListeningPortPort used by WireGuard tunnels. Felix sets up WireGuard tunnel on each node specified by this port. Available for configuration only in the global FelixConfiguration resource; setting it per host, config-file or environment variable will not work.1-65535int51820
wireguardListeningPortV6Port used by IPv6 WireGuard tunnels. Felix sets up an IPv6 WireGuard tunnel on each node specified by this port. Available for configuration only in the global FelixConfiguration resource; setting it per host, config-file or environment variable will not work.1-65535int51821
wireguardMTUMTU set on the WireGuard interface created by Felix. Zero value means auto-detect. See Configuring MTU.intint0
wireguardMTUV6MTU set on the IPv6 WireGuard interface created by Felix. Zero value means auto-detect. See Configuring MTU.intint0
wireguardRoutingRulePriorityWireGuard routing rule priority value set up by Felix. If you change the default value, set it to a value most appropriate to routing rules for your nodes.1-32765int99
wireguardHostEncryptionEnabledExperimental: Adds host-namespace workload IP's to WireGuard's list of peers. Should not be enabled when WireGuard is enabled on a cluster's control plane node, as networking deadlock can occur.true, falsebooleanfalse
wireguardKeepAliveWireguardKeepAlive controls Wireguard PersistentKeepalive option. Set 0 to disable. [Default: 0]5s, 10s, 1m etc.duration0
xdpRefreshIntervalPeriod at which Felix re-checks the XDP state in the dataplane to ensure that no other process has accidentally broken Calico Enterprise's rules. Set to 0 to disable XDP refresh.5s, 10s, 1m etc.duration90s
xdpEnabledWhen bpfEnabled is false: enable XDP acceleration for host endpoint policies. When bpfEnabled is true, XDP is automatically used for Calico policy where that makes sense, regardless of this setting. [Default: true]true,falsebooleantrue
dnsCacheFileThe name of the file that Felix uses to preserve learnt DNS information when restarting.file namestring/var/run/calico/felix-dns-cache.txt
dnsCacheSaveIntervalThe period, in seconds, at which Felix saves learnt DNS information to the cache file.5s, 10s, 1m etc.duration60s
dnsCacheEpochAn arbitrary number that can be changed, at runtime, to tell Felix to discard all its learnt DNS information.intint0
dnsExtraTTLExtra time to keep IPs and alias names that are learnt from DNS, in addition to each name or IP's advertised TTL.5s, 10s, 1m etc.duration0s
dnsTrustedServersThe DNS servers that Felix should trust. Each entry here must be <ip>[:<port>] - indicating an explicit DNS server IP - or k8s-service:[<namespace>/]<name>[:port] - indicating a Kubernetes DNS service. <port> defaults to the first service port, or 53 for an IP, and <namespace> to kube-system. An IPv6 address with a port must use the square brackets convention, for example [fd00:83a6::12]:5353. Note that Felix (calico-node) will need RBAC permission to read the details of each service specified by a k8s-service:... form.IPs or service namescomma-separated stringsk8s-service:kube-dns
dnsLogsFileEnabledSet to true, enables DNS logs. If set to false no DNS logging will occur. DNS logs are written to a file dns.log and sent to Elasticsearch. The location of this file can be configured using the DNSLogsFileDirectory field. File rotation settings for this dns.log file can be configured using the fields DNSLogsFileMaxFiles and DNSLogsFileMaxFileSizeMB. Note that DNS log exports to Elasticsearch are dependent on DNS logs getting written to this file. Setting this parameter to false will disable DNS logs.true, falsebooleanfalse
dnsLogsFileDirectoryThe directory where DNS logs files are stored. This parameter only takes effect when DNSLogsFileEnabled is true.directorystring/var/log/calico/dnslogs
dnsLogsFileMaxFilesThe number of files to keep when rotating DNS log files. This parameter only takes effect when DNSLogsFileEnabled is true.intint5
dnsLogsFileMaxFileSizeMBThe max size in MB of DNS log files before rotation. This parameter only takes effect when DNSLogsFileEnabled is true.intint100
dnsLogsFlushIntervalThe period, in seconds, at which Felix exports DNS logs.intint300s
dnsLogsFileAggregationKindHow much to aggregate DNS logs. Bear in mind that changing this value may have a dramatic impact on the volume of flow logs sent to Elasticsearch. 0 means no aggregation, 1 means aggregate similar DNS logs from workloads in the same ReplicaSet.0,1int1
dnsLogsFileIncludeLabelsWhether to include client and server workload labels in DNS logs.true, falsebooleantrue
dnsLogsFilePerNodeLimitLimit on the number of DNS logs that can be emitted within each flush interval. When this limit has been reached, Felix counts the number of unloggable DNS responses within the flush interval, and emits a WARNING log with that count at the same time as it flushes the buffered DNS logs.intint0 (no limit)
dnsLogsLatencyIndicates to include measurements of DNS request/response latency in each DNS log.true, falsebooleantrue
dnsPolicyModeDNSPolicyMode specifies how DNS policy programming will be handled.NoDelay, DelayDNSResponse, DelayDeniedPacketDNSPolicyModeDelayDeniedPacket
dnsPolicyNfqueueIDDNSPolicyNfqueueID is the NFQUEUE ID to use for DNS Policy re-evaluation when the domains IP hasn't been programmed to ipsets yet. This value can be changed to avoid conflicts with other users of NFQUEUEs. Used when DNSPolicyMode is DelayDeniedPacket.0-65535int100
dnsPolicyNfqueueSizeDNSPolicyNfqueueID is the size of the NFQUEUE for DNS policy re-evaluation. This is the maximum number of denied packets that may be queued up pending re-evaluation. Used when DNSPolicyMode is DelayDeniedPacket.0-65535int100
dnsPacketsNfqueueIDDNSPacketsNfqueueID is the NFQUEUE ID to use for capturing DNS packets to ensure programming IPSets occurs before the response is released. Used when DNSPolicyMode is DelayDNSResponse.0-65535int101
dnsPacketsNfqueueSizeDNSPacketsNfqueueSize is the size of the NFQUEUE for captured DNS packets. This is the maximum number of DNS packets that may be queued awaiting programming in the dataplane. Used when DNSPolicyMode is DelayDNSResponse.0-65535int100
dnsPacketsNfqueueMaxHoldDurationDNSPacketsNfqueueMaxHoldDuration is the max length of time to hold on to a DNS response while waiting for the dataplane to be programmed. Used when DNSPolicyMode is DelayDNSResponse.5s, 10s, 1m etc.duration3s
bpfEnabledEnable eBPF dataplane mode. eBPF mode has some limitations, see the HOWTO guide for more details.true, falsebooleanfalse
bpfDisableUnprivilegedIf true, Felix sets the kernel.unprivileged_bpf_disabled sysctl to disable unprivileged use of BPF. This ensures that unprivileged users cannot access Calico's BPF maps and cannot insert their own BPF programs to interfere with the ones that Calico Enterprise installs.true, falsebooleantrue
bpfLogLevelIn eBPF dataplane mode, the log level used by the BPF programs. The logs are emitted to the BPF trace pipe, accessible with the command tc exec bpf debug. This is a tech preview feature and subject to change in future releases.Off,Info,DebugstringOff
bpfDataIfacePatternIn eBPF dataplane mode, controls which interfaces Felix should attach BPF programs to catch traffic to/from the external network. This needs to match the interfaces that Calico workload traffic flows over as well as any interfaces that handle incoming traffic to NodePorts and services from outside the cluster. It should not match the workload interfaces (usually named cali...).. This is a tech preview feature and subject to change in future releases.regular expressionstring^(en.*|eth.*|tunl0$)
bpfConnectTimeLoadBalancingEnabledIn eBPF dataplane mode, controls whether Felix installs the connect-time load balancer. In the current release, the connect-time load balancer is required for the host to reach kubernetes services. This is a tech preview feature and subject to change in future releases.true,falsebooleantrue
bpfExternalServiceModeIn eBPF dataplane mode, controls how traffic from outside the cluster to NodePorts and ClusterIPs is handled. In Tunnel mode, packet is tunneled from the ingress host to the host with the backing pod and back again. In DSR mode, traffic is tunneled to the host with the backing pod and then returned directly; this requires a network that allows direct return.Tunnel,DSRstringTunnel
bpfKubeProxyIptablesCleanupEnabledIn eBPF dataplane mode, controls whether Felix will clean up the iptables rules created by the Kubernetes kube-proxy; should only be enabled if kube-proxy is not running. This is a tech preview feature and subject to change in future releases.true,falsebooleantrue
bpfKubeProxyMinSyncPeriodIn eBPF dataplane mode, controls the minimum time between dataplane updates for Felix's embedded kube-proxy implementation.5s, 10s, 1m etc.duration1s
BPFKubeProxyEndpointSlicesEnabledIn eBPF dataplane mode, controls whether Felix's embedded kube-proxy derives its services from Kubernetes' EndpointSlices resources. Using EndpointSlices is more efficient but it requires EndpointSlices support to be enabled at the Kubernetes API server.true,falsebooleanfalse
bpfMapSizeConntrackIn eBPF dataplane mode, controls the size of the conntrack map.intint512000
bpfMapSizeIPSetsIn eBPF dataplane mode, controls the size of the ipsets map.intint1048576
bpfMapSizeNATAffinityIn eBPF dataplane mode, controls the size of the NAT affinity map.intint65536
bpfMapSizeNATFrontendIn eBPF dataplane mode, controls the size of the NAT front end map.intint65536
bpfMapSizeNATBackendIn eBPF dataplane mode, controls the size of the NAT back end map.intint262144
bpfMapSizeRouteIn eBPF dataplane mode, controls the size of the route map.intint262144
bpfPolicyDebugEnabledIn eBPF dataplane mode, controls whether felix will collect policy dump for each interface.true, falsebooleantrue
routeSourceWhere Felix gets is routing information from for VXLAN and the BPF dataplane. The CalicoIPAM setting is more efficient because it supports route aggregation, but it only works when Calico's IPAM or host-local IPAM is in use. Use the WorkloadIPs setting if you are using Calico's VXLAN or BPF dataplane and not using Calico IPAM or host-local IPAM.CalicoIPAM,WorkloadIPsstringCalicoIPAM
mtuIfacePatternPattern used to discover the host's interface for MTU auto-detection.regexstring^((en|wl|ww|sl|ib)[opsvx].*|(eth|wlan|wwan).*)
bpfForceTrackPacketsFromIfacesForces traffic from these interfaces in BPF mode to skip Calico's iptables NOTRACK rule, allowing traffic from those interfaces to be tracked by Linux conntrack. Use only for interfaces that are not used for the Calico fabric, for example, a docker bridge device for non-Calico-networked containers.A list of stringsA list of stringsdocker+
bpfDisableGROForIfacesBPFDisableGROForIfaces is a regular expression that controls which interfaces Felix should disable the Generic Receive Offload [GRO] option. It should not match the workload interfaces (usually named cali...).regexstring""
egressIPSupportDefines three different support modes for egress gateway function. Disabled means egress gateways are not supported. EnabledPerNamespace means egress gateway function is enabled and can be configured on a per-namespace basis (but per-pod egress annotations are ignored). EnabledPerNamespaceOrPerPod means egress gateway function is enabled and can be configured per-namespace or per-pod (with per-pod egress annotations overriding namespace annotations).Disabled,
EnabledPerNamespace,
EnabledPerNamespaceOrPerPod
stringDisabled
egressIPVXLANPortPort to use for egress gateway VXLAN traffic. A value of 0 means "use the kernel default".intint4790
egressIPVXLANVNIVirtual network ID to use for egress gateway VXLAN traffic. A value of 0 means "use the kernel default".intint4097
egressIPRoutingRulePriorityControls the priority value to use for the egress gateway routing rule.intint100
egressGatewayPollIntervalControls the interval at which Felix will poll remote egress gateways to check their health. Only Egress Gateways with a named "health" port will be polled in this way. Egress Gateways that fail the health check will be taken our of use as if they have been deleted.5s, 10s, 1m etc.duration10s
egressGatewayPollFailureCountControls the minimum number of poll failures before a remote Egress Gateway is considered to have failed.intint3
externalNetworkSupportDefines two different support modes for external network function. Disabled means external network is not supported. Enabled means external network support is enabled.Disabled,
Enabled
stringDisabled
externalNetworkRoutingRulePrioritycontrols the priority value to use for the external network routing rule.intint102
captureDirControls the directory where packet capture files are stored.stringstring/var/log/calico/pcap
captureMaxSizeBytesControls the maximum size in bytes for a packet capture file before rotation.intint10000000
captureRotationSecondsControls the rotation period in seconds for a packet capture file.intint3600
captureMaxFilesControls the maximum number rotated packet capture files.intint2

* When dropActionOverride is set to LogAndDrop or LogAndAccept, the syslog entries look something like the following.

May 18 18:42:44 ubuntu kernel: [ 1156.246182] calico-drop: IN=tunl0 OUT=cali76be879f658 MAC= SRC=192.168.128.30 DST=192.168.157.26 LEN=60 TOS=0x00 PREC=0x00 TTL=62 ID=56743 DF PROTO=TCP SPT=56248 DPT=80 WINDOW=29200 RES=0x00 SYN URGP=0 MARK=0xa000000

** Duration is denoted by the numerical amount followed by the unit of time. Valid units of time include nanoseconds (ns), microseconds (µs), milliseconds (ms), seconds (s), minutes (m), and hours (h). Units of time can also be used together e.g. 3m30s to represent 3 minutes and 30 seconds. Any amounts of time that can be converted into larger units of time will be converted e.g. 90s will become 1m30s.


genericXDPEnabled and xdpRefreshInterval are only relevant when bpfEnabled is false and xdpEnabled is true; in other words when XDP is being used to accelerate denial-of-service prevention policies in the iptables dataplane.

When bpfEnabled is true the "xdp" settings all have no effect; in BPF mode the implementation of policy is always accelerated, using the best available BPF technology.

Health Timeout Overrides

Felix has internal liveness and readiness watchdog timers that monitor its various loops. If a loop fails to "check in" within the allotted timeout then Felix will report non-Ready or non-Live on its health port (which is monitored by Kubelet in a Kubernetes system). If Felix reports non-Live, this can result in the Pod being restarted.

In Kubernetes, if you see the calico-node Pod readiness or liveness checks fail intermittently, check the calico-node Pod log for a log from Felix that gives the overall health status (the list of components will depend on which features are enabled):

+---------------------------+---------+----------------+-----------------+--------+
| COMPONENT | TIMEOUT | LIVENESS | READINESS | DETAIL |
+---------------------------+---------+----------------+-----------------+--------+
| CalculationGraph | 30s | reporting live | reporting ready | |
| FelixStartup | 0s | reporting live | reporting ready | |
| InternalDataplaneMainLoop | 1m30s | reporting live | reporting ready | |
+---------------------------+---------+----------------+-----------------+--------+

If some health timeouts show as "timed out" it may help to apply an override using the healthTimeoutOverrides field:

...
spec:
healthTimeoutOverrides:
- name: InternalDataplaneMainLoop
timeout: "5m"
- name: CalculationGraph
timeout: "1m30s"
...

A timeout value of 0 disables the timeout.

ProtoPort

FieldDescriptionAccepted ValuesSchema
portThe exact port match0-65535int
protocolThe protocol matchtcp, udp, sctpstring
netThe CIDR matchany valid CIDR (e.g. 192.168.0.0/16)string

Keep in mind that in the following example, net: "" and net: "0.0.0.0/0" are processed as the same in the policy enforcement.

  ...
spec:
failsafeInboundHostPorts:
- net: "192.168.1.1/32"
port: 22
protocol: tcp
- net: ""
port: 67
protocol: udp
failsafeOutboundHostPorts:
- net: "0.0.0.0/0"
port: 67
protocol: udp
...

AggregationKind

ValueDescription
0No aggregation
1Aggregate all flows that share a source port on each node
2Aggregate all flows that share source ports or are from the same ReplicaSet on each node

DNSPolicyMode

ValueDescription
DelayDeniedPacketFelix delays any denied packet that traversed a policy that included egress domain matches, but did not match. The packet is released after a fixed time, or after the destination IP address was programmed.
DelayDNSResponseFelix delays any DNS response until related IPSets are programmed. This introduces some latency to all DNS packets (even when no IPSet programming is required), but it ensures policy hit statistics are accurate. This is the recommended setting when you are making use of staged policies or policy rule hit statistics.
NoDelayFelix does not introduce any delay to the packets. DNS rules may not have been programmed by the time the first packet traverses the policy rules. Client applications need to handle reconnection attempts if initial connection attempts fail. This may be problematic for some applications or for very low DNS TTLs.

On Windows, or when using the eBPF dataplane, this setting is ignored and NoDelay is always used.

A linux kernel version of 3.13 or greater is required to use DelayDNSResponse. For earlier kernel versions, this value is modified to DelayDeniedPacket.

RouteTableRange

The RouteTableRange option is now deprecated in favor of RouteTableRanges.

FieldDescriptionAccepted ValuesSchema
minMinimum index to use1-250int
maxMaximum index to use1-250int

RouteTableRanges

RouteTableRanges is a list of RouteTableRange objects:

FieldDescriptionAccepted ValuesSchema
minMinimum index to use1 - 4294967295int
maxMaximum index to use1 - 4294967295int

Each item in the RouteTableRanges list designates a range of routing tables available to Calico. By default, Calico will use a single range of 1-250. If a range spans Linux's reserved table range (253-255) then those tables are automatically excluded from the list. It's possible that other table ranges may also be reserved by third-party systems unknown to Calico. In that case, multiple ranges can be defined to target tables below and above the sensitive ranges:

 target tables 65-99, and 256-1000, skipping 100-255
calicoctl patch felixconfig default --type=merge -p '{"spec":{"routeTableRanges": [{"min": 65, "max": 99}, {"min": 256, "max": 1000}] }}

Note, for performance reasons, the maximum total number of routing tables that Felix will accept is 65535 (or 2*16).

Specifying both the RouteTableRange and RouteTableRanges arguments is not supported and will result in an error from the api.

AWS IAM Role/Policy for source-destination-check configuration

Setting awsSrcDstCheck to Disable will automatically disable source-destination-check on EC2 instances in a cluster, provided necessary IAM roles and policies are set. One of the policies assigned to IAM role of cluster nodes must contain a statement similar to the following:

{
"Effect": "Allow",
"Action": [
"ec2:DescribeInstances",
"ec2:ModifyNetworkInterfaceAttribute"
],
"Resource": "*"
}

If there are no policies attached to node roles containing the above statement, attach a new policy. For example, if a node role is test-cluster-nodeinstance-role, click on the IAM role in AWS console. In the Permission policies list, add a new inline policy with the above statement to the new policy JSON definition. For detailed information, see AWS documentation.

For an EKS cluster, the necessary IAM role and policy is available by default. No further actions are needed.

Supported operations

Datastore typeCreateDeleteDelete (Global default)UpdateGet/ListNotes
Kubernetes API serverYesYesNoYesYes