DNS policy

Big picture

Use domain names to allow traffic to destinations outside of a cluster.

Value

Using domain names in policies to identify services outside of the cluster is often operationally simpler and more robust than using IP addresses. In particular, they are useful when an external service does not map to a well known set of static IP addresses.

Domain names can include a wildcard (*), making it easier to manage large numbers of domains/subdomains.

Concepts

Allowed egress domains

Using domain names in policy rules is limited to only egress allow rules. Calico Cloud allows connections only to IP addresses returned from DNS lookups to trusted DNS servers. The supported DNS types are: A, AAAA, and CNAME records. The domain name must be an exact match; for example, google.com is treated as distinct from www.google.com.

note

Kubernetes labels provide a similar convenience for services within the cluster. Calico Enterprise does not support using domain names for services within the cluster. Use Kubernetes labels for services within the cluster.

Domain name matching

When a configured domain name has no wildcard (*), it matches exactly that domain name. For example:

• microsoft.com
• tigera.io

With a single asterisk in any part of the domain name, it matches 1 or more path components at that position. For example:

• *.google.com matches www.google.com and www.ipv6.google.com, but not google.com
• www.*.com matches www.sun.com and www.apple.com, but not www.com
• update.*.mycompany.com matches update.tools.mycompany.com, update.secure.suite.mycompany.com, and so on

Not supported are:

• Multiple wildcards in the same domain, for example: *.*.mycompany.com
• Asterisks that are not the entire component, for example: www.g*.com
• More general wildcards, such as regular expressions

Workload and host endpoints

Policy with domain names can be enforced on workload or host endpoints. When a policy with domain names applies to a workload endpoint, it allows that workload to connect out to the specified domains. When policy with domain names applies to a host endpoint, it allows clients directly on the relevant host (including any host-networked workloads) to connect out to the specified domains.

Trusted DNS servers

Calico Cloud trusts DNS information only from its list of DNS trusted servers. Using trusted DNS servers to back domain names in policy, prevents a malicious workload from using IPs returned by a fake DNS server to hijack domain names in policy rules.

By default, Calico Cloud trusts the Kubernetes cluster’s DNS service (kube-dns or CoreDNS). For workload endpoints, these out-of-the-box defaults work with standard Kubernetes installs, so normally you won’t change them. For host endpoints you will need to add the IP addresses that the cluster nodes use for DNS resolution.

Before you begin

Not supported

DNS policy is not supported at egress of egress gateway pods. Domain-based rules will either never match in that hook, or, they may match intermittently. Intermittent matches occur when a pod on the same node as the egress gateway pod happens to make a matching DNS query. This is because the DNS-to-IP cache used to render the policy is shared node-wide.

How to

You can specify allowed domain names directly in a global network policy or namespaced network policy, or specify domain names in a global network set (and then reference the global network set in a global network policy).

• Use domain names in a global network policy
• Use domain names in a namespaced network policy
• Use domain names in a global network set, reference the set in a global network policy

Best practice

Use a global network set when the same set of domains needs to be referenced in multiple policies, or when you want the allowed destinations to be a mix of domains and IPs from global network sets, or IPs from workload endpoints and host endpoints. By using a single destination selector in a global network set, you can potentially match all of these resources.

Use domain names in a global network policy

In this method, you create a GlobalNetworkPolicy with egress rules with action: Allow and a destination.domains field specifying the domain names to which egress traffic is allowed.

In the following example, the first rule allows DNS traffic, and the second rule allows connections outside the cluster to domains api.alice.com and *.example.com (which means <anything>.example.com, such as bob.example.com).

apiVersion: projectcalico.org/v3
kind: GlobalNetworkPolicy
metadata:
  name: allow-egress-to-domains
spec:
  order: 1
  selector: my-pod-label == 'my-value'
  types:
    - Egress
  egress:
    - action: Allow
      protocol: UDP
      destination:
        ports:
          - 53
          - dns
    - action: Allow
      destination:
        domains:
          - api.alice.com
          - '*.example.com'

Use domain names in a namespaced network policy

In this method, you create a NetworkPolicy with egress rules with action: Allow and a destination.domains field specifying the domain names to which egress traffic is allowed.

In the following example, the first rule allows DNS traffic, and the second rule allows connections outside the cluster to domains api.alice.com and *.example.com (which means <anything>.example.com, such as bob.example.com).

apiVersion: projectcalico.org/v3
kind: NetworkPolicy
metadata:
  name: allow-egress-to-domains
  namespace: rollout-test
spec:
  order: 1
  selector: my-pod-label == 'my-value'
  types:
    - Egress
  egress:
    - action: Allow
      protocol: UDP
      destination:
        ports:
          - 53
          - dns
    - action: Allow
      destination:
        domains:
          - api.alice.com
          - '*.example.com'

The difference between this and the GlobalNetworkPolicy example is that this namespaced NetworkPolicy can only grant egress access, to the specified domains, to workload endpoints in the rollout-test namespace.

Use domain names in a global network set

In this method, you create a GlobalNetworkSet with the allowed destination domain names in the allowedEgressDomains field. Then, you create a GlobalNetworkPolicy with a destination.selector that matches that GlobalNetworkSet.

In the following example, the allowed egress domains (api.alice.com and *.example.com) are specified in the GlobalNetworkSet.

apiVersion: projectcalico.org/v3
kind: GlobalNetworkSet
metadata:
  name: allowed-domains-1
  labels:
    color: red
spec:
  allowedEgressDomains:
    - api.alice.com
    - '*.example.com'

Then, you reference the global network set in a GlobalNetworkPolicy using a destination label selector.

apiVersion: projectcalico.org/v3
kind: GlobalNetworkPolicy
metadata:
  name: allow-egress-to-domain
spec:
  order: 1
  selector: my-pod-label == 'my-value'
  types:
    - Egress
  egress:
    - action: Allow
      destination:
        selector: color == 'red'

Use domain names in a network set

In this method, you create a NetworkSet with the allowed destination domain names in the allowedEgressDomains field. Then, you create a NetworkPolicy with a destination.selector that matches that NetworkSet.

In the following example, the allowed egress domains (api.alice.com and *.example.com) are specified in the NetworkSet.

apiVersion: projectcalico.org/v3
kind: NetworkSet
metadata:
  name: allowed-domains-1
  namespace: rollout-test
  labels:
    color: red
spec:
  allowedEgressDomains:
    - api.alice.com
    - '*.example.com'

Then, you reference the network set in a NetworkPolicy using a destination label selector.

apiVersion: projectcalico.org/v3
kind: NetworkPolicy
metadata:
  name: allow-egress-to-domain
  namespace: rollout-test
spec:
  order: 1
  selector: my-pod-label == 'my-value'
  types:
    - Egress
  egress:
    - action: Allow
      destination:
        selector: color == 'red'

note

Newly initialized pods sometimes fail to connect to domains that are allowed by a DNS policy (staged or enforced). In these cases, the pod tries to connect to a domain before Calico Cloud has finished processing the DNS policy that allows the connection. As soon as the processing is complete, the pod is able to connect.

To avoid these failed connections, you can add the following to your FelixConfiguration resource:

...
spec:
  DNSPolicyMode: DelayDNSResponse
...

For more information, see DNSPolicyMode.

Additional resources

To change the default DNS trusted servers, use the DNSTrustedServers parameter.

For more detail about the relevant resources, see GlobalNetworkSet, GlobalNetworkPolicy, NetworkPolicy and NetworkSet.