Log aggregation

Big picture

Calico Cloud aggregates log entries before writing them to disk. Rather than emitting one record per event, it merges events that share the same values across a chosen set of fields — the aggregation key — into a single entry that carries a count of how many events were merged.

Aggregation trades detail for volume:

• Including a field makes the log more granular: more entries, fewer events per entry.
• Excluding a field collapses events that differ only in that field into one entry and increments its count for each merged event.

The same mechanism applies across Calico Cloud log types (flow, DNS, L7); only the available fields differ. The example below uses L7 (HTTP) traffic to make the behavior concrete.

Aggregation playground

1. Observed traffic (7 requests)

Consider these 7 requests as the raw L7 events captured between workloads in your cluster — one row per request, exactly as seen on the wire before any aggregation is applied. This is the input that the aggregation key collapses into log entries.

src_name_aggr	dest_name_aggr	method	url	response_code
frontend	cart	GET	/cart	200
frontend	cart	GET	/cart	404
frontend	cart	GET	/items	200
frontend	cart	POST	/cart	200
orders	pay	POST	/charge	200
edge	pay	POST	/charge	200
orders	canary	POST	/charge	200

2. Aggregation key fields

Include or exclude each field from the aggregation key.

Source infoDestination infoHTTP methodURLResponse code

3. Configuration

The FelixConfiguration produced by the selected fields.

apiVersion: projectcalico.org/v3
kind: FelixConfiguration
metadata:
  name: default
spec:
  l7LogsFileEnabled: true
  l7LogsFileAggregationSourceInfo: IncludeL7SourceInfo
  l7LogsFileAggregationDestinationInfo: IncludeL7DestinationInfo
  l7LogsFileAggregationHTTPMethod: IncludeL7HTTPMethod
  l7LogsFileAggregationTrimURL: IncludeL7FullURL
  l7LogsFileAggregationResponseCode: IncludeL7ResponseCode

4. Resulting log (7 entries)

What is written to the log file: one JSON entry per unique combination of the included fields, with count requests merged into it. Excluded fields collapse to "-".

{"count":1,"src_name_aggr":"frontend","dest_name_aggr":"cart","method":"GET","url":"/cart","response_code":"200"}
{"count":1,"src_name_aggr":"frontend","dest_name_aggr":"cart","method":"GET","url":"/cart","response_code":"404"}
{"count":1,"src_name_aggr":"frontend","dest_name_aggr":"cart","method":"GET","url":"/items","response_code":"200"}
{"count":1,"src_name_aggr":"frontend","dest_name_aggr":"cart","method":"POST","url":"/cart","response_code":"200"}
{"count":1,"src_name_aggr":"orders","dest_name_aggr":"pay","method":"POST","url":"/charge","response_code":"200"}
{"count":1,"src_name_aggr":"edge","dest_name_aggr":"pay","method":"POST","url":"/charge","response_code":"200"}
{"count":1,"src_name_aggr":"orders","dest_name_aggr":"canary","method":"POST","url":"/charge","response_code":"200"}

Aggregation fields

The fields that control the aggregation key, including their accepted values and defaults, are documented in the Felix Configuration reference.