allowed / denied / bypassed), the container’s attribution (agent, project, container_id), the destination 4-tuple, and the resolved domain when DNS context is available.
The headline use is bypass-mode forensic coverage. The firewall’s bypass switch (clawker firewall bypass <duration> --agent <name>) intentionally short-circuits enforcement so the operator can perform supervised exploration without rule churn. Before netlogger, bypassed traffic flowed without leaving an enforcement record — the so-called “forensic black hole” of bypass mode. netlogger emits an action=bypassed record at the same decision points that would have emitted allowed/denied, so an audit trail exists for every bypass window without changing enforcement semantics.
Record Shape
Each record is an OTel log emitted on the trusted infra OTLP lane with:service.name = ebpf-egress(distinct fromclawkercpso retention + volume profile are independent)event.name = ebpf.egress.connect/ebpf.egress.sendmsg/ebpf.egress.sock_create(per-emit-site so dashboards can filter by record kind without inspecting flag bits)body = "ebpf egress"severity = INFO
dst_ip / dst_port / dst_host are omitted when their source value is absent so operators can partition via _exists_:attributes.<key> in OS Discover):
The BPF event struct is fixed-size with explicit padding (48 bytes; layout asserted at compile time). Operators filter and aggregate at dashboard/query time — the emitter never decides which fields are “interesting” for a given verdict, except for the
sock_create carve-out above where BPF carries no destination.
Where Records Land
Records flow:clawker-opensearch-bootstrap one-shot service every time clawker monitor up runs. The retention policy (default 7 days, throwaway-stack semantics) auto-attaches via the same ISM policy that covers the other clawker indices. Cross-index queries against clawkercp,clawker-envoy,clawker-coredns,clawker-ebpf-egress work out of the box — ingest_source is stamped on every record for filtering.
Per-Connection Bytes and Duration
Not in this stream. netlogger records the decision — the moment the kernel approved, denied, or bypassed an outbound connection. Byte counts and durations belong to the L7 proxy lifecycle, not the decision point. Foraction=allowed records, the matching Envoy access log carries bytes_sent, bytes_received, duration_ms. Operators pivot from a netlogger record to the corresponding Envoy record by 5-tuple at query time. For action=denied records there are no bytes to record — no traffic flowed. For action=bypassed records, only the netlogger record exists — Envoy and CoreDNS enforcement are skipped under bypass by design.
Sock_ops-based per-connection byte tracking inside BPF is not on this stream’s roadmap. It would double the BPF surface area, leave UDP/connectionless flows without an analogous signal, and overlap with Envoy’s access-log emission for the cases where it matters.
Domain Resolution
dst_host is populated for every record whose destination IP came from a dnsbpf-resolved A record under a firewall-allowed zone. The translation is control-plane-driven: the control plane allocates a sticky route identity per firewall destination, the BPF dns_cache map stores {identity, expire_ts} keyed by IPv4, and netlogger reads the identity → destination table directly from the allocator on a 5-second refresh tick. dnsbpf writes the same CP-allocated identity (delivered per zone via its Corefile directive), so the two sides agree by construction — attribution is a direct read of the allocation, not a hash inversion.
dst_host will be empty when:
- The destination IP was reached without DNS resolution (direct-IP
connect). - The IP was resolved through a path other than the managed CoreDNS (e.g.,
/etc/hostsentry inside the agent container). - A rule was removed and netlogger hasn’t yet refreshed (worst case: 5 seconds of stale records on the previously-allowed domain).
Reliability
netlogger is engineered to fail open with respect to the firewall — enforcement runs whether or not netlogger is healthy.- BPF token-bucket rate limiter keyed by
cgroup_id(burst 64, refill 64 tokens/100ms ⇒ ~640 records/sec/cgroup ceiling). A misbehaving container cannot monopolize the ringbuf; throttled events are counted inratelimit_drops, keyed by the noisy cgroup. - Kernel-fault drop counter (
events_drops, PERCPU_ARRAY) bumps whenbpf_ringbuf_reservereturns NULL on a full buffer — distinct from rate-limit drops so the operator response is different (ringbuf size vs. noisy-agent triage). - Userspace queue between the ringbuf reader and the processor is bounded with drop-newest semantics; the reader never blocks on the consumer. Drops are counted in
clawker_netlogger_queue_dropped_total(Prom counter declared; scrape exposure is not wired). - Circuit breaker wraps the OTLP exporter: three consecutive
Export()failures permanently trip the breaker for the rest of the CP lifetime. Records drop on the floor afterward; the BatchProcessor queue drains via the SDK’s own drop-oldest path. No background reconnect — telemetry availability is binary per-CP-lifetime by design. Operator response: restart CP after fixing the collector. - Preflight TLS dial runs at CP boot with a 20-second deadline against the configured OTLP endpoint. Failure degrades netlogger to a no-op for the rest of the CP lifetime and emits
event=netlogger_unavailable(warn for “no endpoint configured”, error for actual failures like cert problems or unreachable collector). The firewall, AdminService, agent dispatch, and registry are unaffected — netlogger’s failure is contained.
Trust Lane
netlogger emits on the trusted infra lane — the same OTLP/gRPC + mTLS path the CP zerolog bridge, the Envoy access logger, and the CoreDNS otel plugin use. Identity reuse:- Cert: per-handshake ephemeral leaf minted by
otelcerts.Service(LoadTLSConfig("netlogger")), chained through the infra intermediate CA — not the CLI root. - Endpoint: the collector’s
otlp/infrareceiver atotel-collector:4319(not the unauthenticated host CLI receiver). The control plane, Envoy, and CoreDNS reach it over an internal telemetry network that agents never join. The receiver is not published on the host. - The OTLP endpoint must be
https://(or bare host:port). A plaintexthttp://endpoint is rejected at boot — pushing infra telemetry over plaintext would smuggle records onto the unauthenticated host CLI receiver, defeating the trust-lane separation.
otlp/agents receiver, ingest_source=cp_gateway). Agent processes export to a loopback OTLP/HTTP endpoint served by clawkerd inside the container; clawkerd carries each batch to the control plane over its mTLS leaf, and the control plane overwrites the project, agent, and container_id resource attributes with the identity it verified against the agent registry. When the monitoring stack is down the control plane drops those batches after three failed forwards and probes once a minute, so telemetry resumes when the stack comes up.
Agent containers cannot forge service.name=ebpf-egress records onto the trusted index — they don’t hold a leaf chained through the infra intermediate, so the receiver’s TLS handshake fails the chain check. The strict-directive promise (every field on every record, no discretion) only delivers if the records actually originate from the CP — the mTLS boundary is what makes that promise enforceable.
Configuration
netlogger inherits its endpoint from the standardOTEL_EXPORTER_OTLP_ENDPOINT resolution path used by the CP zerolog bridge — when clawker monitor up is running, the CP boot sequence wires the collector’s infra receiver automatically. No netlogger-specific knobs ship; the BatchProcessor sizing, retry cap (10s vs. SDK default 1 min), and circuit-breaker threshold are CP-level constants.
To point netlogger at a custom collector (for a centralized SIEM, or to bypass the local stack entirely), override OTEL_EXPORTER_OTLP_ENDPOINT in the CP container’s environment and ensure the receiver presents a server cert chained through the infra intermediate. Plaintext endpoints are rejected by design — see Trust Lane above.
Current Limitations
- Prom counters not scraped — the
clawker_netlogger_*counters are declared but not wired into a/metricsendpoint. The structured CP log surface is the operational signal for throughput, queue drops, parse errors, and OTLP export success/error. Kernel-side drop counters (events_drops,ratelimit_drops) live on the firewall/eBPF subsystem surface — they are subsystem health, not security telemetry, and intentionally don’t ride on the netlogger OTel stream.
See Also
- Firewall — the source of every record on this stream (decision points, bypass mechanics, rule lifecycle).
- Monitoring — the OpenSearch + OpenSearch Dashboards + Prometheus stack that hosts the
clawker-ebpf-egressindex.