Clawker — Self-Hosted AI Coding Agent Sandbox

Clawker includes an optional monitoring stack that collects telemetry from your agents. It provides dashboards for logs and metrics — giving you visibility into what every agent is doing, how much it costs, what tools it’s calling, and whether it’s making progress. The firewall egress, DNS, CLI, and control-plane log streams cover every harness; agent-side session telemetry (cost, tokens, tool calls) rides Claude Code’s OpenTelemetry pipeline, so it flows when running the claude harness.

What You Get

The stack ingests logs into OpenSearch and exposes them through OpenSearch Dashboards for ad-hoc querying and dashboarding. Prometheus backs metrics with its own UI. You can see per-agent and per-project activity for:

Cost and token usage — API call costs, input/output tokens, rate limiting
Coding activity — file edits, tool invocations, command executions
Tool usage — which tools the agent calls, how often, and how long they take
Session detail — full event timeline for individual agent sessions
Egress traffic — Envoy access logs and CoreDNS query logs from the firewall stack (when enabled)

Architecture

The monitoring stack runs as four long-lived Docker Compose services on the clawker-net network, plus one short-lived bootstrap container that runs at bring-up (and on monitor reload) to preconfigure the cluster:

Claude Code (in container)                Control plane (host)
    |                                          |
    | OTLP (HTTP/gRPC) — agent lane            | OTLP/gRPC + mTLS — trusted infra lane
    | logs, metrics, traces                    | zerolog bridge, Envoy access logs,
    |                                          | CoreDNS query logs, netlogger ebpf-egress
    v                                          v
OTEL Collector ──┬──> OpenSearch (logs + traces) ──> OpenSearch Dashboards
                 └──> Prometheus (metrics + spanmetrics) ──> Prometheus UI
                                                       \\
                                                        \\── (also read by OpenSearch Dashboards
                                                              via a preconfigured datasource)

Claude Code emits logs, metrics, AND traces. Span export gates on the CLAUDE_CODE_ENHANCED_TELEMETRY_BETA=1 flag (baked into the image alongside CLAUDE_CODE_ENABLE_TELEMETRY=1). The OTEL Collector’s spanmetrics connector also produces RED (rate/error/duration) metrics from incoming spans and feeds them into the same Prometheus pipeline.

Services

Service	Image	Default Port	Purpose
OTEL Collector	`otel/opentelemetry-collector-contrib`	4318 (HTTP), 4317 (gRPC)	Receives telemetry from agents and routes it to backends
OpenSearch	`opensearchproject/opensearch`	9200	Stores telemetry across the `claude-code` (Claude Code OTLP), `clawker-cli` (host CLI), `clawkercp` (clawkercp mTLS push), `clawker-envoy` (firewall HTTP/TLS/TCP access logs), `clawker-coredns` (firewall DNS query logs), `clawker-ebpf-egress` (eBPF per-decision egress events) log indices, plus the `traces`/`clawker` (SS4O) trace index
OpenSearch Dashboards	`opensearchproject/opensearch-dashboards`	5601	UI for querying logs, dashboards, and (via the preconfigured `clawker_prometheus` direct-query datasource) Prometheus metrics
Prometheus	`prom/prometheus`	9090	Time-series metrics storage and UI
clawker-opensearch-bootstrap	`curlimages/curl`	—	One-shot init container. Runs after OpenSearch + Prometheus are reachable, before the collector starts; applies templates, ISM, datasource, workspace, saved objects, then exits

All containers are pre-configured with labels (dev.clawker.purpose=monitoring) and attached to the clawker-net network. The OpenSearch security plugin is disabled by default for local development — Dashboards is reachable at http://localhost:5601 with no login required. Set OPENSEARCH_JAVA_OPTS via the opensearch_heap_mb setting if you need more heap.

Service Hostnames

The monitoring services have fixed, stable hostnames on clawker-net. Agent containers resolve otel-collector and prometheus through the firewall’s DNS, since agents push telemetry through the collector and never query OpenSearch directly. opensearch-node and opensearch-dashboards are deliberately left out of that forwarded set — other containers on clawker-net that do need them reach them via Docker’s embedded resolver.

How Agents Connect

Infrastructure telemetry (egress, DNS, CLI, control plane) is emitted host-side for every harness. Agent-session OTEL endpoints are baked into images whose harness bundle declares them — the claude bundle ships a full OTLP env block. At container creation, Clawker checks whether the monitoring stack is running and conditionally enables telemetry:

OTEL_EXPORTER_OTLP_ENDPOINT — single base URL pointing at the collector’s OTLP/HTTP listener (baked into image). The Claude Code SDK appends /v1/metrics, /v1/logs, /v1/traces per signal
OTEL_METRICS_EXPORTER / OTEL_LOGS_EXPORTER / OTEL_TRACES_EXPORTER — baked to otlp, overridden to none at container creation when the otel-collector container is not detected as running on clawker-net
CLAUDE_CODE_ENABLE_TELEMETRY — always 1 (baked into the image; not conditionally toggled — the exporter overrides above are what turn telemetry off)
CLAUDE_CODE_ENHANCED_TELEMETRY_BETA — set to 1 (gates Claude Code’s span export; both this and CLAUDE_CODE_ENABLE_TELEMETRY must be 1 for traces to flow)
OTEL_RESOURCE_ATTRIBUTES — tags with project= and agent= for filtering

Key behavior: Telemetry is only active when the monitoring stack is running at the time the container is created. If the stack isn’t up, Clawker explicitly disables telemetry so agents don’t waste time attempting exports to an unreachable collector. You can force-enable telemetry via agent.env in your .clawker.yaml even when the stack is down, but Claude Code won’t retry failed collector connections. Start the monitoring stack before starting your agents to ensure telemetry is captured.

Egress Traffic Visibility

When the firewall is enabled, Envoy and CoreDNS access logs flow into OpenSearch alongside agent telemetry, plus a per-decision-point eBPF event stream that closes the bypass-mode forensic gap.

Envoy access logs (clawker-envoy) — server.address (host the client asked for), network.peer.address/network.peer.port (post-resolution upstream), network.transport/network.protocol.name/network.protocol.version, tls.established/tls.protocol.version/tls.cipher, action (clawker firewall verdict: allowed/denied), response_code/response_code_details (distinguishes Envoy direct_response from upstream via_upstream), duration breakdown (req_duration_ms/resp_duration_ms/resp_tx_duration_ms/duration_ms), and byte counters. Query action:denied for clawker firewall blocks; response_code:>=400 AND action:allowed for upstream errors.
CoreDNS query logs (clawker-coredns) — queried domain, qtype (A/AAAA), rcode (NOERROR / NXDOMAIN), resolution duration. rcode:NXDOMAIN is ambiguous: it covers both DNS-layer firewall blocks (non-allowlisted host) and legitimate misses inside allowed zones (typo, missing record). CoreDNS makes no per-query allow/deny decision — correlate domain against the allowlist, or use the eBPF egress stream below (explicit action:denied) for the authoritative block signal.
eBPF egress events (clawker-ebpf-egress) — one record per firewall decision, including bypass-mode traffic that skips Envoy and CoreDNS entirely, so bypass windows still leave a complete audit trail. See Egress Observability for the full record shape and per-attribute reference.

Monitoring Extensions

What the stack observes beyond the core firewall and control-plane telemetry is contributed by monitoring extensions your project selects. An extension ships the OpenSearch indices, ingest pipelines, and dashboards for one telemetry stream; clawker’s built-in claude-code extension is what lands Claude Code session telemetry (cost, tokens, tool calls) and its two dashboards. Select the extensions a project contributes under monitor.extensions in clawker.yaml:

# clawker.yaml
monitor:
  extensions:
    - claude-code

The default selection is the built-in claude-code extension; set an explicit empty list (extensions: []) to opt out of all monitoring. clawker monitor up is bring-up only: it seeds the current project’s selection when it starts the stack, and exits untouched when the stack is already running — there is no host-side registry, and the collector routes from the union of every extension ever seeded. To apply an extension change to a running stack, run clawker monitor reload (seeds the selection, re-renders the config, and recreates the collector). See Monitoring Extensions for selection semantics and Authoring monitoring extensions for writing your own.

Setup

1. Initialize Configuration

clawker monitor init

This scaffolds configuration files in your data directory (~/.local/share/clawker/monitor/):

compose.yaml — Docker Compose definition for all services (four long-lived + one-shot bootstrap)
otel-config.yaml — OpenTelemetry Collector pipeline configuration
prometheus.yaml — Prometheus scrape targets

Use --force to regenerate files if they already exist.

2. Start the Stack

clawker monitor up

The stack runs in the background. Service URLs are printed on startup:

OpenSearch Dashboards: http://localhost:5601
OpenSearch API:        http://localhost:9200
Prometheus:            http://localhost:9090

3. Run Agents

Start agents as usual — telemetry flows automatically:

clawker run -it --agent dev @

4. Explore in OpenSearch Dashboards

Open OpenSearch Dashboards at http://localhost:5601. From the splash / welcome screen, under the Analytics panel on the far right, click Clawker to enter the auto-created workspace. Inside the workspace, the left navbar has an Explore section — click Logs or Metrics to browse. The preconfigured index patterns (claude-code, clawker-cli, clawkercp, clawker-envoy, clawker-coredns, clawker-ebpf-egress) and the clawker_prometheus direct-query datasource are already wired, so logs and Prometheus metrics are both reachable from inside OSD. Raw Prometheus is also still available at http://localhost:9090 if you prefer it. Dashboards. Which dashboards are preinstalled depends on the monitoring extensions your project selects. Clawker Networking is core firewall telemetry and is always present; the two Claude Code dashboards come from the built-in claude-code extension (when selected):

Claude Code Cost & Usage — KPI strip over Claude Code sessions, cost, input/output/cache tokens. Sourced from Prometheus counters. (From the claude-code extension.)
Claude Code Activity — Claude Code security activity audit: prompts, tooling, code editing, permissions, hooks, MCP, plugins — totals, distributions, and full event tables. (From the claude-code extension.)
Clawker Networking — per-source firewall telemetry: three event-stream panels (Envoy access logs, CoreDNS query log, eBPF egress decisions) plus three verdict pies (Envoy action, CoreDNS rcode, eBPF action). Verdict pies use each source’s own field — values not normalized so each component’s truth stays visible (e.g. CoreDNS denies show as NXDOMAIN, not collapsed into denied). Red slices = denials. Sourced from OS log indices; no Prometheus counters for network telemetry today.

Build your own off the index patterns + Prometheus datasource for anything not covered, or ship them as a monitoring extension.

Prometheus label naming: use kind instead of typeClaude Code’s metrics docs reference a type label on a few counters (claude_code_token_usage_tokens_total, claude_code_active_time_seconds_total, claude_code_lines_of_code_count_total). The monitoring stack stores these as kind instead — same values (input/output/cacheRead/cacheCreation, cli/user, added/removed), only the key is renamed.Why: a known bug in the current OpenSearch SQL plugin’s direct-query Prometheus connector causes the OSD Explore “Metrics” UI to error out (missing type id property 'type') on any Prom series that carries a label literally named type. The OTel collector renames the label at ingest so the UI works. The native Prometheus UI at http://localhost:9090 is unaffected either way — type would work there if it existed.When writing your own PromQL or saved searches against these metrics, query with kind= rather than type=.

Bootstrap

The stack is preconfigured at bring-up (and again on every monitor reload). A one-shot clawker-opensearch-bootstrap compose service runs after OpenSearch reports healthy, applies:

Component templates with shared mappings (@timestamp as date, service.name / ingest_source / project / agent as keyword)
Index templates with per-source field mappings for each log index plus the SS4O traces mapping (so dynamic mapping never locks the wrong type at first ingest)
ISM retention policy (7-day rollover-to-delete, auto-attached via ism_template.index_patterns)
Empty index pre-creation for the log indices so OSD Discover and dashboards don’t error on initial load before the first record arrives
clawker_prometheus direct-query datasource registered via the OpenSearch SQL plugin so OSD can read Prometheus metrics without a federated proxy
OSD Clawker workspace with features: ["use-case-all"] so the explore-flavor metrics / logs / traces nav groups all mount
Saved objects imported INTO the Clawker workspace — the core index patterns and the Clawker Networking dashboard, plus the index patterns, ingest pipelines, and dashboards contributed by every monitoring extension seeded into the stack (the claude-code extension adds the Claude Code Cost & Usage and Claude Code Activity dashboards). Build additional dashboards off the index patterns + Prometheus datasource, or ship them as a monitoring extension.

otel-collector gates on bootstrap completing successfully — it never starts until the cluster is preconfigured. Prometheus starts in parallel; bootstrap depends on Prometheus being up so the clawker_prometheus datasource registration can validate the configured URI. Bootstrap failure surfaces in docker logs clawker-opensearch-bootstrap and leaves the stack half-up by design (so wrong-mapped indices can’t be silently created). The throwaway-stack model means picking up template/policy edits requires monitor down --volumes && monitor up; templates only apply at index creation.

Telemetry Controls

Fine-tune what telemetry is collected via settings.yaml:

monitoring:
  telemetry:
    log_tool_details: true       # Include tool invocation details
    log_user_prompts: true       # Include user prompts in logs
    include_account_uuid: true   # Include account identifier
    include_session_id: true     # Include session identifier

Port Configuration

Override default ports in settings.yaml if they conflict with other services:

monitoring:
  otel_collector_port: 4318
  otel_grpc_port: 4317
  opensearch_port: 9200
  opensearch_dashboards_port: 5601
  opensearch_heap_mb: 512        # JVM -Xms/-Xmx for the OpenSearch node
  prometheus_port: 9090
  prometheus_metrics_port: 8889

After changing ports, regenerate config files and restart:

clawker monitor init --force
clawker monitor down
clawker monitor up

Checking Status

clawker monitor status

Shows container status (running/stopped) and service URLs for running services.

Teardown

# Stop the stack (preserves data)
clawker monitor down

# Stop and remove all data volumes
clawker monitor down --volumes

Without --volumes, monitoring data persists across restarts (named volume clawker-opensearch for indices, clawker-prometheus for TSDB) — as do the monitoring extensions seeded into the stack, so subsequent monitor up runs from any project keep them. --volumes clears both the telemetry data and the seeded extensions, resetting the stack. The clawker-net network is preserved for other Clawker services (firewall, agents).

Get started

Configuration

Running Agents

Security

Building images

Bundles & extensions

Operations

Under the hood

Developer guide

CLI Reference

Monitoring

What You Get

Architecture

Services

Service Hostnames

How Agents Connect

Egress Traffic Visibility

Monitoring Extensions

Setup

1. Initialize Configuration

2. Start the Stack

3. Run Agents

4. Explore in OpenSearch Dashboards

Bootstrap

Telemetry Controls

Port Configuration

Checking Status

Teardown

​What You Get

​Architecture

​Services

​Service Hostnames

​How Agents Connect

​Egress Traffic Visibility

​Monitoring Extensions

​Setup

​1. Initialize Configuration

​2. Start the Stack

​3. Run Agents

​4. Explore in OpenSearch Dashboards

​Bootstrap

​Telemetry Controls

​Port Configuration

​Checking Status

​Teardown

What You Get

Architecture

Services

Service Hostnames

How Agents Connect

Egress Traffic Visibility

Monitoring Extensions

Setup

1. Initialize Configuration

2. Start the Stack

3. Run Agents

4. Explore in OpenSearch Dashboards

Bootstrap

Telemetry Controls

Port Configuration

Checking Status

Teardown