Hardened Communication Plane: Edge Worker Security

Edge-mode agents need mTLS control planes and tightly scoped object storage — not flat networks and shared buckets. Assume the remote LAN is hostile; authenticate every hop and deny ambient egress.

This is Part 11 and the start of Phase 4: Architectural Best Practices. Phases 1–3 hardened identity, host runtime, and telemetry. This post answers a different geography question: when clawql-agent runs on remote or customer infrastructure — factory floor, branch office, developer edge node — what stops a MitM on the LAN, or a compromised agent, from treating the control plane and object store as a flat, friendly intranet?

The LAN That Was “Internal”

An edge agent boots on a shared VLAN. NATS and MinIO are “only on the private network,” so TLS is optional for latency. Object storage uses one access key for agents/*. The gateway listens on 0.0.0.0:8080 behind a “we’re behind NAT” shrug.

A laptop on the same segment — or a DNS-rebinding browser page aimed at a mis-published hostname — talks to NATS in cleartext, scrapes control messages, and pulls every tenant prefix under the shared MinIO key. Phase 2 never saw a bad syscall. Phase 3’s TraceIDs dutifully documented the theft.

“Internal” is not a trust boundary. Edge geography is an adversarial network until every hop proves identity.

Edge Is Not “Cluster Lite”

Cluster agents inherit Istio/NetworkPolicy gravity. Edge agents often lose both and compensate with faith.

Hop	Threat without hardening	Control
Agent → gateway / NATS	MitM, spoofed tasks, injected tool results	mTLS + workload identity
Agent → MinIO / S3	Cross-tenant read, bulk exfil	Prefix-scoped credentials (Parts 2/3 style)
Client → gateway	Open port, DNS rebinding	Bind + Host/Origin + VPN/Tunnel
Agent / sandbox → internet	Prompt-injection pipe out	ServiceEntry / DNS allowlist + DLP
Peer agents on LAN	Lateral movement	No ambient allow; identity-aware L7 policies

Parts 4–7 still apply on the edge host. This part adds the wire.

Host diet without a hardened plane is a vault with the front door on a public corridor.

The Architecture Pattern: Hardened Communication Plane

Every control-plane connection is mutually authenticated — SPIFFE SVIDs, mesh sidecars, or equivalently pinned mTLS client certs for edge NATS. L3/L4 default-deny (host firewall / NetworkPolicy) and L7 identity rules work together, not as alternatives. Object storage credentials open only that agent’s prefix (tenants/t/agents/a/*). Gateways never bind 0.0.0.0 for agent MCP; prefer Unix sockets, loopback, Tailscale/WireGuard, or outbound-only Cloudflare Tunnel with Access. Egress is allowlist-only; sandboxes default networkEgress: deny (Part 7). Unexpected listeners and unexpected peers are Part 8/9 security events.

                    ┌─────────────────────────────┐
   MCP clients      │  VPN / Cloudflare Tunnel    │
   (allowlisted) ──►│  Access / Tailscale ACL     │
                    └─────────────┬───────────────┘
                                  │ Host/Origin OK
                                  ▼
                           Gateway (127.0.0.1
                           or unix socket)
                                  │ mTLS
                    ┌─────────────┼─────────────┐
                    ▼             ▼             ▼
                 NATS/control   Panguard     Vault exchange
                    │
                    ▼
              clawql-agent (edge)
                    │
                    ├─ mTLS ─► peer services (explicit principals only)
                    │
                    └─ scoped STS ─► MinIO/S3  prefix agents/<id>/*
                           │
                           └─ egress allowlist ─► declared APIs only

mTLS for NATS and Control Traffic

Cluster path (ClawQL default):

SPIRE issues short-lived SVIDs bound to ServiceAccount attestation: spiffe://cluster.local/ns/agents/sa/clawql-edge. Istio PeerAuthentication STRICT rejects plaintext. AuthorizationPolicy scopes so only the gateway principal may hit Panguard /enforce, and only agents may subscribe to their subject prefixes.

Edge path when a full mesh is heavy:

Client certificates to NATS (verify: true, CA pinned) with the same short-TTL mindset as Part 2. Prefer a thin tunnel (WireGuard / Tailscale) so NATS isn’t on the ambient LAN even if TLS misconfigures once. No shared NATS user/password in the agent image.

Istio-style authorization policy:

apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: nats-agents-only
spec:
  selector:
    matchLabels:
      app: nats
  action: ALLOW
  rules:
    - from:
        - source:
            principals:
              - spiffe://cluster.local/ns/agents/sa/clawql-edge
              - spiffe://cluster.local/ns/gateway/sa/gateway
      to:
        - operation:
            methods: ['CONNECT', 'POST']

Default-deny NetworkPolicy still wraps the namespace first — mesh L7 doesn’t excuse open L3.

Object Storage: One Agent, One Prefix

Shared buckets with a god key are Phase 1’s static secret problem wearing S3 clothes.

Anti-pattern	Hardened posture
One `MINIO_ROOT` on every edge agent	STS / Vault-issued keys scoped to `agents/<id>/*`
`ListBucket` on `*`	List/Get/Put only under workspace prefix
Long-lived keys in env	Short TTL; exchange at gateway (Parts 2–3)
Public `GetObject` for debugging	Private; signed URLs with tiny TTL if human access needed

Policy sketch:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject", "s3:ListBucket"],
      "Resource": [
        "arn:aws:s3:::clawql-workspaces",
        "arn:aws:s3:::clawql-workspaces/tenants/${tenant}/agents/${agentId}/*"
      ],
      "Condition": {
        "StringLike": {
          "s3:prefix": ["tenants/${tenant}/agents/${agentId}/*"]
        }
      }
    }
  ]
}

If an agent can read another agent’s memory prefix, Parts 6 and 13 are theater.

Gateway Reachability: Bind, Firewall, Rebinding

The gateway module is specific on this. Never bind 0.0.0.0 for agent MCP HTTP/SSE. Default 127.0.0.1 or unix:///var/run/clawql/gateway.sock (0660, clawql group). Host firewall denies the port on non-loopback; allows only lo or a Tailscale IP / wg0. Remote access via Tailscale/Headscale/WireGuard or Cloudflare Tunnel (outbound-only) with Access — no inbound public port forward. Host/Origin allowlist middleware is mandatory for HTTP/SSE; it strips DNS rebinding. Unix sockets are immune by construction.

allowedHosts:
  - localhost
  - 127.0.0.1
  - clawql-gateway.tailnet-name.ts.net

Alert via Falco/Tetragon if clawql-api binds 0.0.0.0. Drift after upgrades is the silent revert.

Egress: The Exfil Pipe on the Edge

Edge agents sit on networks with “full internet for patches.” Treat that as a failure mode to design around.

Default-deny egress at host firewall / mesh Sidecar; declare ServiceEntries (or OS allowlists) for model APIs and update endpoints only. DNS allowlist via CoreDNS RPZ or Cloudflare Gateway — NXDOMAIN everything else; heuristic alerts for tunneling (long labels, high entropy, query storms). SSRF guards on URL tools: block RFC1918, loopback, link-local (169.254.0.0/16 — cloud metadata), dangerous schemes; pin IP after first lookup; no redirect follow by default. DLP on tool params — detect equals block, not “alert later.” Sandboxed skills with networkEgress: deny unless the manifest’s egressRequirements match exactly (Part 7). Session volume trips from Part 9 apply here too: MB egress or external call storms trigger quarantine.

The Trust Map

Before calling an edge site production, draw and verify this:

[MCP client] --(Access/VPN)--> [Gateway bind OK?] --mTLS--> [NATS]
                                                          |
                                                          v
                                              [clawql-agent identity]
                                                     |          |
                                           scoped STS          egress AL
                                                     v          v
                                                  [MinIO prefix] [APIs]

Ask out loud:

Who can speak to NATS without a client cert?
Which prefixes can this agent’s key list?
What happens if someone joins the LAN with Wireshark?
What happens if a browser rebinds a hostname to the gateway loopback?
What outbound DNS answers succeed that you didn’t write down?

Any “uh” in response means the plane isn’t hardened.

Honest Failure Modes

Latency vs STRICT mTLS. Pay the handshake; “temporary PERMISSIVE” on edge fleets tends to become permanent.

Cert distribution on air-gapped edges. Plan SPIRE nested / offline CA or short-lived bootstrap — static forever-certs recreate Part 2.

Docker iptables vs ufw. Docker can bypass ufw; pin rules where the CNI enforces.

Cloudflare Tunnel alone isn’t authorization. Access policies still required; Tunnel is reachability, not identity.

Telemetry egress. Parts 8–10 collectors need declared destinations too — or agents phone home through shadow channels.

Multi-tenant edges. One box hosting two customers needs namespace/identity separation as strict as the cluster — a shared MinIO root fails both.

Getting Started

Inventory edge agents: control URLs, bucket prefixes, gateway bind addresses — write the trust map. Turn on STRICT mTLS (or pinned client certs) for NATS/control; kill shared passwords. Replace god object-store keys with prefix-scoped STS; verify cross-agent deny. Rebind gateways to loopback/Unix or Tunnel/VPN; enable Host/Origin allowlists; audit listeners. Default-deny egress and DNS allowlist; test SSRF and sandbox undeclared egress canaries. Emit NETWORK anomalies into Phase 3 telemetry with traceId and sessionId.

Part 12: Cosign and Kyverno so unsigned images and skills never board that plane.

Companion: DevSecOps-boilerplate. Docs: Zero-trust network · Egress / DNS / DLP · Gateway hardening.