Ephemeral Execution Sidecars: Isolated Tool Execution

Unsafe tools should never run on the agent host. Dispatch them to ephemeral throwaway sidecars (Kata/gVisor where needed) and destroy the workspace after — Panguard decides which tool may fire; the sidecar decides where leftover code is allowed to land.

This is Part 7 and the close of Phase 2: Runtime Integrity. Parts 4–6 hardened the agent host — which programs may start, which syscalls remain, which paths are crown jewels. This post answers the residual question: when a tool still has to run untrusted or high-blast-radius code, why is that code allowed to share a kernel, filesystem, and network with the agent?

---

The Tool That Shared Too Much

The agent host is carefully dieted. ATR allows run_tests and a narrow exec_skill. Panguard allows the call. The handler runs on the same machine as the long-lived agent:

clawql-agent (hours-long session)
  └─ skill runner
       └─ npm test / python script / "helpful" postinstall
            └─ still on the agent's mounts, UID space, and egress path

FIM locked down $HOME. ptrace was denied. Curl children were killed. Then the skill bind-mounted Docker.sock “so CI works,” or wrote into a shared /tmp the agent later reads as trusted context, or opened an egress path the agent’s NetworkPolicy never intended for subprocesses.

The blast radius was never the binary name. It was co-residency: durable agent process plus transient hostile code on one side of every policy Parts 4–6 spent time tuning.

Host hardening answers “what may this agent do here?” Sandboxing answers “must this work happen here at all?”

---

Panguard Says Yes; The Host Should Still Say “Not Here”

Panguard remains the structured gate: ATR claims, JSON Schema, rate limits, HITL with fallback: deny. That does not place the handler on a safe machine.

For exec-class tools: run them inside a sandbox (Kata where trust is lowest), keep a command allowlist at both Panguard and the sandbox, and set networkEgress: deny inside exec sandboxes so executed code cannot reach the network even if the string filter misses.

Panguard makes malicious execution structurally harder. Sidecars make residual execution someone else’s blast radius — briefly.

---

The Architecture Pattern: Ephemeral Execution Sidecar

Classify tools: in-process/same-pod (pure functions, trusted schemas) vs must-sandbox (exec, untrusted documents, third-party skills, package installs). On sandbox-class calls: start a fresh container/VM with a tight mount set, seccomp diet (Part 5), no crown-jewel paths (Part 6), and role-shaped egress. Copy in only the inputs the tool needs; copy out only the declared outputs; treat outputs as data, never as new instructions. Destroy the workspace when the task ends — no warm skill pools that accumulate /tmp gifts for the next tenant. Prefer stronger runtimes as trust drops: standard+seccomp → gVisor → Kata.

User / ATR session
        │
        ▼
   Panguard (allow / block / HITL)
        │ allow sandbox-class tool
        ▼
   Dispatcher
        │
        ├─ create ephemeral sandbox (Job / one-shot container)
        │     mounts: /in (ro), /out (rw), scratch tmpfs
        │     seccomp: RuntimeDefault (+ role diet)
        │     network: deny or allowlist
        │     runtimeClass: runc | gvisor | kata
        │
        ├─ run tool → collect /out → stream logs with session labels
        │
        └─ delete sandbox + volumes   ← non-negotiable
                │
                ▼
   Result (schema-validated) → agent as data

If the destroy step is “best effort,” you built sticky residences with a marketing name.

---

What Never Goes Into the Sidecar

A Kata VM that bind-mounts Docker.sock is an expensive way to stay unsafe.

---

Choosing the Runtime

Do not mix runtime classes on one node pool for security-sensitive workloads. Isolation boundaries are enforced at the node and RuntimeClass level; a shared pool that sometimes runs Kata and sometimes runc invites the weakest assumption to win.

---

Dispatch Sketch: Job Per Tool Call

apiVersion: batch/v1
kind: Job
metadata:
  generateName: clawql-tool-
  labels:
    clawql.io/session: 'sess_123'
    clawql.io/tool: 'exec_skill'
    clawql.io/trace: '…'
spec:
  ttlSecondsAfterFinished: 60
  backoffLimit: 0
  template:
    spec:
      runtimeClassName: kata-qemu
      restartPolicy: Never
      securityContext:
        runAsNonRoot: true
        seccompProfile:
          type: RuntimeDefault
      containers:
        - name: tool
          image: ghcr.io/example/clawql-skill-runner@sha256:…
          args: ['--in', '/in', '--out', '/out']
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            capabilities:
              drop: ['ALL']
          volumeMounts:
            - name: in
              mountPath: /in
              readOnly: true
            - name: out
              mountPath: /out
            - name: tmp
              mountPath: /tmp
          resources:
            limits:
              cpu: '1'
              memory: 1Gi
      volumes:
        - name: in
          emptyDir: {}
        - name: out
          emptyDir: {}
        - name: tmp
          emptyDir:
            medium: Memory

Helm flags:

helm upgrade --install clawql ./charts/clawql-mcp -n openclaw --create-namespace \
  --set security.kata.enabled=true \
  --set security.kata.runtimeClassName=kata-qemu \
  --set security.toolSandbox.ephemeralJobs=true \
  --set security.toolSandbox.networkEgress=deny \
  --set kyverno.runtimeClassPolicy.enabled=true

Kyverno should fail closed if a sandbox-class tool pod requests the wrong RuntimeClass, mounts a forbidden path, or sets seccompProfile: Unconfined.

---

Defense in Depth, Not Synonyms

Keep the boundaries named. ATR / schema / HITL answers: may this tool+args run for this session? Command allowlist + deny patterns answers: redundant on purpose for exec-class. Sandbox placement answers: separate kernel/userspace interceptor + mounts + egress. Result schema strip answers: unexpected fields never re-enter the agent as structure that looks like instructions.

If HITL times out, deny — then never start the sidecar. Starting a sandbox speculatively while a human thinks is how you pay for isolation and still race the reviewer.

---

Edge and Laptop

Daily coding under Seatbelt; escalate to a VM when the tool class outgrows path policy.

---

Honest Failure Modes

Latency. Kata startup and image pull dominate short tools. Prefetch signed images (Part 12); keep skill runners thin; use gVisor when the threat model allows.

Compatibility. gVisor does not implement every syscall — test skill images. A “works on my runc” skill is not production-ready under runsc.

Warm pools. Reusing sandboxes across tenants or sessions recreates co-residency. Pool images, not dirty writable layers — and never across tenant boundaries.

Result trust. A sandbox that returns a README saying “now run this curl” has succeeded at isolation and failed at confused-deputy hygiene. Schema-validate and treat as data.

Logging. Sidecar stdout can contain secrets from the tool’s inputs. Redact before Loki; keep raw artifacts in WORM with stricter ACL.

Destroy failures. ttlSecondsAfterFinished helps; alert on leaked Jobs/volumes labeled clawql.io/tool. Leftover sandboxes are residual risk with a UUID.

---

Getting Started

Inventory MCP tools — mark sandbox-class (exec, untrusted docs, third-party skills, installs). Implement a dispatcher that creates one-shot Jobs/containers; refuse in-process fallback for those marks. Pin RuntimeClass per tool class; separate node pools; admit fail-closed. Mount only /in + /out + tmpfs; ban Docker.sock and home directories in policy. Default-deny egress inside exec sandboxes; allowlist only what the tool contract requires. Enforce destroy + volume GC; alert on leftovers; pass session/trace labels into logs for Part 8.

Part 8: Phase 3 starts where Phase 2’s kills, denies, and sandbox lifetimes must become one timeline with the prompt that caused them.

---

Companion: DevSecOps-boilerplate. Docs: Sandboxing (Kata, gVisor, Seatbelt) · Panguard / ATR.