Schema Normalization and Fingerprinting: Making Events Groupable

If machines cannot group events, they cannot alert on them. Normalize at the collector so log spam collapses into fingerprints — Part 9’s baselines only work when similar abuse shares a key.

This is Part 10 and the close of Phase 3: Telemetry and Observability. Part 8 joined timelines on TraceIDs. Part 9 baselined tool behavior. This post answers the residual question: when every log line is uniquely snowflaked — session ids in the message, slightly different stack frames, free-text denials — how does a machine ever group “the same abuse” so baselines and alerts can fire?

---

A Thousand One-Off Streams

Injection week. Panguard blocks look like this in raw logs:

denied tool=file_read path=/home/runner/.ssh/id_rsa session=sess_a1b2…
denied tool=file_read path=/home/runner/.ssh/id_rsa session=sess_c3d4…
denied tool=file_read path=/workspace/../.env session=sess_e5f6…
SIGKILL binary=/usr/bin/npx parent=/usr/bin/node exit=137 pod=agent-7f2c…
SIGKILL binary=/usr/bin/npx parent=/usr/bin/node exit=137 pod=agent-9aa1…

Loki treats each line as novel. Grafana’s “top errors” is a cemetery of uniqueness. Part 9’s volume detector never sees a cluster — cardinality exploded, the signal dissolved into snow.

Humans can pattern-match free text. Automated baselines need stable keys. Without them, Phase 3 is expensive storage with optimistic dashboards.

---

Schema as a Security Control

The SIEM module’s first failure mode is alert fatigue from low-quality events. Unnormalized agent logs are how you get there.

payloadHash correlates without re-exposing secrets. Path classes (ssh_private_key, kubeconfig, env_file) beat raw paths in labels. Session ids belong in JSON fields for drilldown — not in fingerprints and not in Mimir labels.

Normalization is how Part 9’s firewall gets a nervous system.

---

The Architecture Pattern: Data Normalization for Machine Consumption

Emit structured events at the source when you control the source (Panguard, agent wrapper, sandbox dispatcher). At the collector (Alloy / OTel Collector), coerce third-party and host sensors into the same envelope. Compute fingerprints that survive volatile tokens — ids, paths with usernames, timestamps, URLs. Use fingerprints as grouping labels with bounded cardinality; keep raw detail in the log body for humans. Version the schema and pin SIEM/recording rules to schemaVersion. Redact before anything weaker than WORM sees payloads.

Raw producers (agent, Panguard, Tetragon, app)
        │
        ▼
 Alloy / OTel Collector
        ├─ parse / map → canonical fields
        ├─ normaliseMessage / pathClass
        ├─ fingerprint = hash(stable parts)
        ├─ labels: fingerprint, atr_role, event_subtype   (bounded)
        └─ body: full JSON (ids, hashes — not secrets)
                │
                ├─► Loki   (fingerprinted streams)
                ├─► WORM   (immutable decisions)
                └─► Mimir  (counters by role/tool/decision only)

---

Canonical Fields

Minimum security event shape (Parts 8–9 already assume this):

{
  "schemaVersion": "1.0",
  "eventId": "uuid",
  "timestamp": "…",
  "source": { "component": "panguard", "version": "…" },
  "principal": {
    "agentId": "…",
    "sessionId": "…",
    "atrRole": "DiagnoseService"
  },
  "event": {
    "type": "POLICY",
    "subtype": "PATH_DENIED",
    "outcome": "BLOCKED",
    "severity": "HIGH"
  },
  "detail": {
    "tool": "file_read",
    "pathClass": "ssh_private_key",
    "ruleId": "deny-home-ssh",
    "policy_fingerprint": "a1b2c3d4e5f60718"
  },
  "traceContext": { "traceId": "…", "spanId": "…" },
  "payloadHash": "sha256:…"
}

New components that only console.log prose denials should not ship. If a vendor sensor is prose-only, normalize at Alloy — don’t wait for the vendor.

---

Fingerprinting Exceptions

Same shape as the observability essay — stable type plus normalized message plus top frame:

export function createErrorFingerprint(event: ExceptionEvent): string {
  const err = event.payload.exceptions?.[0];
  const topFrame = err?.stacktrace?.frames?.at(-1);

  const raw = [
    err?.type ?? 'UnknownError',
    normaliseMessage(err?.value ?? ''),
    topFrame?.function ?? '',
    topFrame?.filename ?? '',
  ].join('|');

  return sha256(raw).slice(0, 16);
}

function normaliseMessage(msg: string): string {
  return msg
    .replace(/\b[0-9a-f]{8,}\b/gi, '<hash>')
    .replace(/\b\d+\b/g, '<n>')
    .replace(/https?:\/\/\S+/g, '<url>')
    .replace(/\/home\/[^/]+/g, '/home/<user>')
    .replace(/sess_[a-z0-9]+/gi, 'sess_<id>');
}

Attach error_fingerprint in beforeSend / log processor. Alert on new fingerprints for agent services — first-seen attack classes often arrive as first-seen exception shapes.

---

Fingerprinting Policy Events

Policy spam needs a sibling of error fingerprints:

export function createPolicyFingerprint(ev: SecurityEvent): string {
  const raw = [
    ev.event.type,
    ev.event.subtype,
    ev.detail?.ruleId ?? '',
    ev.detail?.tool ?? '',
    ev.detail?.pathClass ?? '',
    ev.detail?.binary ?? '', // for exec kills: basename only
  ].join('|');

  return sha256(raw).slice(0, 16);
}

export function pathClass(pathname: string): string {
  if (/\/\.ssh\//.test(pathname) || /id_rsa|id_ed25519/.test(pathname)) return 'ssh_private_key';
  if (/\/\.kube\/|kubeconfig/.test(pathname)) return 'kubeconfig';
  if (/\/\.env$|\.env\./.test(pathname)) return 'env_file';
  if (/\/var\/run\/secrets\/kubernetes\.io\//.test(pathname)) return 'k8s_sa_token';
  if (/\/etc\/shadow/.test(pathname)) return 'shadow';
  return 'other';
}

Now “900 unique path denies” become one stream: policy_fingerprint=… + pathClass=ssh_private_key + rising count — exactly what Part 9’s detectors need.

---

The Collector: Where Volatile Tokens Die

Conceptual OTel Collector / Alloy processors:

processors:
  attributes/clawql_security:
    actions:
      - key: policy_fingerprint
        action: insert
      - key: pathClass
        action: upsert
      - key: sessionId
        action: delete # from metric datapoints only — keep on logs
  transform/normalise:
    log_statements:
      - context: log
        statements:
          - set(attributes["pathClass"], pathClass(attributes["path"]))
            where attributes["path"] != nil
          - replace_pattern(body, "sess_[A-Za-z0-9]+", "sess_<id>")

Loki label set for security logs (bounded):

service, deployment_tier, event_type, event_subtype, policy_fingerprint, atr_role

sessionId, traceId, raw path, and raw agentId stay in JSON for Part 8 joins via | json filters — they’re not stream labels.

---

The Key Visual: Cardinality Collapse

After a week of traffic, you should see something like this:

Raw deny messages / day     ~ 50,000 unique lines
Policy fingerprints / day   ~     40–120 stable keys
New fingerprints / day      ~     single digits (page these)

If fingerprint cardinality tracks raw cardinality, normalization is lying — usually volatile tokens still leaking into the hash (pod names, timestamps, full paths).

---

How This Unlocks Parts 8–9

Part 9 without Part 10 is a ruler pointed at a blizzard.

---

Honest Failure Modes

Over-normalization. Collapsing everything to <n> can merge unrelated bugs. Keep error type and top frame function in the hash.

Under-normalization. Leaving usernames or pod hashes in the fingerprint recreates the original problem.

Label explosion. Fingerprints are labels only because they’re short and bounded. Adding “helpful” high-cardinality dimensions “just for now” is how cardinality budgets evaporate.

Schema drift. Bump schemaVersion and update recording rules in the same PR. Orphaned rules are silent detection debts.

Security vs privacy. Fingerprints must not encode secret material. Hash after redaction.

Agent stdout. Sidecar tools that print unique UUIDs every line will still blow streams unless the sandbox logger wraps them — prefer structured tool results (Part 7 schema strip) over scrapes of noisy stdout.

---

Getting Started

Publish schemaVersion 1.0 for security events; reject prose-only denials in new code. Map Tetragon/Falco/Wazuh through Alloy into the envelope; add pathClass / basename fields. Ship error_fingerprint and policy_fingerprint processors; attach as Loki labels. Rebuild Part 9 counters from normalized fields; drop free-text rate hacks. Alert on first-seen fingerprints for production ATR roles (observe 14 days, then page). Add a cardinality panel (raw vs fingerprint) to the NOC — treat regression as an incident.

Part 11: Phase 4 starts at the communication plane — mTLS and scoped object storage for edge agents.

---

Companion: DevSecOps-boilerplate · Observability essay. Docs: Security monitoring / SIEM.