Files
LibreMail-Bug-Report-Ingest/worker/telemetry_wasm.go
T
JMR-devandClaude Opus 4.8 45ac7236ee #17 Observability: OpenTelemetry logging + tracing + alerting
Instrument the Worker with OpenTelemetry traces + structured logs over OTLP,
plus alertable signals, via a minimal hand-rolled OTLP/HTTP exporter that fits
the TinyGo/Wasm Worker build.

New internal/telemetry package (build-tag-free, host-tested):
- Span/log shim: Telemetry provider, Span (attrs/status/events/end), Log
  (Info/Warn/Error), trace/span-id correlation, W3C-style ids from crypto/rand.
- Exporter seam: MemoryExporter (in-memory, for tests) and OTLPHTTPExporter
  (OTLP/HTTP JSON over net/http). No go.opentelemetry.io/otel/sdk dependency:
  the full OTEL-Go SDK + OTLP exporters pull in a large, reflection-heavy tree
  (protobuf, grpc) that bloats the Wasm binary and is unreliable under TinyGo.
  The shim uses only stdlib already proven under this project's js/wasm target
  (net/http per #26, encoding/json, crypto/rand). OTLP is the wire format, so
  any OTLP backend can ingest it.
- Behaviour-preserving by construction: instrumentation is threaded through
  context. Instrumented code pulls an optional *Telemetry from ctx; absent (or
  nil exporter) => every method is a no-op. No public signatures change
  (NewHandler, handler.New, publish.New/Publish, schedule.Run are untouched), so
  parallel work built on the current APIs keeps compiling.

Instrumentation:
- ingest: an "ingest.request" server span + correlated log per request,
  classifying accepted / rejected / rate_limited / error. Observe-only (wraps the
  response writer to read the status); the HTTP contract is unchanged. A 5xx
  (e.g. 503 storage-unavailable) sets the span to Error and emits the alertable
  ingest.error signal; 4xx client rejections are INFO, not alerts.
- publish: a "publish.run" span with per-report "publish.report" child spans and
  a log per report (published/failed). A failed report/run sets Error and emits
  alert.type=publish.run_failed. The per-run cap-hit (folding in the #14
  follow-up) is now emitted as a structured, alertable OTEL signal
  (alert.type=publish.cap_hit + counts), not merely a log line.
- schedule: a "schedule.run" span parenting the publish run; a list/publish
  failure emits alert.type=schedule.run_failed.

Config (OTLP endpoint TBD, issue #17):
- OTEL_EXPORTER_OTLP_ENDPOINT (plain var) - base OTLP/HTTP URL; empty => telemetry
  disabled (Worker behaves as before). /v1/traces and /v1/logs are appended.
- OTEL_EXPORTER_OTLP_HEADERS (Secrets Store secret) - auth header(s), never
  committed. OTEL_SERVICE_NAME (plain var) - service.name override.
- worker/telemetry_wasm.go builds the exporter lazily per run and injects the
  provider into the request/scheduled context; wrangler.jsonc gains only these
  OTEL keys.

Alerting: run-failure, cap-hit, and elevated-ingest-error are emitted as span
status=Error and structured log records carrying alert=true + a specific
alert.type, so a backend alert rule can key on them once the OTLP endpoint is
chosen.

Tests: host unit tests with the in-memory exporter assert the ingest spans+logs
for accepted/rejected/error, the publish run span + per-report spans + the
cap-hit and run-failed signals, the schedule run span + list-error alert, and
the OTLP/JSON encoding + HTTP round trip (httptest, no real backend). No-op
default verified. go vet ./... and go test ./... green; GOOS=js GOARCH=wasm
go build ./... compiles.

Closes #17

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-02 17:14:21 -05:00

121 lines
4.7 KiB
Go

//go:build js && wasm
// This file wires OpenTelemetry (#17) into the Cloudflare Worker: it builds the
// OTLP/HTTP exporter from Worker config and injects the telemetry provider into
// the fetch request context so internal/ingest (and, via scheduled_wasm.go,
// internal/schedule + internal/publish) emit spans and structured logs.
//
// The OTLP endpoint/backend is deliberately TBD (issue #17): it is read from a
// plain Worker var and the auth header from a Secrets Store secret, never
// hardcoded. When the endpoint var is unset the provider is nil and all
// instrumentation is a no-op, so the Worker runs exactly as before until an
// endpoint is configured.
//
// Compiled only into the js/wasm Worker; excluded from host builds and tests.
// The non-trivial telemetry logic lives in the build-tag-free, host-tested
// internal/telemetry package; this file is the thin runtime adapter that reads
// the Cloudflare bindings.
package main
import (
"context"
"log"
"net/http"
"sync"
"github.com/syumai/workers/cloudflare"
"github.com/JMR-dev/LibreMail-Bug-Report-Ingest/internal/storage"
"github.com/JMR-dev/LibreMail-Bug-Report-Ingest/internal/telemetry"
)
// OTEL config names, declared in wrangler.jsonc. The endpoint and service name
// are plain (non-secret) vars; the headers (which carry auth) are a Secrets
// Store secret read via binding.get(), like the encryption keyring.
const (
// otelEndpointVar is the base OTLP/HTTP URL, e.g. "https://otlp.example.com".
// TBD (#17): empty/unset => telemetry disabled (no-op).
otelEndpointVar = "OTEL_EXPORTER_OTLP_ENDPOINT"
// otelServiceVar overrides the reported service.name.
otelServiceVar = "OTEL_SERVICE_NAME"
// otelHeadersSecret is the Secrets Store secret holding the OTLP auth
// headers in "key=value,key2=value2" form (e.g. "Authorization=Bearer ...").
otelHeadersSecret = "OTEL_EXPORTER_OTLP_HEADERS"
otelServiceDefault = "libremail-bug-report-ingest"
otelScopeName = "github.com/JMR-dev/LibreMail-Bug-Report-Ingest"
)
var (
telMu sync.Mutex
telProvider *telemetry.Telemetry // cached for the isolate once built
telBuilt bool
)
// workerTelemetry lazily builds the telemetry provider from Worker config and
// caches it for the isolate lifetime. It returns nil (a valid no-op provider)
// when OTEL_EXPORTER_OTLP_ENDPOINT is unset — the endpoint is TBD (#17).
//
// It must be called from within a request or scheduled handler: the headers
// secret read is async and per-request on the Workers runtime (mirroring how the
// encryption keyring and admin token are read). The auth header is optional — if
// the secret is unavailable the exporter is still built (some backends accept
// unauthenticated ingest, or auth may live in the endpoint URL); the failure is
// logged, never fatal.
func workerTelemetry() *telemetry.Telemetry {
telMu.Lock()
defer telMu.Unlock()
if telBuilt {
return telProvider
}
endpoint := cloudflare.Getenv(otelEndpointVar)
if endpoint == "" {
// Endpoint TBD/unset: leave telemetry disabled. Do not cache, so a later
// deploy that sets the var (new isolate) picks it up.
return nil
}
headers := map[string]string{}
if raw, err := storage.ReadSecret(otelHeadersSecret); err != nil {
log.Printf("telemetry: OTLP headers secret %q unavailable (%v); exporting without auth headers", otelHeadersSecret, err)
} else {
headers = telemetry.ParseHeaders(string(raw))
}
service := cloudflare.Getenv(otelServiceVar)
if service == "" {
service = otelServiceDefault
}
exp := telemetry.NewOTLPHTTPExporter(telemetry.OTLPConfig{
Endpoint: endpoint,
Headers: headers,
Resource: []telemetry.KeyValue{telemetry.String("service.name", service)},
Scope: telemetry.Scope{Name: otelScopeName},
})
telProvider = telemetry.New(exp, telemetry.WithErrorHandler(func(err error) {
// Best-effort: a telemetry export failure must never break the request.
log.Printf("telemetry: export failed: %v", err)
}))
telBuilt = true
return telProvider
}
// withWorkerTelemetry wraps the fetch handler so every request carries the
// telemetry provider in its context (context propagation). When telemetry is
// disabled the provider is nil and the wrapped handler behaves identically.
func withWorkerTelemetry(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
ctx := telemetry.NewContext(r.Context(), workerTelemetry())
next.ServeHTTP(w, r.WithContext(ctx))
})
}
// scheduledContext returns a context carrying the telemetry provider for the
// weekly publish run, so internal/schedule and internal/publish emit their run
// and per-report spans/logs.
func scheduledContext(ctx context.Context) context.Context {
return telemetry.NewContext(ctx, workerTelemetry())
}