Section 02 Β· Monitoring
🌾 Silo · Monitoring

Everything we watch β€” stored and surfaced.

One self-hosted kube-prometheus-stack per region replaces poll-only SolarWinds; hosts, network and Azure in one place across three regions.

3
Regions EUΒ·BRΒ·CN
~2,000
Monitored elements
$0
Licences (was $120–250k)
15β†’scrape
min poll, gone

The stack

Exporters feed an HA Prometheus pair that fans out to alerting, dashboards and long-term storage.

LayerComponentRole
Metrics corePrometheus (HA pair)scrape, store, alert rules
AlertingAlertmanagerdedup, route by owner, silence
DashboardsGrafana (OSS)Entra-ID SSO, per-team folders
Hostsnode_exporterCPU/mem/disk/net on VMs + on-prem
Networksnmp_exporter + blackbox_exporterSNMP + HTTP/TCP/ICMP probes
Azureazure-metrics-exporterAzure Monitor per subscription
Kuberneteskube-state-metrics + cAdvisorcluster + workload health
Long-termThanos / remote-writeretention beyond local TSDB

Three regions, three rulesets

Each region runs its own full stack; residency and isolation rules differ by jurisdiction.

EMEA Β· West Europe Global Azure

Full stack; primary. May hold a read-only Thanos view of Brazil aggregates.

Brazil Β· Brazil South LGPD

Full stack; metrics stay in-region. Only non-sensitive aggregates leave.

China Β· China North 3 21Vianet Β· isolated

Fully isolated full stack, own Grafana + identity. NO cross-border scraping or querying, ever.

From SolarWinds to Prometheus

A parallel-run migration that never leaves us blind.

Parallel-run cut-over

Stand the stack up alongside SolarWinds β†’ roll out exporters via Ansible/AWX β†’ confirm parity β†’ cut over per service β†’ decommission SolarWinds last.

  • Kills the licence: $120–250k/yr recovered.
  • First-class coverage: K8s, Azure and network as native targets vs 15-min polling.
  • Runbook links survive: alert names kept as <service>-<severity>-<symptom>.
  • ADR realised: new single-zone workload alert realises ADR-0001.
Note: all cost figures are documented ranges, [validate with finance].

Open the docs & dashboards

Source architecture, residency notes and the decision records behind the stack.

Where to go next

Neighbouring sections of the workspace.