Architecture

System overview

┌──────────────────────────────────────────────────────────────────────┐
│                        Consul Guardian                               │
│                                                                      │
│  ┌─────────────┐   ┌─────────────┐   ┌─────────────┐               │
│  │  Watcher    │   │  Snapshot   │   │  Dashboard  │               │
│  │  (blocking  │   │  Creator    │   │  (HTTP +    │               │
│  │   queries)  │   │             │   │   WebSocket)│               │
│  └──────┬──────┘   └──────┬──────┘   └──────┬──────┘               │
│         │                  │                  │                      │
│  ┌──────▼──────┐   ┌──────▼──────┐   ┌──────▼──────┐               │
│  │  Git Syncer │   │  Storage    │   │  Drift      │               │
│  │  (go-git)   │   │  Backend    │   │  Detector   │               │
│  └──────┬──────┘   └──────┬──────┘   └──────┬──────┘               │
│         │                  │                  │                      │
│  ┌──────▼──────┐   ┌──────▼──────┐   ┌──────▼──────┐               │
│  │  Serializer │   │  Local │ S3 │   │  Restore    │               │
│  │  (KV→file)  │   │             │   │  Executor   │               │
│  └─────────────┘   └─────────────┘   └─────────────┘               │
│                                                                      │
└───────────────────────────┬──────────────────────────────────────────┘
                            │
                    ┌───────▼───────┐
                    │  Consul KV    │
                    │  (via API)    │
                    └───────────────┘

Components

Watcher (internal/watcher)

The watcher uses Consul blocking queries to detect KV changes in near-real-time. It maintains per-prefix goroutines, each tracking a ModifyIndex. When the index changes, the watcher diffs the new state against its tracked state using an IndexTracker and emits KVChange events.

Key types:

• BlockingWatcher -- Runs the blocking query loop for a single prefix.
• IndexTracker -- Tracks ModifyIndex per key to detect adds, updates, and deletes.
• AdaptiveRateLimiter -- Backs off on errors, resets on success. Max backoff: 15 seconds.
• KVChange -- Represents a single change: Added, Modified, or Deleted.

Git Syncer (internal/sync)

Receives KVChange events and writes them to a Git repository using go-git (pure Go, no Git binary dependency for commits). Each batch of changes becomes a single commit.

Key types:

• GitSyncer -- Orchestrates file writes and Git commits. Handles auto-push.
• Serializer -- Converts Consul KV keys to filesystem paths and vice versa.

Drift Detector (internal/drift)

Reads the desired state from the Git repository (filesystem) and the actual state from Consul KV (API), then compares key-by-key. Reports three categories:

• MISSING -- In Git, not in Consul.
• EXTRA -- In Consul, not in Git.
• DRIFTED -- Both exist, values differ.

Restore (internal/restore)

Creates a RestorePlan by comparing Git state against current Consul state, then executes the plan using Consul transactions with CAS.

Key types:

• Planner -- Generates a plan of SET and DELETE operations.
• Executor -- Executes the plan with CAS-based transactions. Supports dry-run.

Snapshot (internal/snapshot)

Takes full cluster snapshots via the Consul snapshot API and stores them through a pluggable backend interface.

Key types:

• Creator -- Takes a snapshot and uploads it with SHA-256 integrity verification.
• RetentionManager -- Enforces count-based and age-based retention policies.
• Scheduler -- Runs snapshots on a cron schedule.

Storage (internal/storage)

Pluggable backend for snapshot storage.

Key types:

• Backend -- Interface with Upload, Download, List, Delete, Exists methods.
• LocalBackend -- Stores snapshots on the local filesystem.
• S3Backend -- Stores snapshots in Amazon S3 using aws-sdk-go-v2.

Dashboard (internal/dashboard)

HTTP server with REST API and WebSocket support. Serves the React frontend as static files with SPA fallback routing.

API endpoints:

• GET /api/status -- Cluster status and health.
• GET /api/kv -- List KV pairs.
• GET/PUT/DELETE /api/kv/{key} -- CRUD operations.
• GET /api/kv/history/{key} -- Git log for a specific key.
• GET /api/changes -- Recent change events.
• GET /api/drift -- Run drift detection.
• POST /api/snapshot/save -- Take a snapshot.
• GET /api/snapshots -- List snapshots.
• POST /api/restore -- Execute a restore.
• GET/PUT /api/settings -- Dashboard settings.
• /ws -- WebSocket for real-time updates.

Consul Client (internal/consul)

Thin wrapper around hashicorp/consul/api that provides:

• KVReader -- Read and list KV pairs with blocking query support.
• KVWriter -- Write, delete, and execute transactions.
• SnapshotManager -- Save and restore cluster snapshots.

Data flows

Watch flow

Consul KV → Blocking Query → IndexTracker.DetectChanges()
  → []KVChange → Serializer.WriteFile() → Git commit
  → (optional) Git push to remote

Snapshot flow

Consul Snapshot API → io.Reader → SHA-256 hash
  → Storage.Backend.Upload() → RetentionManager.Enforce()

Restore flow

Git filesystem → Planner.PlanFromGit() → RestorePlan
  → Executor.DryRun() (preview) or Executor.Execute()
  → Consul Transaction API (CAS)

Drift detection flow

Git filesystem → map[key]value (desired state)
Consul KV API → map[key]value (actual state)
  → key-by-key comparison → DriftReport

Technology choices

Component	Choice	Rationale
Language	Go	Single binary, native Consul SDK, goroutines for concurrent watchers
CLI framework	Cobra + Viper	Industry standard, env var + config file support
Git library	go-git	Pure Go, no binary dependency
Consul SDK	hashicorp/consul/api	Official client
WebSocket	gorilla/websocket	Battle-tested, simple API
Frontend	React 19 + TypeScript + Vite	Fast builds, shadcn/ui components
S3 SDK	aws-sdk-go-v2	Current AWS SDK
Testing	testify + testcontainers-go	Assertions + real Consul in integration tests
Scheduling	robfig/cron	Proven cron library for Go