Skip to main content

Drift Detection

Drift happens when the live state in Consul diverges from the desired state in Git. Someone edits a key directly in the Consul UI, a deploy script sets a value that was never committed, or a key gets deleted by accident. Drift detection finds these differences.

How it works

Guardian reads the desired state from the Git backup repository and the actual state from Consul KV, then compares them key by key.

Git (desired state)          Consul (actual state)
───────────────── ──────────────────────
config/db/host = "db1" config/db/host = "db2" → DRIFTED
config/db/port = "5432" config/db/port = "5432" → In Sync
config/cache/ttl = "300" (missing) → MISSING
(not in Git) config/temp/data = "abc" → EXTRA

Running drift detection

CLI

consul-guardian drift \
--consul-addr http://localhost:8500 \
--prefix "config/" \
--git-repo ./consul-backup

Output:

Drift Report (2026-04-04 14:30:00 UTC)
================================================
In Sync: 42
Missing: 2 (in Git, not in Consul)
Extra: 1 (in Consul, not in Git)
Drifted: 3 (values differ)

MISSING config/cache/ttl
MISSING config/cache/max-size
EXTRA config/temp/debug-mode
DRIFTED config/database/host
DRIFTED config/database/max-connections
DRIFTED config/redis/timeout

Dashboard

Navigate to the Drift Detection page and click "Run Scan". Results appear inline with the expected and actual values for drifted keys.

Drift categories

StatusMeaningExample
MISSINGKey exists in Git but not in ConsulA key was deleted from Consul but the Git record remains
EXTRAKey exists in Consul but not in GitSomeone added a key directly in Consul without going through the normal process
DRIFTEDKey exists in both, values differA value was changed in Consul after Guardian's last sync

Using drift in CI/CD

The drift command exits with code 1 when drift is detected. This makes it easy to add as a CI check:

# GitHub Actions
- name: Check for config drift
run: |
consul-guardian drift \
--consul-addr ${{ secrets.CONSUL_ADDR }} \
--consul-token ${{ secrets.CONSUL_TOKEN }} \
--prefix "config/" \
--git-repo ./consul-backup
# GitLab CI
drift-check:
stage: validate
script:
- consul-guardian drift --prefix "config/" --git-repo ./consul-backup
allow_failure: false

A failed drift check blocks the pipeline, forcing the team to reconcile the difference before proceeding.

Scheduled drift checks

Run drift detection on a cron schedule to catch manual changes early:

# GitHub Actions - run every hour
on:
schedule:
- cron: '0 * * * *'

jobs:
drift-check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Run drift detection
run: consul-guardian drift --prefix "config/"
- name: Notify on drift
if: failure()
run: |
curl -X POST ${{ secrets.SLACK_WEBHOOK }} \
-d '{"text":"Config drift detected in production Consul!"}'

Auto-reconciliation

Currently, drift detection is read-only. To fix drift, use the restore command to push the Git state back to Consul.

A future version will support --auto-reconcile to automatically fix detected drift.

Real-world example

A team runs Guardian's drift check as a daily cron job. One morning, the check fails: config/payment/timeout has drifted from "30s" (Git) to "5s" (Consul). Investigation reveals a developer set the timeout to 5 seconds during load testing yesterday and forgot to revert it. Without drift detection, this would have caused payment timeouts in production when traffic spiked.