Articles › Performance
How long does a Patroni failover take? We measured it
“A few seconds” is the usual answer. It's true for one kind of failover and wrong for the one that matters most. Here are real numbers from a 3-node cluster, what drives them, and what tuning actually buys you.
The short answer
| What happened to the leader | Writes unavailable | Why |
|---|---|---|
Planned switchover (patronictl switchover) | 4.7–5.0 s | Leader steps down cleanly and hands over the lock |
Clean stop (systemctl stop patroni, reboot) | 4.7–7.0 s | Patroni releases the lock as it shuts down |
Hard crash (power loss, kernel panic, kill -9) | 24–30 s | Nobody releases the lock; it has to expire |
Default timings (ttl 30 s, loop_wait 10 s, retry_timeout 10 s). Asynchronous replication, HAProxy routing on the Patroni REST API. Ranges cover repeated runs. Zero committed transactions were lost in every run.
How we measured
A writer opened a new connection through HAProxy's read-write port and committed one row every 200 ms, the way an application with a retrying connection pool behaves. We then took the leader away and recorded:
- Write downtime: from the first failed write to the first successful write on the new leader. This is what your users feel.
- Lost transactions: every row the client saw as committed, checked against the database afterwards.
- Rejoin time: until the old leader streamed again as a replica, which shows that
pg_rewindworked.
We measured the write path end to end, not just Patroni's log, because what matters is when writes succeed again. That includes Patroni's election, PostgreSQL's promotion, and HAProxy noticing the new leader.
Why a crash takes ~30 seconds
Patroni's leader holds a lock (a key with a time-to-live) in etcd and renews it every loop_wait seconds. When the leader is stopped or switched over, Patroni deletes the key, so a replica can take over on its next loop.
When the leader crashes, nothing deletes the key. The replicas must assume the leader might still be alive, perhaps just slow, until the lock expires after ttl seconds. That waiting is deliberate: it's what stops two nodes believing they're both the primary. So crash failover time is roughly:
ttl (time since the last renewal, up to 30 s) + promotion of the best replica (~1 s) + HAProxy health check notices it (1–3 s)
With the default ttl: 30, we measured 24 to 30 seconds. The spread comes from when in the renewal cycle the crash happens.
What tuning buys you
The three settings live in Patroni's cluster configuration (patronictl edit-config) and must satisfy ttl ≥ loop_wait + 2 × retry_timeout:
| ttl / loop_wait / retry_timeout | Crash downtime measured |
|---|---|
| 30 / 10 / 10 (default) | 24–30 s |
| 20 / 5 / 5 | 19.9–20.8 s |
Cutting the timings by a third saved us about 5–8 seconds. It comes at a cost: retry_timeout is also how long Patroni tolerates a slow etcd or a network blip before it gives up. Set it too low, and a 6-second hiccup on a busy network demotes a healthy primary. You'd trade a rare 30-second outage for more frequent, unnecessary 5-second ones. For most clusters on a normal LAN, the defaults are right.
Two settings that matter more than people expect:
failsafe_mode: true. Without it, losing etcd quorum demotes the primary even though PostgreSQL is fine. In our test, with 2 of 3 etcd members down and failsafe mode on, the primary kept accepting writes for the whole test. Only automatic failover was lost.- HAProxy's check interval.
inter 2s fastinter 500ms fall 2 rise 1detects the change within a couple of seconds. The common tutorial valueinter 3s fall 3 rise 2can add 6–9 seconds on its own.
Does synchronous replication slow failover down?
Not measurably. With synchronous_mode: true a crash took 23.2 seconds, within the async range. What synchronous mode changes is what you can lose: it guarantees zero lost committed transactions, at the cost of one network round trip on every commit. Async lost nothing in our runs too, but it can't promise that.
Why a switchover still takes ~5 seconds
Even a planned switchover interrupts writes: the old leader must stop accepting writes, flush its last WAL to the candidate, and restart as a replica, while the candidate is promoted. Existing connections are cut. Your application needs to reconnect. That's why a connection pool with automatic reconnect matters as much as anything in the database layer.
Measure your own cluster
Your network, disks and HAProxy placement will produce different numbers. Measure before you promise an RTO to anyone:
- Build a staging cluster that matches production.
- Run a writer through the same connection path your applications use.
- Test all three: switchover, clean stop, and a real crash (
SIGKILLto Patroni and PostgreSQL). - Check lost rows, not just downtime. Confirm the old leader rejoins by itself.
- Repeat each test several times and report the range, not the best run.
The Twinhull kit automates all of this
failover-drill runs the writer, performs the switchover, stop or crash, verifies every committed row and writes a dated Markdown report. The same tests produced every number on this page.