What Your Five-Minute Failover Number Isn't Telling You
Somewhere in your organization there is a slide that says failover completes in under five minutes. It was produced by a replication platform, validated in a test window, and it is almost certainly true — in the narrow sense that the technology can move a workload from point A to point B in 300 seconds.
It is also, in most enterprises we assess, misleading. Not because anyone lied. Because the number answers a question nobody actually asked.
The Metric and the Mission Diverged
RTO as reported by a vendor dashboard measures one thing: how quickly the platform can complete a mechanical cutover when everything it needs is available, current, and expected. Real outages provide none of those guarantees. What they provide instead is a cascade of complications the dashboard has no field for:
- Stale credentials. The break-glass account that expired, the service principal whose secret rotated during the last quarter's security sweep, the VPN certificate nobody remembered was load-bearing.
- Untested dependencies. The application fails over beautifully, then waits on a batch scheduler, an on-prem license server, or a third-party API that was never in scope for replication.
- Unqualified data. Replication is not validation. The first proof that last night's snapshots were consistent is frequently the first morning of the outage.
- Human latency. The five-minute figure excludes the 40 minutes of conference calls, approval chains, and runbook archaeology that precede anyone touching the console.
None of this appears in a vendor's marketing. All of it appears in post-incident reviews.
Why Dashboards Can't Save You
There is a structural reason vendor-reported recovery numbers flatter reality: the vendor is measured on the platform, and the platform is the easy part. The hard parts — credential hygiene, dependency mapping, organizational muscle memory — live inside your enterprise, and no dashboard can observe them.
This is not an argument against replication tooling. It is an argument against confusing a capability with a readiness. Capability is what the system can do. Readiness is what your organization can do, at 2 a.m., with half the on-call roster unreachable and the person who wrote the runbook having left for a competitor in Austin eight months ago.
The Case for Drills That Hurt a Little
The only measurement that predicts real recovery is a controlled failure with full business involvement — not an IT-only tabletop exercise, not a passive walkthrough. We mean actually degrading a service, during business hours, with finance, customer support, and operations in the room experiencing the consequences.
The discomfort is the point. A drill that hurts a little surfaces the answers to questions no checklist can:
- Who has authority to declare the failover, and can they be reached?
- Do the runbooks match the current architecture, or the architecture from two re-platformings ago?
- Does the data that comes back actually reconcile with the business's expectations — orders posted, payments recorded, inventory accurate?
- Where does the recovery plan quietly assume a person, a system, or a vendor that no longer exists?
A Checklist for Designing a Real Drill
- Pick a service with real stakeholders, not a sandbox. Choose something whose absence is felt, ideally within 24 hours.
- Run it during business hours. Nightly drills test the technology. Daytime drills test the organization.
- Freeze nothing. Do not pre-rotate credentials, pre-warm caches, or fix the known gaps the week before. The gaps are the curriculum.
- Time from declaration, not invocation. Start the clock when the business notices the problem, not when the engineer types the failover command.
- Validate data, not just uptime. Bring in the people who can confirm the recovered state is trustworthy, not merely available.
- Debrief within 48 hours, in writing. Every gap gets an owner and a date. A drill without remediation is theater with a timestamp.
Readiness Is a Practice, Not a Number
The enterprises that recover well are not the ones with the best slides. They are the ones that have failed on purpose, recently enough that the runbooks are warm and the escalation paths are memorized. Their five-minute failover figure may be no better than yours. Their actual recovery, when the real outage arrives, will be unrecognizably better.
Schedule the drill. Let it sting. The alternative is learning the same lessons during an outage, at full price.
