Skip to content

BlogDeployment & releases

Why staging and production drift apart, and how to keep them in step

The usual causes of environment drift between staging and production, how to detect it before a release, and the practices that keep environments comparable without freezing them.

Abstract illustration of two parallel lines diverging and being brought back together.

Key takeaways

  • Drift is a process problem: changes reaching environments by different routes.
  • Compare configuration, versions, schema and infrastructure before every release, not after an incident.
  • Hand edits and hotfixes are the main sources; give them a reviewed path that is faster than the workaround.
  • Record the known differences so a release is judged against reality.

“It worked in staging” is the most common sentence in a post-incident review. The release was the same; the environment was not. Environment drift is rarely dramatic. It accumulates through small, reasonable actions until the staging result stops predicting the production one.

Where drift comes from

  • Hotfixes applied straight to production under pressure and never back-ported.
  • Configuration and secrets edited by hand in one environment.
  • Infrastructure updated out of band: a managed service version, a node pool, a network rule.
  • Schema changes applied manually to unblock someone.
  • Data that diverges in shape, volume or edge cases, so the same code takes different paths.
  • Feature flags set differently and forgotten.

How to detect drift before a release

Compare, do not assume. Before a release, diff the things that matter between the target environment and the one you tested in: deployed versions of every service, configuration values, schema, infrastructure definitions and flag states. The diff should be produced by a tool, read by a person and attached to the release so the decision to proceed is made against the real target.

How to keep environments in step

  • Give every change one path. If a hotfix must go straight to production, it goes through the same review and record, just faster.
  • Define infrastructure and configuration as code, and apply it to every environment from the same source.
  • Run migrations through the release plan in every environment, never by hand.
  • Refresh staging data on a schedule, masked where needed, so its shape tracks production.
  • Review flag states as part of the release, and expire flags that have served their purpose.

Accept known differences, document them

Some differences are deliberate: scale, cost, external integrations pointed at sandboxes. Write them down next to the environment so a release is judged against a known list rather than a guess. A documented difference is a constraint; an undocumented one is a surprise.

Dorvi keeps environments comparable by design: it holds each environment’s versions, configuration and infrastructure, shows the difference before a release, and records what changed and who approved it, across the clouds you already run.

Enterprise operations

Dorvi

Review every change with its context, ship services and migrations as one release, and keep the evidence.

Explore Dorvi

Questions

Before you ask.

What is environment drift?

The gradual divergence between environments that are meant to be equivalent, such as staging and production, in versions, configuration, schema, infrastructure or data, so that a result in one no longer predicts the other.

Is drift inevitable?

Some deliberate differences are, such as scale or sandboxed integrations. Accidental drift is not: it comes from changes reaching environments by different routes, which a single reviewed path removes.

How often should we compare environments?

Before every release to the target environment, and on a schedule in between. The comparison should be automated and the result attached to the release.