Skip to main content
Theme

Explanation

How a migration works

Every migration is the same four stages: extract, transform, load, verify. Each one can be looked at on its own, and nothing reaches the destination until a person has read the plan and approved it.

  1. ExtractRead the whole source into an encrypted copy
  2. TransformApply the rules: rename, match, redact
  3. PlanMeasure it against the destination
  4. You approveNothing is written before this
  5. LoadRows with their ids, people, code, files
  6. VerifyEvery check, each tested first on something wrong

Extract

In the tool this is the Analyze step, the first step of Transform. The tool reads every record table, every person with their profiles, your code, file references, constants' names and each module's version, into a copy encrypted with the tool's own key. It counts what the instance says it holds first, and every later check is measured against that count. Constant values and password hashes are never put in that copy.

Transform

The rules that make v5 data fit v6: the CRM's tables are copied under their v6 names, lookup values are matched to the ones a new v6 instance already has, and anything that looks like a key is held until you decide. Each rule reports what it changed.

Plan

The transformed copy is measured against the live destination: which tables exist, which need creating or extending, which ids are taken, which people already exist. Anything that would fail part way through the load stops the plan instead, with a fix.

Load

Rows are written with their original ids and dates. A row whose id the destination already uses gets a new one, which is safe because v6 relationships use uuids, not ids. A row the destination could only take by writing over a different row stops the plan instead. Before anything on the destination is changed, the way it was is saved, so a rollback can always put it back. A load that runs out of time stops between batches and carries on, and running it again writes nothing twice. A big load runs in parts of about ten minutes, and each part starts after the databases the parts before it finished, so nothing finished is read again. A new plan, or carrying on after a failure, checks every database again.

Verify

Every row, person, profile, password, file and page is checked against the source. The checks.

Why copy first

A migration can run over a v6 instance that already holds an earlier copy: rows that are already there are compared, brought up to date where the source changed them, and never duplicated. So the safe order is to copy the site while it stays live, check it, and at cutover copy only what changed. Copy first, then cut over.