Data Migration: Moving Data to a New System

Short answer: data migration is not a copy job. It is three jobs at once — field mapping, data cleaning and reconciliation — which is why it is almost always the line item that delays a new software project. There is only one measure of success: once the migration is done, do the record counts and the monetary totals match exactly between the old system and the new one? And it is never a single event; a real migration is rehearsed two or three times before go-live.

Why migration takes longer than anyone estimates

In a quote, migration usually shows up as one line: "existing data will be transferred". In reality the duration is driven by data quality, not record count. Here is what we run into most often.

• Duplicates: the same customer recorded three times with three spellings. Software cannot decide which one is correct — that is a business decision, and the person who makes it should be named up front.
• Empty mandatory fields: if the new system requires a tax number and thousands of legacy records have none, the migration stops. Either the field becomes optional temporarily, or the data gets completed in bulk.
• Free-text fields: what the old system kept in one "address" or "notes" box gets split into five fields in the new one. Parsing is never a hundred percent accurate, so leave room for manual review.
• Format differences: date order, decimal separator, currency, tax inclusive or exclusive. A single decimal mistake can shift every balance by a factor of a hundred.
• Character encoding: broken accented characters in exports from legacy systems are a classic. Agree on UTF-8 before you accept the files.
• Broken relationships: the order moved, but the link between the order line and the product did not. This is the most expensive class of error because it surfaces weeks after go-live.

The duration of a data migration is set by data quality, not by record count. Five thousand clean records move in a day; five thousand dirty ones can cost two weeks.

First decision: what actually moves

The most effective way to cut migration cost is not negotiation, it is narrowing the scope. Not every record needs to enter the new system, and a blanket "let us move everything" typically adds thirty to forty percent of unnecessary work.

• Always moves: open balances, active customers and suppliers, current stock, open orders, active users and permissions, the product or service catalogue.
• Usually moves: the last one or two years of transactions, which you need for report comparisons and seasonal analysis.
• Usually does not move: ten years of history, closed records, old campaign data. A read-only archive database or a one-off report export is cheaper and entirely sufficient.
• Never moves: unused fields, test records and personal data you no longer need. A migration is a natural opportunity to clean house.

This is a budget decision as much as a technical one; we broke down how scope drives price in the cost of custom software.

A four-step method

• 1. Inventory and profiling: how many records arrive from which sources — a database, a set of spreadsheets, an accounting export? For each table, measure empty-field ratio, duplicate ratio and outliers. Without that report, any duration estimate is not even a guess.
• 2. Field mapping table: source field, target field, transformation rule, owner. This table is the contract of the migration; every later argument is settled by looking at it.
• 3. Dry run: run the whole thing end to end with real data into a test environment. The first dry run is almost always wrong — finding those errors is its purpose. Do at least two, and run the last one on an environment identical to production.
• 4. Cutover plan: the hour the old system freezes, the delta migration, the verification list and the rollback scenario, all written down.

Reconciliation: not "done", but "it matches"

You do not verify a migration by looking at screens. Every run should end with an automated reconciliation report: do record counts and monetary sums agree on both sides, and are there any orphaned records? That report is the only sound basis for the go-live decision.

-- Check 1: do record counts and totals match on both sides?
select 'source' as side, count(*) as records, sum(tutar) as total
from legacy.siparisler where durum <> 'iptal'
union all
select 'target', count(*), sum(total_amount)
from public.orders where status <> 'cancelled';

-- Check 2: orphans (order moved, its customer did not)
select o.id, o.customer_id
from public.orders o
left join public.customers c on c.id = o.customer_id
where c.id is null;

-- Check 3: the riskiest field - customers whose balance changed
select c.id, c.title, l.bakiye as old_balance, c.balance as new_balance
from public.customers c
join legacy.cariler l on l.kod = c.legacy_code
where round(l.bakiye, 2) <> round(c.balance, 2);

If any of the three queries returns rows, the migration is not finished. Run them during the dry runs too: a shrinking result list from one rehearsal to the next is the only objective sign of progress.

Cutover day: freeze, delta, parallel run

The cutover plan should fit on one page, because nobody reads long documents on the day. The sequence usually looks like this.

• Freeze window: data entry into the old system stops at an agreed hour, usually over a weekend or after hours, and it is announced to the team in advance.
• Delta migration: records created between the last dry run and the freeze are moved. Skip this step and the orders entered two days earlier simply do not exist in the new system.
• Verification: the reconciliation queries run and the result is shared in writing. Name the person who signs it off before the day starts.
• Parallel run: keep the old system available read-only for a while. Do not rush to delete it; the need to look something up in the first month is close to guaranteed.
• Rollback scenario: write down under which conditions you go back, and how many minutes that takes. The logic is the same as the RPO and RTO thinking in data backup and disaster recovery.

How migration is priced in a quote

Migration is the hardest line item to fix-price, because the thing that drives the price — data quality — is invisible when the quote is written. A healthy quote separates three things.

• Number of sources: migrating from one database is an entirely different job from migrating four spreadsheets plus an accounting package.
• Who cleans the data: only the operating business knows which record is the correct one. Without a clause stating that cleaned data is supplied by the client, the duration becomes unpredictable.
• Number of dry runs: two is standard. A third is usually the consequence of a scope change and is extra work.
• Acceptance criteria: write down when the migration counts as accepted — normally when all three reconciliation queries come back clean. Without that clause the project never formally closes. We covered what belongs in writing in the software development contract.

Data protection: migration is a natural cleanup

Legacy systems accumulate personal data that no longer serves any purpose: decade-old job applications, records people asked to have deleted, identity details nobody needs. Moving them across means moving the liability across too. Two practical rules: migrate only the data whose processing purpose still stands, and never use real personal data in test environments — mask names, phone numbers and email addresses. The wider principles are in building a data-protection compliant website.

Conclusion

Data migration is the most underestimated and most schedule-damaging step of a new software project, and its difficulty is not technical — it is the sheer number of decisions. Narrow the scope, write the field mapping table first, rehearse at least twice, and judge the result by the reconciliation report rather than by the word "done". If you are planning a move from an existing system, let us map the scope together: take a look at our custom software service or request a quote to talk through your data sources.