How We Secured a 3.3M-Record Dev/UAT Migration for a Leading Bank - Masked, Encrypted & Validated in 14 Minutes
At a glance -
the essentials
One of the Top 5 Canadian Banks
- The bank's Portfolio Project Management business needed to migrate 3.3 million production records across 61 tables into the Dev/UAT environment for realistic application performance testing. The data included sensitive columns that had to be masked before landing in non-production environments, while preserving referential integrity across related tables, and several PROD columns were encrypted and needed to be decrypted during extraction and re-encrypted in UAT using different encryption keys. No existing process could move data of this scale and sensitivity quickly while guaranteeing it arrived accurately in the target environment.
- Run the migration as a two-phase pipeline required by technical restrictions: PROD data extracted to CSV files, then loaded from CSV into the UAT/Dev database
- Use DQV to profile production data and mask sensitive columns in-flight during CSV processing, before any data reached the non-production environment
- Decrypt encrypted PROD columns during extraction and re-encrypted them in UAT with different encryption keys, so no production key material was reused downstream
- Migrate all 3.3 million records across 61 tables - including 24 tables containing 62 masked sensitive columns - while preserving referential integrity between related tables
- Run a full source-to-target comparison after migration to validate every migrated record and flag any mismatches
- 3.3 million records across 61 tables migrated and made available for realistic performance testing, with production-like fidelity
- 62 sensitive columns across 24 tables masked deterministically in flight, protecting sensitive data without breaking referential integrity
- Encrypted PROD columns safely decrypted and re-encrypted with different UAT keys, with no production key material reused in lower environments
- Automated field-level comparison confirmed all 61 tables fully matched, giving the team confidence in migration accuracy
- Entire pipeline - migration plus validation - completed in 14 minutes (7 + 7), instead of a slower manual process
- Repeatable process now available for future Dev/UAT refreshes on this portfolio
From a two-phase, security-sensitive migration to 3.3M records validated in 14 minutes.
Kumaran partnered with the Portfolio Project Management team at one of the top 5 Canadian banks to build a secure, repeatable pipeline for refreshing Dev/UAT with production-like data. Using DQV, the team masks sensitive columns and handles encryption key changes in flight while moving data through a two-phase PROD-to-CSV-to-UAT process, then runs a full comparison to confirm every record landed correctly. The result is 3.3 million records across 61 tables migrated, masked, and validated in just 14 minutes, with all 61 tables fully matched.
The Background
Portfolio Project Management needed realistic data in its Dev/UAT environment to support application performance testing, but production data couldn't be copied over as-is - sensitive columns needed masking, referential integrity between related tables had to be preserved, and several columns were encrypted with keys that couldn't be reused outside production. Technical restrictions also meant the migration had to run in two phases, extracting to CSV before loading into the target database, which made a well-validated process even more important to get right.
Want to know how we achieved these results?
Contact us today to learn more about our DQV-driven secure data migration approach and the success story behind this engagement with the Portfolio Project Management team at one of the top 5 Canadian banks.
More proof
RECORDS VALIDATED PER MINUTE
1.24M
Validating Complex CreditLens Migrations for a Top Bank - 26.6M Records Checked in 21.5 Minutes
The bank's Credit Risk Technology's CreditLens Migration Testing business needed to migrate data from CreditLens into a new in-house Financial Assessment application across roughly 9 main transaction tables...
RECORDS COMPARED PER MINUTE, END TO END
1.9M
How We Automated ETL Regression Testing at Scale for an Enterprise Bank - 200M Records Validated in Under 2 Hours
The bank's Equity Markets & Prime Brokerage Technology business ran roughly 800 batches in production. Whenever batch logic changed, the team needed to regression-test every affected ETL job...