Validating Complex CreditLens Migrations for a Top Bank - 26.6M Records Checked in 21.5 Minutes
At a glance -
the essentials
One of the Top 5 Canadian Banks
- The bank's Credit Risk Technology’s CreditLens Migration Testing business needed to migrate data from CreditLens into a new in-house Financial Assessment application across roughly 9 main transaction tables with complex, varying JSON structures - including array and scalar array fields - that had to be compared as part of the migration. The source and target data structures were different, and every migration-related fix required revalidating data that had already been migrated. With tight delivery timelines and no appropriate tools available during SIT and UAT, comprehensive regression testing was difficult to execute consistently, and the same validation activity had to be repeated across multiple testing environments and phases, making the process operationally heavy and prone to inefficiency.
- Flatten the nested JSON data from the CreditLens source tables into a common, comparable structure
- Flatten the JSON data from the post-migration target tables in the in-house Financial Assessment application to the same common structure
- Use DQV to compare the flattened source and target data across all 10 tables, down to the field level
- Run DQV comparisons periodically throughout the SIT and UAT phases, re-validating migrated data after every migration-related fix
- Continue this comparison cycle until the migration scripts were confirmed to run cleanly and were promoted to production
- 26.6 million records across 10 tables (spanning columns from 5 to 523 per table) validated automatically, at a scale that would not have been practical to check by hand
- Full source-to-target comparison completed in 21.5 minutes, letting the team re-validate quickly after every migration-script fix
- 2 of 10 tables fully matched; 8 of 10 tables had mismatches
- 2.03 million mismatched records identified - a 7.6% mismatch rate overall
- Field-level mismatch reporting pinpointed exactly where and why records diverged, speeding up defect triage and resolution
- Repeatable DQV comparisons closed the multi-environment testing gap, running consistently across SIT and UAT without extra manual effort
- Comparison process reused after every fix cycle, giving the team a dependable regression check all the way through to production go-live
From complex, nested-JSON migration testing to 26.6M records validated in 21.5 minutes.
Kumaran partnered with the Credit Risk Technology team at one of the top 5 Canadian banks to validate a complex data migration from CreditLens into a new in-house Financial Assessment application. Using DQV, the team flattened nested JSON data from both source and target systems into a common structure and ran repeatable, field-level comparisons across every migration-related fix throughout SIT and UAT. The result is 26.6 million records across 10 tables compared in just 21.5 minutes, with detailed mismatch reports guiding defect resolution at every stage.
The Background
The CreditLens Migration Testing effort involved moving transaction data - spanning hardlock, user, address, customer, financial, and statement tables, among others - out of CreditLens and into a new in-house Financial Assessment system. Several of these tables carried complex, varying JSON structures with array and scalar array fields, and source and target data structures didn't line up directly, so a straightforward comparison wasn't possible. Every fix to the migration logic meant previously migrated data needed to be re-checked, and with tight delivery timelines and no purpose-built comparison tooling in place, running that regression consistently across SIT, UAT, and later phases had become a real operational strain on the team.
Want to know how we achieved these results?
Contact us today to learn more about our DQV-driven migration testing approach and the success story behind this engagement with the Credit Risk Technology’s CreditLens Migration Testing team at one of the top 5 Canadian banks.
