Audit-proof data migration in validated environments

31. March 2023

Data migration strongly adheres to the core principle of Data Integrity, which is one of the most critical GxP inspection focus areas and a core aspect of Computer System Validation (CSV).

Data migration refers to the process of transferring electronic data (including documents and configuration settings) from one system, also known as a legacy system, to another. In validated environments especially, data migration is a complex process that requires careful planning and execution to ensure that data is accurately and completely transferred from the legacy system to the new system with minimal disruption and downtime.

DOWNLOAD FACTSHEET

Key principle

According to ALCOA+, data integrity ensures that data is Attributable, Legible, Contemporaneous, Original, and Accurate throughout its entire lifecycle from creation to decommissioning.

The essential control logic in the flow of CSV activities must also be guaranteed during data migration:

1. Planning and Specification
2. Data Migration
3. Verification and Reporting

A data migration strategy, plan or specification with mapping tables must be approved before data migration takes place in a qualified and controlled infrastructure environment. A migration plan shall determine both the legacy and the new systems, define which data will be migrated and which will not, approach/tools, and if data will be transferred identically, changed or enriched.

Quantitative & qualitative verification

To ensure correctness, completeness, and consistency, it is vital to perform both, a documented quantitative and a qualitative verification. Verification must cover all migration relevant infrastructure environments including production.

Quantitative verification typically involves counting, measuring, or comparing data. Verifying the number of records or columns transferred, checking the data type and format, and comparing the migrated data to the source data for consistency are all examples of quantitative checks. In most cases this is done by a qualified tool which counts the data and checks the data formatting on a technical level.

In contrast, qualitative verification typically involves business expertise reviewing the content of the data to ensure the records make sense and serve their intended purpose.

For a thorough qualitative verification, we typically recommend the following approach:

1. Decide which fields and data elements are essential to the application or business process by collaborating with subject matter experts.
2. Define the sample size, acceptance criteria and necessary confidence level (% of accuracy) based on a statistical significance calculation related to data criticality and complexity (e.g., at least 95% of all migrated data are correct and complete).
3. Verify the essential elements of the migrated data sample for correctness, ensuring that all required content is present and accurate by comparing the entries to those of the legacy system by a subject matter expert. Qualified automated tools can support and scan the data within the system to identify misspelled text fields or formatting errors in date fields.