Data Migration Checklist

Data migrations, sometimes referred to as data conversions, are a common, if not necessary step in the growth of any nonprofit organization. Both donor data and program outcomes data expand exponentially as the nonprofit grows.

Furthermore, data migrations are notorious for creating as many problems as they solve. Why? Everything from unrealistic expectations to a lack of planning and documentation to the “pealing back of the onion” where the more work that you do, the more issues you find with the data.

 A successful data migration depends on planning, realistic expectations, experience, and flexibility during the process. Planning … write it down … like a data migration checklist.

 Remember this simple rule: you can’t take all of your data with you to the new system, nor should you want to.

 To help you with your plan, start with a good data migration checklist. Here’s one.

 Pre-migration planning – 6 steps:

  1. Team

    Establish a migration management team. Include key stakeholders (ex. marketing, fundraising, leadership). In particular, include the person(s) responsible for configuration management of the new system, if this is not the same person(s) managing the data migration.

  2. Plan

    Document your plan and include a flexible schedule. Identify the tough decisions upfront and address them. For example, how much data cleaning is required with your migration, and when should that cleaning occur? Do you have legacy fields that need to be parsed (i.e., separated into more than one data field)? Look to your data governance policies for help.

  3. Establish security protocols

    Make sure everyone understands the ground rules. Create new login credentials for third parties working with your systems. Who can access the data, where can it be stored, and what flexibility does a consultant or CRM vendor have in working with your data?

  4. Prioritize

    Prioritize the reasons why you are migrating. For example, if you are moving to a new CRM system in order to support expanded fundraising campaigns, then focus on the features and benefits of working with new data in the new system over the stress of trying cram more of your poorly formatted or incomplete legacy data from your old system.

  5. Define standards

    Define the standards for a successful migration. Most importantly, determine what data should and should not be migrated. For example, in a CRM data migration, do you want to migrate a record that has not been edited in 5 years? 10 years? Another example, how will you handle file attachments?

  6. Write a test plan

    Develop a test plan to determine if you are meeting the standards for success. Do not wait until after the migration has been attempted to figure out how you will measure success. Larger data migrations usually benefit from independent validation resources, including software – if you need them, line them up now.

 Migration tasks – 14 steps:

  1. Analyze the data, revise the plan

    No data migration project gets very far without a thorough data analysis. This will determine the starting point for your data quality, uncover potential problems, and either affirm the original schedule and plan or cause them to be revised.

  2. Map the data

    Map the data schema from the legacy database to the data schema in the new system database. Identify inconsistencies, missing fields, and fields requiring either consolidation, conversion or parsing. This is a time-consuming step that gets short-changed when organizations are in a rush to complete the migration. For example, if you are migrating from an out of date donor database to a new CRM, the differences in data schemas can be substantial.

  3. Configure the database

    Configure the new database system. Pay attention to field attributes. Be sure to check storage capacity required to support the migration import – and take a moment to consider projected growth.

  4. Prepare supporting technology

    Prepare any data migration software or custom scripts being used to support the data load to the new system. For example, will the data migration require conversion scripts?

  5. Test, test, test

    We can’t stress this enough. Create a test file from legacy and import to the new system database. For example, test the accuracy of the imported records. Test new system data management and reporting. Test all export capabilities of the new system. In particular, how do you extract all of your data in the future, when you decide to migrate to another CRM?

  6. Re-configure the database

    Based on test results, make any necessary changes to the new system database configuration.

  7. Extract the legacy data and create new import files

    Extract the necessary legacy data – this may require multiple steps. Then prepare new import files for uploading.

  8. Normalize and clean

    Normalize poorly formatted records, purge corrupt data. Apply additional cleaning steps (e.g. de-duplication) now if this is the appropriate time in your plan. Otherwise, you will need to practice data hygiene post migration.

  9. Load data file(s)

    Import the full data. Pay attention to load times, file orders, interruptions, and other data management best practices. Any bad records found in the export file should fail on import. Compare raw results of the load – numbers of records expected to import, number expected to fail, file size, etc.

  10. Test, test, test

    Re-run your tests, following your test plan and any revisions from prior test periods. If you encounter data quality problems with the import, research, repair, and repeat until you achieve the standards for a successful migration. Your validation work may identify additional changes required in the database – make them.

  11. Final cleaning

    Assuming that your plan did not address all data cleaning steps as part of the migration tasks, now is the time to revisit remaining data hygiene. Do you have de-duplication work or planned data consolidation? Did you have additional data files schedule to append your newly migrated records?

  12. Parsing

    Parsing is a task that we almost always recommend be addressed outside of the scope of the core data migration. Why? Because it is a project in and of itself. Parsing involves additional analysis, file preparation, database configuration and testing. It may warrant its own budget and schedule. Small amounts of parsing, like a name field being separated into first and last name fields, can be done as part of the migration, when the database is configured and the import files are created. But addressing multiple fields with lots of data to be address is often best managed post-migration. By the way, did you know that Microsoft Excel has parsing tools that may suit your needs just fine?

  13. Support

    Your work is done, but don’t jump ship yet. Turn over the new system data to the business users – that’s the “real” test, by the way. Keep the team assembled and support the new database post-migration for at least 30 days before concluding that the migration is finished and letting the team go. Make sure any consultants or CRM vendors who have assisted on the project remain available if needed.

  14. Archives

    Archives are your piece of mind. Retain a complete copy of the legacy database – you may need to return to it if you have missed any important data. Your especially want to archive any data not migrated. The good news is that you don’t need to keep a copy of your old CRM software running. Instead, use simple data storage tools like Microsoft Excel and/or Access.

Follow Us

Subscribe to Our Newsletter