Data Migration (Integration) principles in Microsoft Dynamics 365 Deliveries

Hello,

As COVID-19 epidemic is ongoing and keeping us at home, I would like to share my thoughts and experiences about the position of "Data" during Dynamics 365 deliveries.

Information is regularly generated by the daily business activities through multiple channels. It is vital to track the information with necessary applications by converting it into data then storing data among systems for an organization so we should consider data in legacy systems for most Dynamics 365 deliveries.

Since data might be distributed in the systems, It is challenging to consolidate business information that is coming from legacy systems, in the new delivery.
I do not remember any CRM projects that were not asked to skip existing customer details in Microsoft CRM :)

Data are mostly considered very late, last quarter of the project life-cycle. I think, It must be taken care early phase of any deliveries and project teams should discuss data migration as well as data integration while designing the Dynamics 365 deliveries.

Here are the itemized criteria to be considered for Data <--> Dynamics 365 relationships

1. Data Ownership: We should clarify ownership of the data before Dynamics 365 project go live. Data integration goes through accordingly.
   
   For instance;
   If the project is about Dynamics 365 CE delivery, Customer information should be handled by only Dynamics 365 CE in the organization whereas Product related details should be kept by Dynamics 365 Finance&Operations, as long as F&O is used by the client.
    
2. Data Regulation Concerns: Legacy systems frequently do not have clear and defined data regulation(s) compliance or sometimes out-dated or even not comply at all so we should carefully consider regulations for data before migration and we have to clarify data consent.
    
3. Data Mapping (Transitions): Legacy system data structure mostly does not meet Dynamics 365 data character.
   
   For instance;
   Legacy System may keep only customer Full Name in single field while Dynamics 365 CE keeps First Name - Last Name, two separated fields .
   
   This is the simple case. Sometimes we face complex situation like, we may need to map fields of one entity in legacy system with couple entities in target Dynamics 365.
   
   For instance;
   Legacy system may not split customers into corporate and individual just keep all customers under Account entity so we may need to map Legacy Account entity into Account and Contact (different) entities in Dynamics 365 CE based on customer type.
    
4. Data Cleansing: Data Cleansing (or Data Scrubbing) is the action of identifying and then removing or amending any data within a legacy system that is:
   
   - Incorrect
   - Incomplete
   - Duplicated
   - Irrelevant/unnecessary (under GDPR)
   
   We need to identify and resolve data quality problems during data migration process from legacy system into Dynamics 365 system.
    
5. Archive: Do we really need to migrate all data from legacy system to the new Dynamics 365? It is crucial to understand demography of data and should not move unnecessary data to the Dynamics 365 so Archiving data could save us huge amount of data to display Dynamics 365 users. Unnecessary information increases confusion for end users.
    
6. Data Governance:  make organizational's data assets guarantee generally understandable, correct, complete, trustworthy, secure and discoverable. Data should be tested a lot during data migration.
    
7. Just in Time Data Served: Kindly inviting you to think fetching records from legacy system when they are needed.
   
   For Instance:
   Supposing end client uses Dynamics 365 CE so we can start customer data movement at the first call of the customer in call center.
     

Please avoid using of CRUD operations through SQL Statements for any data migration and integration, just follow supported way.

There are tools in the market or you can develop in-house solution or Microsoft out of box utilities.

happy delivering !