What is “Data Migration”?
Whenever I talk about data migration, I always think about moving. Think about data migration in a moving situation, what will you do if you are moving into a new place? Moving data is the same way, what you will do is moving the data from one system and put them into another, just like moving. You will pack everything up in your old house and move into the new one. Now here comes the tricky part, your old house might be a lot smaller than your new house, or maybe your new place is a four-bedrooms house instead of your old one bedroom apartment now. What do we do in that situation???
I have done a couple of data migrations by now, each of them have a similar resemblance but yet they are all different. There are couple types, and I love to use moving as an analogy when I explain data migration to people, which I am going to use that here. Hope this helps you understand data migration a little better.
Moving between 2 similar system Link to heading
Data moving between two similar system, it is pretty straight forward, that’s like moving from 3rd floor of your current apartment building to 4th floor, the configure of the apartment is exactly the same, and you know exactly where to put everything, the same spot where you were putting them, haha. This is assuming that you don’t want to reconfigure your space, the reconfiguration should come after you moved all of your stuff into the new space! This is what we call lift and shift.
Moving between 2 similar system with different size Link to heading
This is still a lift and shift, however, this is the case where you move you cannot move all of your old stuff to the new place, so you will need to get rid of stuff, either sell the old furniture or donate them to Goodwill, for the rest of the stuff that cannot fit, you may consider putting them into a storage unit. Same as when you move data, you would probably move the most relevant and transactional data into your environment, if you really would love to keep all of the data from 1900, then we can put that into a data warehouse, and only keep the supporting data into the new system.
The benefits of doing it this way is you can 1) save space in your current space, 2) less data in the current system allows data retrieval to be more efficient, 3) easier to construct data
With a data warehouse, you can still preserve all of your data, you can still track the historical data and run reports on it, if you are interested in running machine learning or training your own AI, you can still use those data, and they are in a concentrated place, which is easier to access.
Moving between 2 different system Link to heading
This is the tricky one, it is like you were living in an apartment then moving to a house, or moving from a house to an apartment. The floor layout is all different, you need to reconfigure your space to figure out where to put what. This goes the same compare to the data migration here. When you move data from one system to another whole different system, you will not only need to consider data, but also you need to consider the functionalities as well.
All those direct to BUSINESS PROCESS, BUSINESS PROCESS, BUSINESS PROCESS, important things need to be repeated three times. People don’t necessarily know what functionalities they would like to bring over, they will tell you they want everything the same as the good old days, it is also my responsibility to remind them that “Hey, if we can make it better, why not? so what was your pain point?” The solution can come in later, I always believe in that the key is to get the requirements and understand what can truly help them, then we can think about solutions. There are different ways of solving problems, as long as you know what the problems are. :)
Among all those craziness, the data is the key to any system, data is the one to support your process, data is also the one to show on your front web page, all in all, without data, the system is just an empty shell. The crucial part here is how do you manage the data when they are coming from a different system. In the old system, the data is supporting a different set of business process, and they exist for that specific reason. In the new system, we now have a whole different process we need to support, then of course, the data structure/data model will need to change as well. The new data model needs to not only be able to hold the data from the old system, but also they need to make sense in the new system. This is by no means an easy task. You will be constantly finding yourself dealing with 5 or 4 tables from the old system that need to be merged into 1 single table in the new system, or you will need to merge 2 tables from the old system and then split them into 3 different tables for the new system. It requires you to understand every single bit of the old system, and know what will happen in the future in the new system. As the data architect, you will need to have the data vision aligned with every single part of the project.
Moving between 2 different system with different storage size Link to heading
Moving between 2 different system is not easy as we mentioned before, now if you factor in the different storage size, you really need to pick the bread and butter out of the weeds. I always tell people, think about the things you cannot live without. It would be easier to grab everything and move, but it would be impossible if you are moving from a house with 3 bedroom to an apartment, and you want to take everything with you, unless you have a big storage room. Take your essentials!
Summary Link to heading
The whole point of data model and database is to help you down the road to retrieve data easier, and to save your data in a more efficient way. Depends on how you are showing data in the front, or how you would like to retrieve data, you would figure out how to save the data in the backend. To put this in a different way, when you watch Marie Kondo, you will notice that she organize the items in a way that it would be easy for you to retrieve, and be visible. Honestly, now I am thinking about it, the whole system migration is basically using Marie Kondo method to organize your house. In the end, you will be with a system which spark joy for you!
I like work with data, it presents itself with different kinds of challenges, but I love a challenge like always. :) Working with data not only requires you to understand the data, but most importantly, you need to have the capability of fish out the most important information out of million or billion amount of data; also present those data with meanings to users. Data without context is just useless information.
In the end Link to heading
Overall, I see an architect as a product manager, system architects are the ones design the system, the system is your product. Thus, as an architect I truly believe you need to understand all perspective, you may not need to understand everything in detail, but you should have a general idea. Hehe.
Happy Architecting!
About me Link to heading
I am Xuan Jin, I am a system architect, and help clean up and help fix things when needed. All those fixes and helping others built me up to who I am today. My superpower is to learn new things quickly, both by getting my hands dirty and read documentations. I also run a group of people who are passionate about salesforce, where I help answer any questions they may have and encourage everyone to help each other.