System Migration Approach

Migration of applications or systems from existing platform to new platform has always been challenging for organizations. When it comes to migration there are several aspects which are required to be considered including but not limited to choice of technology (among available technology stack), underlying platform, migration approach, implementation time, existing investments (on licensing of tools, softwares and platforms), available skillset, time to production and platform switch strategy.

In this paper we will be focusing on migration approach with the help of a use case. We already understand that every application or system is different and might require a different migration approach, intent here is to address the migration approach in a generic way and present a perspective which I have developed during my time in industry. I hope the approach we will discuss here will fit in most of the cases and trigger the critical thinking process which is required for any defining migration strategy and process. So, with this let’s START.

To start on the right foot, we need a use case or a customer requirement where migration is needed from one platform to another platform. Let’s define a requirement for this paper.

Customer context and requirement

A multinational vehicle manufacturing customer having business spread across the globe has an IoT platform for its connected vehicle system (commercial and private vehicle divisions).

Old Platform is an inhouse developed system by various internal teams. System has evolved organically over the period of time; development teams has developed whatever functionality they want to build whenever they want to build. While this approach has given them some of the best industry leading features and has helped grow their business exponentially, but along the way this has added more complexities to the systems which are heavily dependent on people having knowledge about the tools and technology choices they made during development. Ambiguity is one of the problems which was introduced because of this approach and various teams are having different solutions to solve similar problems like device connectivity, data processing and communication. Any change in the system requires significant effort in testing and there is always a consistent fear of things falling apart.

From infrastructure perspective Old System is deployed on on-premise data center and customer is spending a considerable amount of money as licensing fees for various software licenses employed by the development teams and on required infrastructure and management of the infrastructure.

With evolution of cloud technologies and to overcome current challenges and cost effectiveness in mind customer developed a new Platform leveraging cloud technologies. New Platform is developed leveraging Azure IoT services with PaaS first approach. With development of new platform customer has realized that new system is more cost effective as there is no need to pay any licensing fee upfront, high availability and scalability is available on demand, billing will happen only for the resources consumed which reduced cost on infrastructure and there is no need to baby sit the infrastructure for routine updates like OS and security patches.

At present these both platforms are up and running, catering two different geographical regions. Old system is deployed and supporting business in US region. Whereas the new system is developed as cloud native solution and catering to Europe region. Customer is looking forward to having a single unified IoT Platform to cater all geographies.

Approach

With above set context and requirement let’s try to build a platform migration approach. So, first let’s summarize our understanding of the problem. Here it goes.

As we understand that at present customer has two IoT platforms catering two different regions for two different business units i.e. Commercial Vehicle and Private Vehicle divisions (Europe & US respectively).

IoT platform for the US region is custom developed leveraging various tools and technologies as foundation component for IoT Platform and for Europe region IoT platform is custom developed leveraging Microsoft Azure IoT hub and related IoT services.

Customer has taken a strategical decision to standardize IoT platform across business units/verticals by leveraging azure IoT hub and related azure services with PaaS first approach.

Any system migration is combination of multiple activities divided in multiple phases. Activities for our case will include development of connectivity agent to connect OBD (on board device) to new platform, update firmware (FOTA) to rollout deployment of new firmware to existing OBD, migration of existing OBD connectivity to new platform, data migration and sync between two platform, changes in telemetry data process flow and related business applications and respective 3rd party data eco system integrations.

Migration solution approach is primarily divided in two phases assessment phase and implementation phase.

Phase-1 — Assessment

In this phase various teams do an assessment of existing platform application and technology landscape. This will be a deep dive activity with customer’s platform teams to understand technical details like device and platform connectivity, protocols, APIs, application interfaces and various integration points with other enterprise or 3rd party systems. Assessment teams will capture and address existing platform pain points (if any), areas of improvement and prepare a migration plan covering short term, intermediate and long-term goals and implementation roadmap aligned with customer’s enterprise strategy and future vision.

Assessment report will/should contain details around:

1. Current state of Existing Platforms containing details like current system architecture, system flow, dependencies and couplings between various services and components, identification of complexity in migration of components.
2. Number of existing defects/issues and any functional enhancements which are in progress within the platforms with severity and impact on both platforms during implementation phase.
3. Categorization of identified issues and recommended resolutions i.e. immediate fix before migration or post migration fix.
4. Architecture and system flow diagrams depicting transition of platform from current state to target state with detailed out intermediate state.
5. Implementation roadmap containing details about code level changes in existing platform, development & deployment of new connectivity agents, processing components, platform switch strategy for new and existing OBD devices, remediation strategy for unforeseen issues during platform switch.

Phase-2 — Implementation

In this phase development teams will start development and modifications to existing platforms. There are lot of ways to approach implementation as per individual or team experiences; for me I always prefer two phase/plans (short term and long term) approach for implementations activities.

In short term phase, focus will be on fixing any existing issues or any functional enhancements which are required to be fixed/completed before migration, partial switch, data migration and pilot with new devices and integrations related issues. Primary activities include:

1. Changes in connectivity agents for OB devices for platform connectivity (Old & New)
2. Device and platform interfaces changes
3. Integrations changes to enterprise applications and 3rd party systems
4. APIs (device onboarding and service activation)
5. Existing identified issues and ongoing functional enhancements which are categorized as critical and require to be fixed/completed before migration.
6. Establishing platform level connectivity, development of Azure IoT hub connector/adapter to collect data from old platform rest APIs and push data to new platform.
7. Creation of device models using device twins to represent old platform device models and related changes to the old platform back-end system to make it compatible with Azure back-end system.
8. Partial switch of the OB devices to new platform and data migration and sync activities.
9. Pilot run or parallel execution of both platforms to identify and fix any unforeseen issues. After successful pilot and complete switch activities will start.

Long term phase includes activities focused on stabilizing the new platform and phased switching of OB devices to new platform. OB devices can be migrated in two ways:

Remote migration where a new firmware will be deployed on existing OB devices (FOTA) having a connectivity agent developed based on azure IoT edge SDK.

1. New agent will stop the old agent and restrict it to connect to old platform and establish a connection with new platform in azure.
2. Once connection is established with new platform and all operations start working fine new firmware will remove old agent and remove all configurations related to connectivity with old platform.
3. In case of connectivity failure with new platform or any breaking changes firmware will rollback and connectivity will be re-established with existing platform.

Physical migration where existing OB devices will be brought in physically to the nearest facility where new firmware will be deployed on the devices and the firmware will perform all the operations as mentioned above section. Connectivity and smooth operations will be verified after deployment and device will be marked as migrated to new platform.

Benefits of Two phased Implementation approach

Two phased migration approach will help in moving towards the target state by introducing changes to existing system in a systematic (step by step) way. This approach gives the flexibility to revert to previous working state of the system in case of any issues, hence reducing impact on ongoing business. Below is the pictorial view of the journey.

Phase 1 (short term) will ensure the seamless integration between old platform to new platform making appropriate changes at both ends. Once connectivity is established and data integration is done successfully between platforms, then application level changes will be initiated at new platform to accommodate any existing functionality/feature which is not developed or supported by new platform.

After changes, both systems will run in parallel for some time (ideally 1–3 months) to identify any breaking changes or high priority issues. During this parallel run some operations will be transferred to new system to validate whether new system is ready for complete switch. During this phase activities for next phase (long term) will be identified and foreseen challenges will be resolved before start of the phase 2.

In Phase 2 (long term) new OB devices will connect to azure IoT Hub based platform and new firmware will be rolled out (FOTA) for existing OB devices to use new platform and send data to respective platform. Old platform will be phased out once all existing OB devices migrated to Azure IoT Hub based platform. Phase 2 is a long-term implementation (usually 1–2 years) depending upon number of on field devices and new features to be incorporated. Both platforms are required to up till the time all existing OB devices are completely switched to new platform to support ongoing business.

Conclusion

System migrations are always challenging and complex task. It demands experience, comprehensive and cohesive thinking, collaborative approach, open mindset and right skillset. Right approach and strategy help in reducing impact on ongoing business and helps streamlining the migration process by introducing changes to the systems in a systematic way.

Originally published at https://www.linkedin.com.