Use Spinnaker for Seamless Cloud Migration

In my role as Customer Success lead for OpsMx, I’ve had the opportunity to talk with literally hundreds of IT, Dev, and DevOps leaders from a wide range of industries and regions around the world, and virtually all of them are at some stage of a cloud migration journey. In this blog, I’d like to explore how Continuous Delivery (CD), and specifically Spinnaker as it’s becoming the standard for Open Source CD, can have a very meaningful impact on an enterprise’s cloud migration journey. 

Cloud migration is a de facto today

The strategy of moving workloads, both monoliths, and microservices, to the cloud, has many  obvious benefits for modern enterprises:

• • IT can decrease costs by at least 25%
  • Infrastructure availability is improved by more than 3X 
  • Enables deployment of the application to multiple regions
  • Improves the efficiency of infrastructure management by approx 50% 
  • And most importantly, it enables the business to innovate and scale faster

The benefits of cloud migration are clear, but do the strategies currently being used for cloud migration enable the level of scale needed by enterprises?   

Most enterprises commencing a cloud migration journey employ Rehosting ( Lift and Shift) or Replatforming( Lift, Tinker, and Shift) as their strategy because they’re the easiest ways to achieve digital transformation. They both involve sequential phases of: 

1. 1. Discovery and evaluation of applications for migration
   2. Evaluation and design of a migration strategy
   3. Prioritization (micro)apps and building of a migration plan
   4. Migration, Validation, and Operation ( in Prod)

Refer to the below picture which shows the stages of cloud migration:

However, there is a challenge in executing the migration journey. Many enterprises and SI partners champion the first three phases of rehosting migration strategies. A rehosting strategy needs to have the right technology in place to enable quick and hassle-free migration, however, by delivering applications “frequently” in a sustainable way and validating them at every step of workload migration. Without this, it can take several months (shown in the image above) to successfully lift and shift your code to the cloud. 

Over the course of many cloud migration customer engagements, we’ve found some patterns in requirements and challenges for the final phase of the migration journey that seem to apply across the industry and in both large and mid-sized enterprises.

Teams design and plan the right architecture (multi-cloud or hybrid cloud), but struggle with how to migrate quickly and deploy faster and continuously

We often see customers build a multi-cloud/hybrid cloud architecture to migrate their on-prem workloads, migrating consumer-facing apps to Kubernetes clusters across various cloud data centers (AWS/Azure/GCP/Openshift) in many regions- US East and West, Canada Central, Ireland, Paris, Milan, Mumbai, Hongkong, Singapore, and Tokyo. These cases often require the DevOps team to write hard to maintain custom scripts to deploy their code to multi-cloud, often leading to deployment burnout, manual errors, and slower application delivery.

This is a scenario where the use of Spinnaker can make migration more seamless, avoiding the need for custom manual scripting which is costly to maintain and prone to human error. 

Spinnaker offers abstracted pipelines and APIs to integrate with all of your source and target cloud systems, meaning you don’t have to worry about building custom deployment scripts for multicloud deployments.  Instead, you develop your business logic, wrap it in a VM or container (configuration), and seamlessly deploy it through Spinnaker pipelines. Spinnaker supports almost all varieties of artifacts like AMI, Jars, etc., and repositories like GitHub, GitLab, AWS S3 DockerHUB, etc.

Below is an example of a pipeline created to deploy into AWS EC2 instance (you can watch the whole webinar here):

Similarly, the intended application can be onboarded to Spinnaker easily. The following video demonstrates how easy it is to perform application onboarding into Spinnaker in just 30 secs.

Once your pipeline is created and applications onboarded, it’s a matter of just a few minutes to hours to deploy your application to the desired cloud platforms.

Service uptime is critical. While migrating, is it possible to upgrade workloads without any downtime?

Yes. The concept of CD is all about delivering small batches of change in a repeatable and sustainable manner and the automation and reversibility of CD make migration projects less risky. Spinnaker, an open-source CD platform, provides built-in strategies like blue/green, rolling updates, and canary that enable hassle-free deployment of apps. In the event of production issues, you can quickly roll back to the previous version in seconds with Spinnaker.

The screenshot below shows some of Spinnaker’s built-in strategies:

How can we verify that our software is risk-free after migration?

Once you’ve moved your apps from on-prem to cloud, a Continuous Verification (CV) strategy should be employed to continuously analyze the risk of releases at each stage and identify issues before they are deployed to production.

An example of CV is OpsMx Autopilot which analyses the risk of new deployable releases in the Build, Test, Deploy, and Production stages. Autopilot ingests logs from CI build tools such as Jenkins or Maven, aggregates and correlates critical errors, and uses ML-based algorithms to provide a continuous holistic view of the risk associated with each build. After a build is successful, it is progressed to the testing phase where Autopilot analyzes test logs from tools like test harness or Selenium. Autopilot provides a risk score along with the root cause of failed test cases for quick diagnosis.

Similarly, in the deployment phase, Autopilot uses the metrics and logs from your newly migrated apps in different canary phases and compares them with those of previously deployed apps.  Autopilot’s machine learning algorithms accurately detect anomalies and problems in the new deployments. In the case of anomalies, Spinnaker pipelines can be configured to rollback the deployment to the previous version or wait until an issue is resolved. 

This screenshot shows the risk score of a release in deployment stages, and a baseline comparison between a newly migrated app and an old app.

 

 

Using our pre-built adapters, you set up Autopilot with your existing monitoring/logging tools such as Dynatrace, Appdynamics, or Splunk, and you are then notified in case of any latency or throughput issues.  

After successful migration using Spinnaker, how do we scale the infrastructure to accommodate our evolving needs?

 The principle of immutable infrastructure is built into Spinnaker, and Spinnaker integrates with infrastructure management solutions like Terraform/Ansible/Puppet to commission, decommission, and auto-scale infrastructure on demand (watch the webinar about Integrating Terraform with Spinnaker here). DevOps engineers need to define the topology to run an application by creating a machine image ( AWS AMI, GCP Image, Azure VM IMage) or a container image ( Docker Image). VPC, security group policies, VMs, Load Balancers, configurations can be easily reconciled and provisioned by using Spinnaker’s enterprise integrations. Spinnaker uses Auto Scaling Group under the hood to provision environments on AWS (or equivalent on GCP, Azure, etc.) or uses Deployments, Replicaset to provision environments on Kubernetes (more on deploying apps to K8S). 

Below is an example of an EC2 policy in Spinnaker. It constantly monitors the desired metric of a server with expected sudden load change and scales the server group to keep the metric as close as possible to its target. ( reference: ContinuousDeliveryWithSpinnaker ebook)

All PR is good PR (except when it’s because our security was compromised.)

Without proper security controls, there’s always an unacceptable risk of a security breach. In the report “Six Key Security Considerations for Responsible Cloud Migration”, security firm Symantec examines how controlling access to and ensuring compliance use of cloud applications can be difficult for enterprises with a mix of on-prem and cloud applications, making a unified approach for pushing policies to govern access and usage is critical.

OpsMx Enterprise Spinnaker for (OES) was built to provide enterprise-ready Spinnaker and enables you to easily define policies to control the what, where, and when a deployment can happen and enables the audit of previous deployments to quickly identify the why and who behind failed pipeline executions. Spinnaker also integrates with your existing LDAP/SAML service for authentication and follows RBAC protocols to restrict the access of internal employees to certain accounts, applications, etc..

Conclusion

We’ve seen many enterprises who have successfully completed their rehosting and re-platforming journeys quickly and safely with Spinnaker Continuous Delivery, but more importantly, they empowered their DevOps teams with the ability to automatically deploy applications to the cloud without the pain and cost of custom scripts. In many cases, Spinnaker helped migrate workloads in just a few hours, saving weeks or months of experts’ time compared to manual rehosting. When Continuous Verification such as OpsMx Autopilot is added to the CI/CD pipelines, enterprises can continuously monitor their deployments, trace failed pipelines, and quickly diagnose them, making migrations sustainable by assessing the risk of each release. 

Enterprises embark on a cloud transformation journey to improve operational efficiency, increase innovation, and become hyper-responsive to changing market trends. OpsMx Enterprise for (OES) helps commence and speed that journey by enabling the delivery software more quickly, safely, and securely.

---

OpsMx is a leading provider of Continuous Delivery solutions that help Fortune 500 companies safely deliver software at scale and without human intervention. We help engineering teams take the risk and manual effort out of releasing innovations at the speed of modern business. For additional information, contact us