Use OpsMx Autopilot to Accelerate Your CI/CD Pipeline

This is the second of a three-part series on how to speed up your CI/CD process using OpsMx Autopilot

• Part 1 – “What is slowing down your CI/CD pipeline?” discusses the challenges that a typical continuous delivery (CD) system will have even after implementing a modern CD tool. 
• Part 2 – “Use OpsMx Autopilot to Accelerate Your CI/CD Pipeline” blog, introduces OpsMx Autopilot, the intelligence layer for software delivery. 
• Part 3 –  “How Customers Improve CI/CD Velocity Using Autopilot” outlines the experience of leading companies and provides a quick overview of the benefits that our customers have experienced.

What is Autopilot?

In our part-1 blog we have explained how, in a CI/CD pipeline, the risk of a release is assessed manually. Manual risk assessment and approvals are error-prone, and time-consuming, leading to a slow CI/CD process.

OpsMx Autopilot is a machine-learning-based system that supports any CI/CD platform, provides automated release verification, real-time decisions, governance, compliance, and intelligent visibility.

Autopilot Benefits for CI/CD process

Autopilot improves your CI/CD process through three major benefits: 

1. Reduce risk and shorten approvals – shorten the data gathering drudgery of the verification process and automate the approval decision when possible.
2. Improve governance and compliance – ensure compliance to your corporate and regulatory policies and enable you to audit all the steps in your CI/CD process.
3. Generate Insights and visibility – enable the high-level management of the enterprise-wide software delivery process by sharing CI/CD metrics and DevOps best practices among teams. 

Automated verification and real-time approvals through machine learning

Typically in a delivery workflow, you build your artifacts and go through a series of steps to validate that they are ready for production. The typical stages are  Build, Test, Deploy, and Prod. Between each stage, a decision is made regarding promotion to the next stage. 

During each stage, a tremendous amount of data (build logs, test logs, performance logs & metrics, code analysis and so on) – sometimes called digital exhaust – is generated by the tools you use. 

Autopilot Integrations with 3rd party tools

Autopilot integrates with a long list of products, including CI and build products like Jenkins, CircleCI, Maven, Bamboo; test automation products like JMeter and Selenium; APM, monitoring and log products like Datadog, Appdynamics, Prometheus, Splunk, Sumo Logic, Elastic Search; security products like SonarQube and Vault, and ITSM products like ServiceNow, PagerDuty, and Slack. 

After a simple configuration to integrate these tools, Autopilot automatically works on each execution of each pipeline. It gathers data and feeds it to an AI/ML engine to generate a risk score- ranging from 0 to 100 – essentially depicting the confidence of moving the change to the next stage. 

Autopilot can be configured to automatically promote a change to the next stage in case of a high degree of confidence – removing the delay of human evaluation. Similarly, if the score is low , the system can automatically reject the change and highlight the error to stakeholders. 

In the example below the risk of release is low with 0 critical error, 1 error, and 0 warning, hence the score is 93 and the release will be disapproved automatically.

Log analysis results dashboard

In cases where the decision is unclear, Autopilot organizes the gathered information to assist you in the verification and approval process. 

Further, every time you make a decision, you inherently “teach” Autopilot, so it performs better in the future for similar cases. The image below represents examples of approval stages in one of our customers today. 

Software Delivery Pipeline

Autopilot Risk Analysis Dashboard for CI/CD pipeline

Besides displaying errors, Autopilot also provides charts for easy reference and verification. Team can find out the frequency and criticality of issues – errors, exceptions, and warnings – from Autopilot’s bubble graph (refer to the image below). Autopilot applies its intelligence to aggregate and de-duplicate errors based on their categories, and this provides  the contextual information needed to diagnose an issue and provide feedback to developers on the root cause of the error. 

Autopilot Risk Analysis Dashboard

You can drill down from the high level view to see details on the specific errors and issues that have occurred in the current run.

Aggregated and De-duplicated view of issues

Continuous Governance

Autopilot helps teams adhere to all applicable policies in the software delivery workflow. DevOps, development, and security and compliance teams can use Autopilot to express policy in a declarative language that provide fine-grained controls on the Spinnaker deployment pipeline. 

Autopilot leverages the standard Open Policy Agent (OPA) to automate policies and rules, best practices, and regulations across the CI/CD process. Because the policies are declarative, security managers can quickly add, modify, delete policies in tune with business policy changes.

All you need to do is to plug in Autopilot into your software delivery pipelines and policy compliance is auto-checked before promoting any deployment through the pipeline. 

A few examples of organizational policies that Autopilot can check during the pipeline execution include: 

• Completion of all required integration and performance tests
• Adherence to blackout windows
• Approval of the network configurations in prod or test systems
• Appropriate approval for production deployments
• Security scan completed for all software
• Correct open source versions being deployed
• Deployment of the workload to the correct cluster 
• Specific OS and container verification

Apart from operational controls, adherence to external legal regulatory policy compliance requirements like SOX, HIPAA, GDPR, PCI-DSS can also be enforced.

Compliance with various regulatory standards

Auditing using Autopilot

Autopilot also allows auditors to quickly identify the who, what, when, where, and how of the continuous delivery process using automatically generated audit reports. Autopilot provides the following information and helps internal and external auditors in validating policy compliance.

• List of pipelines created and executed along with complete timing and authorization information. 
• Status of pipelines run in a given time interval
• List of pipelines that were  specific business-critical applications in a time frame
• Any unauthorized deployment attempts
• Lists of deployments into specific clusters during peak business hours violating policies
• And more.

Visibility and Insights

Visibility for Approvals

Autopilot provides a customizable automated verification dashboard, for easier navigation of log and metrics data, confidence scores, and better insights to help your project/product managers make faster approvals.

Learn more about how to automate decision-making in CD pipelines using Autopilot here.

CD Dashboard

Autopilot also provides trend analysis for engineers and managers to get an organization-wide view of deployments. It provides the capability to analyze the health and confidence score of each application over time and identify trends and underlying root causes. Using Autopilot, one can visualize the issues, or navigate log and metrics data generated in each application deployment. Visibility about logs before and after an issue helps you understand the context of threshold violation. 

Furthermore, Autopilot provides reports on frequency, time, and status of deployments into on-prem or public cloud in the past week or month or year, to get a better idea about CI/CD project effectiveness. Real-time insights that Autopilot provides include:

• a list of active pipelines used frequently, 
• number of times a channel has succeeded/failed,
• fast and slow-performing pipelines, and 
• the overall time taken for manual judgments.

Autopilot CD Dashboard 

Conclusion

By adding the Autopilot intelligence layer to your software delivery process, DevOps and SRE teams managing the CI/CD pipelines can quickly make more informed decisions while analyzing the risk of releases and automatically approving them for deployment. The three main Autopilot capabilities include: 

• Automated Analysis and Decision Making – Reduce risk and shorten approvals 
• Continuous Governance – Improve governance and compliance to your corporate and regulatory policies and enable easy audit of your CD process.
• Visibility and Insights – Generate Insights and visibility  that enables the high level management of the enterprise-wide CD process while sharing best practices among teams.

For more information on how Autopilot helped solve problems for specific customers, please read, “How Customers Improve CI/CD Velocity Using Autopilot”.

OpsMx is a leading provider of Continuous Delivery platform that helps enterprises safely deliver software at scale and without any human intervention. We help engineering teams take the risk and manual effort out of releasing innovations at the speed of modern business. For additional information, contact us.