What is a CI/CD Pipeline ? Continuous Integration and Continuous Delivery Explained.

CI/CD pipeline is defined as a series of steps that include all the stages from the outset of the CI/CD process and is responsible for creating automated and seamless software delivery. With a CI CD pipeline, a software release artifact can move and progress through the pipeline right from the code check-in stage through the test,  build, deploy, and production stages. This concept is powerful because once a pipeline has been specified, parts or all of it can be automated, speeding the process and reducing errors. In other words, the goal of a pipeline workflow is to make it easier for enterprises to deliver software multiple times a day automatically.

DevOps engineers often confuse the CI/CD pipeline with automation of individual stages in CI/CD. Though different CI/CD tools may automate each complicated stage in the pipeline,  the whole software supply chain of CI/CD may still be broken because of manual intervention in between. But let us first understand various stages in a CI/CD process and why a CI/CD pipeline is essential for your organization to deliver code at speed and scale.

CI/CD Pipeline Stages: Understanding People, Process, and Technology

A CICD pipeline consists of a series of sub-processes that can be classified into two major buckets – Continuous Integration and Continuous Delivery. For a CI CD pipeline to work, we require a series of sub-processes or stages that need to continuously check checks and verify the code updates. These sub-stages are as follows.

Continuous Integration: Code Commit, Static Code Analysis, Build and Test stages/scenarios

Continuous Delivery: Bake, Deploy, Verification, monitoring, and Feedback

Enterprise application development teams typically consist of developers, testers/QA engineers, Operation engineers, and SRE (Site Reliability Engineers) or IT Operations teams. They work together closely to deliver quality software into customers’ hands. CI/CD is a combination of two separate processes: Continuous Integration and Continuous Deployment. The major steps in each are listed below.

A CICD Pipeline

A CICD Pipeline

What is Continuous Integration?

Continuous integration (CI) is the process where software is built and initial tests are completed. Continuous deployment (CD) is the process of combining code with infrastructure, ensuring that all tests are completed and policies followed, and then deploying code into the intended environment. Of course, many companies have their process, but the major steps can be classified as below. 

CI: Code-Commit 

Continuous Integration – Code Commit

• People:
  Developers and Engineers, Database Administrator (DBA), Infrastructure team
• Technology:
  GitHub, Gitlab, SVM, BitBucket
• Process: 

A code commit stage is otherwise known as version control. A commit is an operation that sends the latest changes written by a developer to the repository. Every version of the code written by a developer is stored indefinitely. After a discussion and review of the changes with collaborators, developers will write the code and commit once the software requirements, feature enhancements, bug fixes, or change requests are completed. The repository where the edits & commit changes are managed is called Source Code Management (SCM tool).. After the developer commits the code (code Push Request), the code changes are merged into the base code branch stored at the central repository like GitHub. 

CI: Static code analysis

• People: Developers and Engineers, Database Administrator (DBA), Infrastructure team, testers
• Technology: GitHub, Gitlab, SVM, BitBucket
• Process:

Once the developer has written a code and pushes it to the repository, the system gets triggered automatically to start the next process of code analysis. Imagine a step where the code committed gets to build directly and it failed during build or deployment. This becomes a slow and costly process in terms of resource utilization, both machine, and man. The code must be checked for static policies. SAST (Static Application Security Test): SAST is a white-box testing method to examine the code from inside using SAST tools like SonarQube, Veracode, Appscan, etc., to find software flaws, vulnerabilities, and weaknesses. (such as SQL injection etc.). This is a fast check process where the code is checked for any syntactic errors. Though this stage lacks the feature to check for runtime errors, which is performed at a later stage.

Placing an additional policy checking into an automated pipeline can dramatically reduce the number of errors found later in the process. OpsMx Enterprise for Spinnaker, which includes OpsMx Autopilot can ensure that the process is followed correctly and that policies are automatically verified. 

CI: Build

Continuous Integration – Code Build and Storage

• People: Developers and Engineers
• Technology: Jenkins, Bamboo CI, Circle CI, Travis CI, Maven, Azure DevOps
• Process:

The Continuous Integration process’s goal is to take the regular code commits and continuously build binary artifacts. The continuous integration process helps to find bugs more quickly by checking if the new module that is added plays well with the existing modules. This helps reduce the time to verify a new code change.  The build tools help in compiling and creating executable files or packages (.exe,.dll, .jar, etc.) depending on the programming language used to write the source code. During the build, the SQL scripts are also generated and then tested along with infrastructure configuration files. In a nutshell, the build stage is where your applications are compiled. Other sub-activities that are a part of the Build process are Artifactory Storage, Build Verification, Unit tests,

Build Verification Test (BVT)/ Smoke Tests and Unit Tests: 

Smoke testing is performed immediately after the build is created. BVT checks if all the modules are integrated properly and if the program’s critical functionalities are working fine. The purpose is to reject a badly broken application so that the QA team does not waste time installing and testing the software application.

Post these checks a UT(unit test) is added to the pipeline to further reduce failures at the production. Unit Testing tests individual units or components of a code written by the developer to validate if they perform as expected.

Artifactory Storage:

Once a build is prepared, the packages are stored in a centralized location or database called Artifactory or Repository tool. There can be many builds getting generated per day, and keeping track of all builds can be difficult. Hence, as soon as the build is generated and verified, it is sent to the repository for storage. Repository tools such as Jfrog Artifactory are used to store binary files such as .rar, .war, .exe. Msi etc. From here, testers can manually pick, deploy an artifact in a test environment to test. 

CI: Test Stages

Continuous Integration – Testing

• People: Testers, QA Engineers
• Technologies: Selenium, Appium, Jmeter, SOAP UI, Tarantula 
• Process: 

Post a build process a series of automated tests validate the code veracity. This stage helps errors from reaching the production. Depending on the size of the build this check can last from seconds to hours. For large organizations where codes are committed and built from multiple teams, these checks are run in a parallel environment to save precious time and notify developers of bugs as early as possible.  

These automated tests are set up by testers ( or known as QA engineers) who have set up test cases and scenarios based on user stories. They perform regression analysis, stress tests to check deviations from the expected output. Activities that are involved in testing are Sanity tests, Integration tests, Stress tests. This is a much-advanced level of testing that happens. Here we find issues that were probably unknown to the developer developing the code.

Integration Tests:

Integration tests are performed using tools like Cucumber, Selenium, etc., where individual application modules are combined and tested as a group while evaluating the compliance with specified functional requirements.  After an integration test, someone needs to approve that the set of updates in that group should be moved to the next stage, which is typically performance testing. This verification process can be cumbersome, but it is an important part of the overall process. There are some emerging solutions to the verification process, such as OpsMx Autopilot.

Load and Stress Testing: 

Load balancing and stress testing are also performed using automated testing tools like Selenium, JMeter, etc., to check if an application is stable and performing well when exposed to high traffic. This test is typically not run on every single update, as the full stress testing is long-running. When major new capabilities are to be released, multiple updates are grouped and full performance testing is completed. In cases where a single update is being moved to the next stage, the pipeline may include canary testing as an alternative.

What is Continuous Deployment? 

Continuous Deployment is the final stage of a CI/CD pipeline where the code that was committed by the developers is deployed into the production post standard testing. It simplifies the process of update by eliminating manual scripting and enabling real-time monitoring.  As we have assumed Spinnaker as the CD tool we will have to introduce a new Bake concept into the Deployment process.

CD: Bake and Deploy

Continuous Deployment- Bake and Deploy

• People: Infrastructure Engineer, Site Reliability Engineer (SRE), Operation Engineer
• Technology: Spinnaker, Argo CD, Tekton CD 
• Process:

After the testing stage is completed the codes that have cleared the criterion are ready to be deployed into the servers where they were meant to integrate with the main application. Before getting deployed into the production they will be deployed to test/ staging or a beta environment that is internally used by the product team. Before the builds are moved to these environments the builds have to pass through two substages named Bake and Deploy. Both of these stages are native to Spinnaker.

CD : Bake

“Baking” refers to creating an immutable image instance from the source code with the current configuration at the production. These configurations may be things like database changes and other infrastructure updates. Spinnaker can either trigger Jenkins to perform this task and some organizations prefer to use Packer.

CD : Deploy

Spinnaker will automatically pass the baked image to the deploy stage. This is where the server group is set to be deployed to a cluster. A functionally identical process is carried out during the Deployment stage similar to the testing processes described above. Deployments are first moved to test, stage, and then finally to prod environments post approvals and checks. This whole process is handled by tools like Spinnaker.

CD : Verification

 This is also a key place for teams to optimize the overall CI/CD process. Because so much testing has happened now, failures should be rare. However, any failure at this point needs to be resolved as quickly as possible so the impact on the end customer is minimized. Teams should consider automating this portion of the process as well, and OpsMx Autopilot can help.

Deploying to prod is carried out using deployment strategies like Blue-Green, Canary Analysis, Rolling Update, etc. During the deployment stage, the running application is monitored to validate whether the current deployment is right or it needs to be rolled back. 

CD : Monitoring

• People: SRE, Ops Team
• Technology: Zabbix, Nagios, Prometheus, Elastic Search, Splunk, Appdynamics, Tivoli
• Process: 

To make a software release failsafe and robust, tracking the release’s health in a production environment is essential. Application monitoring tools will trace the performance metrics like CPU utilization and latency of releases. Log analyzers will scan torrents of logs produced by underlying middleware and OS to identify behavior and track the source of problems. In case of any issue in the production, stakeholders are notified to ensure the production environment’s safety and reliability. Moreover, the monitoring stage helps businesses gather intelligence about how their new software changes contribute to revenue and helps the infrastructure team track the system behavior trend and do capacity planning.

Continuous Deployment: Feedback and Collaboration Tool

Continuous Deployment- Feedback and Collaboration

• People: SRE, Ops, and Maintenance Team
• Technology: JIRA, ServiceNow, Slack, Email, Hipchat
• Process: 

The goal of the DevOps team is to release faster and continuously and then continually reduce errors and performance issues. This is done through frequent feedback to developers, project managers about the new version’s quality and performance through slack or email and by raising tickets promptly in ITSM tools. Usually, feedback systems are a part of the entire software delivery process; so any change in the delivery is frequently logged into the system so that the delivery team can act on it.

Example of a CI/CD Pipeline on OpsMx Enterprise for Spinnaker

CI CD  Pipelines on OpsMx Enterprise for Spinnaker are enabled with IaaS ( Infrastructure as a code) that make it possible to quickly build and deploy. These pipelines are in a declarative format making them flexible and repeatable. Take an example of the below pipeline template on OES (OpsMx Enterprise for Spinnaker) that can be leveraged to maintain configurations and context across multiple microservices or applications.

Template Pipeline on OpsMx Enterprise for Spinnaker

CI/CD Pipeline product screenshot

Broad Features available for your CICD pipeline on OES

• MultiCloud Continuous Deployment: Reduce failure rates by automating deployment with the help of repeatable multi-cloud deployments without scripting across any environment.
• Pipeline as a code: Move pipelines and workflows across multiple microservices and applications with ease.
• Continuous Compliance and Audit: Ensure compliance to industry standards and organizational policies while shipping your releases faster to production.
• Continuous Security: Frequently deploy software into production without compromising on security across all stages of deployment Continuous Verification: Minimize errors during the build, deploy, test, and release process of a new software release through AI/ML technology that leverages data and logs from your existing monitoring platforms.
• Visibility and Traceability: A fully customizable holistic view of all the pipelines to trace vulnerabilities in any artifact.

Summary

An enterprise must evaluate a holistic continuous delivery solution that can automate or facilitate automation of these stages, described above. If you are in the process of implementing a CI CD pipeline or automating your CICD pipeline workflow, OpsMx Enterprise for Spinnaker can help. 

OpsMx Enterprise Cloud leverages cloud architecture to get you started by implementing a CICD workflow on a cloud server of your choice. The feature pipeline as a code simplifies shifting any on-premise CICD pipeline over to the cloud architecture in a matter of minutes.