OpsMx Enterprise for Spinnaker Installation on Azure VM using K3s

OpsMx provides enterprise Spinnaker that comes with a variety of extensions on top, enhancing easier Spinnaker adoption for enterprise customers to deliver their applications at a higher velocity, confidence, and trust.  Here in this blog, we show you how to install OpsMx Enterprise for Spinnaker(OES) on the Azure VM platform using K3s. Let us first briefly understand what is OES, Azure VM and K3s.

What is OpsMx Enterprise for Spinnaker?

OpsMx Enterprise for Spinnaker (OES) is the leading enterprise-ready distribution of the Spinnaker Continuous Delivery platform. OES offers the scalability and reliability of open source Spinnaker for workflow automation. OpsMx scans and hardens the Spinnaker and adds a series of enterprise extensions to speed adoption, increase target application environments, and manage the Spinnaker lifecycle. OpsMx Autopilot adds a layer of intelligence to Spinnaker for automated validation, governance and policy compliance, and intelligent visibility across the Spinnaker Continuous Delivery (CD) pipeline. OES empowers enterprises to transform their application delivery by releasing faster with more confidence and less risk.

What is Azure VM?

Microsoft Azure VM (Virtual Machines) is an on-demand, scalable cloud computing resource that gives you the flexibility of virtualization in the cloud where you can deploy and run your software applications.

What is K3s? 

K3s is an official Cloud Native Computing Foundation sandbox project that brings a lightweight, fully compliant Kubernetes cluster distribution. K3s is a single binary of less than 40MB that completely implements the Kubernetes API. Due to its low resource requirements, it’s possible to run a cluster on anything from 512MB of RAM machines upwards. K3s is designed for lower resource production models like remote locations or on an IoT device. 

K3s has the following features:

1. It is packaged as a single binary.
2. It adds support for the SQLite3 database as the default storage backend. ETCD3, MySQL, and PostgreSQL are also supported.
3. It wraps Kubernetes and other components in a single, simple launcher.
4. It is secure by default with reasonable defaults for lightweight environments.
5. It has minimal to no OS dependencies (just a sane kernel and cgroup mounts needed).
6. It eliminates the need to expose a port on Kubernetes worker nodes for the kubelet API by exposing this API to the Kubernetes control plane nodes over a WebSocket tunnel.

To get enterprise Spinnaker running in K3s, we will install K3s on Azure VM first, and then we will use K3s for installing OES on Azure. 

Prerequisites

• An Azure virtual machine with at least 6 cores and 20 GB memory.
• Helm 3 installed in a virtual machine.

Find more information about installing Helm 3  at https://helm.sh/docs/intro/install/

Install K3s on an Azure virtual machine(VM)

1. Execute the below command as an Azure user to install k3s.

curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--write-kubeconfig=/home/azureuser/.kube/config --write-kubeconfig-mode=644" sh -

By default, the Kubernetes configuration file will be written to the root user. To write to the Azure user use –write-kubeconfig option. Find all the available installation options here.

Install OES (OpsMx Enterprise for Spinnaker) on  K3s

1. Create a namespace and make it as default.

   kubectl create namespace oes
kubectl config set-context --current --namespace oes

2. Add OpsMx Helm repo to your instance.

   helm repo add opsmx https://helmcharts.opsmx.com/

3. Install Helm chart to the namespace created above using the helm repo as added above.

helm install <RELEASE_NAME> opsmx/oes --set imageCredentials.username=<quay-username> --set imageCredentials.password=<quay-password> --set k8sServiceType=NodePort --timeout 15m

Available installation modes are:

• OES-AP (both OES and Autopilot, Default mode) 
• OES (Only OES)
• AP (Only Autopilot) 
• None (Skip OES installation) 

By default, OES-AP gets installed if we are not specifying the installation mode. For more information please visit: https://github.com/OpsMx/enterprise-spinnaker/tree/master/charts/oes

Note: In the above command RELEASE_NAME is the name that you want to give to the particular installation. If you do not wish to specify a release name, you can bypass the –generate-name command while running the helm install command. Find more details here.

Edit configmaps of oes-ui, oes-gate, and oes-sapor microservices

OES installation will work off-the-shelf without any manual intervention on cloud-managed Kubernetes clusters where the load balancers will be assigned to oes-ui and oes-gate services. In this case, the private IP of the instance will be assigned to the oes-ui and oes-gate services. So to get the oes-ui access through the browser it is required to edit the configmap of oes-ui and oes-gate as follows. 

1. Edit the oes-ui config map

   kubectl edit cm oes-ui-config

        Replace OES_GATE_IP with the public IP address of the Azure virtual machine.

   app-config.json: |
{
"endPointUrl":"http://OES_GATE_IP:8084/",
"setApplicationInterval": 300000
}

2. Edit the oes-gate-config map

   kubectl edit cm oes-gate-config
Replace OES_UI_LOADBALANCER_IP with the public IP address of the Azure VM (virtual machine).
cors:
allowed-origins-pattern: ^https?://(?:localhost|OES_UI_LOADBALANCER_IP|opsmx.com)(?::[1-9]\d*)?/?
ldap:

3. Edit the oes-sapor-config map

   kubectl edit cm oes-sapor-config
Replace spin-gate with the public ip address of azure virtual machine.
spinnaker:
authnEnabled: true
baseurl: http://spin-gate:8084
externalBaseUrl: http://spin-gate:8084
ldap:

4. Delete the oes-ui, oes-gate and oes-sapor pods to reflect the changes made

   kubectl delete pod <oes-ui-pod-name>
kubectl delete pod <oes-gate-pod-name>
kubectl delete pod <oes-sapor-pod-name>


5. Edit the config file of the Halyard pod
   1. Get halyard pod name by executing the below command
      kubectl get pods
   2. Edit the Halyard pod config file

      kubectl exec -it <halyard-pod-name&gt; bash
vi ~/.hal/config

   3. Update the ‘overridebase’ URL as below

      security:

                       apiSecurity:

                          ssl:

                            enabled: false

                        overrideBaseUrl: http://<public-ip-of-vm>:31183

                    uiSecurity:

                       ssl:

                          enabled: false

                      overrideBaseUrl: http://<public-ip-of-vm>:30959

      Note: Port numbers will automatically populate over there. Don’t change those.

   4. Save the changes and execute the below commands

     hal deploy apply

  exit

Access OES and Spinnaker through the browser     

• Once all pods are up & running use the below URLs to access OES and Spinnaker

      OES         —- http://<public-ip-of-azure-vm>:<OES-ui-Nodeport>

      Spinnaker  —- http://<public-ip-of-azure-vm>:<Spin-deck-Nodeport>

           To get OES-UI-Nodeport and Spin-deck-Nodeport use the below commands

                 kubectl get svc oes-ui -o jsonpath="{.spec.ports[].nodePort}"

          kubectl get svc spin-deck-np -o jsonpath="{.spects[].nodePor.port}”

Note : Allow these two ports in the Azure Virtual Machine network security group.

            Credentials for accessing OES and Spinnaker are the same as follows:

Username: admin 

            Password: opsmxadmin123

Add worker nodes to k3s cluster (If required)

• Execute the below commands in the worker node:

export K3S_URL=https://<master-private-ip>:6443

export K3S_TOKEN=<paste-the-token-stored-in-master-node>

curl -sfL https://get.k3s.io | sh -

Token will be stored in: /var/lib/rancher/k3s/server

Now that we have shown you the installation process for OES in Azure VMs using K3s you can easily do it yourself. Do let us know in the comments section below about your experience in doing so or if any problems arose.

If you are using Microsoft Azure, there are some articles you may find interesting to streamline your continuous delivery:

1. Continuously deploy to Azure cloud using Spinnaker
2. Configure Spinnaker Authentication using Azure Active Directory

If you want to know more about the OpsMx Enterprise for Spinnaker or request a demonstration, please book a meeting with us.

OpsMx is a leading provider of Continuous Delivery solutions that help enterprises safely deliver software at scale and without any human intervention. We help engineering teams take the risk and manual effort out of releasing innovations at the speed of modern business. For additional information, contact us