How To Deploy Spinnaker In Production ?

Before we begin about Production-ready Spinnaker. Let us see what exactly is Spinnaker ? and why there is so much buzz about the Product Spinnaker?

What is Spinnaker?

Spinnaker is an open-source, multi-cloud continuous delivery platform for releasing software changes with high velocity and confidence. Created at Netflix, it has been battle-tested in production by hundreds of teams over millions of deployments. It combines a powerful and flexible pipeline management system with integrations to the major cloud providers. Deploy across multiple cloud providers including AWS EC2, Kubernetes, Google Compute Engine, Google Kubernetes Engine, Google App Engine, Microsoft Azure, Openstack, Cloud Foundry, and Oracle Cloud Infrastructure, with DC/OS coming soon. Create deployment pipelines that run integration and system tests, spin up and down server groups, and monitor your rollouts. Trigger pipelines via git events, Jenkins, Travis CI, Docker, CRON, or other Spinnaker pipelines. Create and deploy immutable images for faster rollouts, easier rollbacks, and the elimination of hard to debug configuration drift issues. Leverage an immutable infrastructure in the cloud with built-in deployment strategies such as red/black and canary deployments. Join a community that includes Netflix, Google, Microsoft, Veritas, Target, Kenzan, Schibsted, and many others, actively working to maintain and improve Spinnaker.

Why Spinnaker?

It takes many tools to deliver an artifact into production. Lots of Tools for building and testing, tools for creating a deployable artifact like a container image, tools for authentication and authorization, tools for maintaining infrastructure, and many more. For all these, we need to depend on many many tools and Seamlessly integrating these tools into a workflow can be challenging too. As organizations mature, both the number of tools and the number of people managing them tend to grow, often leading to confusing complexity and fragmentation. The continuous delivery (CD) process may work at a smaller scale, But it becomes increasingly challenging to maintain and understand. It can take a long time for new engineers to discover and sort through all the tools needed to deploy even the simplest of changes. That is why Spinnaker comes in to place as a Savior, It is a generalized and extensible tool that provides users with the building blocks to create tailor-made continuous delivery pipelines. There is no need to spend time and take on increased risk inventing your own approach. It is already Battle-tested by Enterprises like Netflix and Google for handling the delivery of thousands of applications in Production.

Now you know why and what is Spinnaker. Let’s see what are the best Security measures for deploying Spinnaker in Production.

1) Ensure Security by Limiting the Container Operations: Run Non-Root Containers.

To deploy containerized applications, you must limit the number of allowed operations to the minimum required. To ensure this, launch containers with a random user different from the root. These are known as non-root containers. For Customers who are in need of this security OpsMx provide Redhat based images which are certified by Redhat and the containers are run as a non-root user. for information – https://medium.com/@dommety/spinnaker-operator-for-kubernetes-and-openshift-fa4640c4a17c

2) Limit Access to the Cluster: Implement Role-based access control (RBAC) Policies

Role-based access control in Spinnaker is a powerful that we have at our disposal to make sure applications and users are able to access what they need and what we grant them. Spinnaker is no different from any other users on your cluster, so having the ability to declare what it gets access to is a major win! in Spinnaker. While Implementing Spinnaker in Kubernetes, That is where Role-based access control policies come into play. For example, if you deploy an infrastructure application that uses kube-apiserver for self-discovery in the namespace “test”, you may only need to allow “get” and “list” operations for pod objects inside that specific namespace. In the past, users were granting cluster-admin privileges (i.e. privileges to perform all operations within the cluster) to applications like the Helm client Tiller. This practice leads to catastrophe in production. Don’t forget to make sure that the applications you deploy using charts have the smallest possible set of RBAC privileges. Spinnaker Does Supports :-

• OAuth
• SAML
• LDAP
• X.509 (OpenSSL)
• External Authentication also Supported. (GitHub, Google Plus Suit, etc..,)

More Info : https://www.spinnaker.io/setup/security/authorization/

3) Auditing

If you are planning to Implement Spinnaker in Production, Have auditing daemon enabled in the pods. Mind you Enabling Auditing may increase the memory consumption of the API server, So Plan the Capacity of your Kube-API Server. You can use [EFK] tools for auditing the Spinnaker pods. Use SSH keys for accessing the instances instead of username/password in the Local-Debian installation. Enable Firewall and Disable unwanted ports. Restrict Local access and enable content trust. Isolate the Execution environment:- Like Tiering the Cots Applications.

4) Logging

Logging is very crucial in Enterprise Environment, where customers deals with lots of Compliance like (SoX, GxP, PCI, HIPPA). If you are planning to have Spinnaker in a fully Enterprise environment please enable Logging for all the pods and redirect the Pod Logs to one centralized server. Just in-case if you lose the pod, the logs are already re-directed to a centralized location, With this, you can get to the bottom of the problem, what exactly the issue with the affected pod and you can remediate it. and Spinnaker supports tools like Use Logging tools like :-

• Splunk
• Syslog
• Sumologic
• Datadog
• EFK
• etc….

5) Pipeline Best Practices

Always trigger pipeline using Git events or using API Triggers. Try to avoid Manual triggers unless you are in of manual triggering. This helps with auditing and it’s easy to trace the job as well. For any sensitive information in the Pipeline / Configuration’s, Try to use Vault like enterprise solution or you can try Base64 Encryption for the same. Use Kayenta and Chaos Monkey for your production rollout. instead of the manual rollout.

6) Accountability

After all the above implementation, the most critical and challenging is accountability. Who Owns which piece of the tool. Try to avoid unwanted access to the pods/instances for troubleshooting. Use Sudo or login trace-set tools for login into K8s to access pods. Document the process of Upgrading, implementation and have the Solution blueprint handy. So, it’s easy for the support team to manage the application in the production infrastructure. Always try to maintain the N-1 version with the latest security patches. Plan well in advance about your maintenance windows, and regular communication is must needed in the process. Always Remember the KISS rule in the Production deployment of any application using Spinnaker.

Conclusion

By following the tips above, you will cover all the basics for Spinnaker production readiness. But there are many more areas that you should explore, such as stability, performance, network, auto-scaling, and much more. Follow OpsMx blog for more such Enterprise-grade solutions for Spinnaker 🙂