How to set up Mutual TLS (mTLS) Authentication for Spinnaker Services

Spinnaker services communicate and exchange sensitive data with each other. When TLS (Transport Level Security) is enabled between the services it ensures that all of this data is encrypted.  Communication between services happens only when they have valid certificates.

Mutual authentication or two-way authentication refers to two parties authenticating each other at the same time. Enabling Mutual TLS (mTLS) provides an additional layer of security for theSpinnaker services as only validated clients can interact with the services.

When a client connects to a server:

1. The server responds with its certificate signed by a valid CA (certificate authorities) and the client validates it. 
2. The server sends requests for a certificate from the client and validates the same after receiving it. 

How to create certificates for mutual tls using cert-manager?

To enable mutual TLS, you need to get a certificate (a type of file) from a Certificate Authority (CA). The cert-manager is a native Kubernetes certificate management controller. It can help with issuing certificates from a variety of CA (certificate authorities) sources, such as Let’s Encrypt, HashiCorp Vault, Venafi, a simple signing key pair, or self-signed. It will ensure certificates are valid and up to date, and attempt to renew certificates at a configured time before expiry. Here’s how you can create certificates using the cert-manager:

Pre-requisites: 

Kubernetes, cert-manager

Steps:

1. Create a cluster issuer to issue self-signed certificates using the below YAML code with kubectl create -f

    

   apiVersion: cert-manager.io/v1alpha2

   kind: ClusterIssuer

   metadata: name: selfsigned-issuer

   spec:selfSigned: {}

   kubectl create -f clusterissuer.yml

   kubectl get clusterissuer

2. Create a certificate authority(CA) certificate that can use the above self-signed issuer

   Change the namespace below to the namespace where spinnaker is installed.

   Also, include any other Subject Alternate Names in the dnsNames field.

   apiVersion: cert-manager.io/v1alpha2

   kind: Certificate

   metadata:

    name: mtlsca

    namespace: spintest

   spec:

    secretName: cacert

    isCA: true

    issuerRef:

      name: selfsigned-issuer

      kind: ClusterIssuer

    commonName: mtlsca

    dnsNames:

     – “*.spintest.svc”

     – localhost

   kubectl create -f cacert.yml

   kubectl -n spintest get certs

   kubectl -n spintest get secret

3. Create a certificate authority issuer that can use the above ca certificate.

   Change the namespace below to the namespace where spinnaker is installed.

   apiVersion: cert-manager.io/v1alpha2

   kind: Issuer

   metadata:

    name: caissuer

    namespace: spintest

   spec:

    ca:

      secretName: cacert

   kubectl -n spintest create -f caissuer.yml

   kubectl -n spintest get issuer

4. Create a certificate using the caissuer.

   Change the namespace below to the namespace where the spinnaker is installed.

   Also, change the dnsNames.

   This expects a pkcs12 passphrase in a secret called passphrasesecret.

   kubectl -n spintest create secret generic passphrasesecret –from-literal=passphrase=mysecrepassphrase

   This secret will be used later in configuring the spinnaker files.

   apiVersion: cert-manager.io/v1alpha2

   kind: Certificate

   metadata:

     name: mtlscerts-pkcs12

     namespace: spintest

   spec:

     secretName: mtlscerts-pkcs12

     duration: 2160h # 90d

     renewBefore: 360h # 15d

     commonName: spintest.svc

     keystores:

      pkcs12:

        create: true

        passwordSecretRef: 

          name: passphrasesecret

          key: passphrase

     dnsNames:

     – “*.spintest.svc”

     – localhost

     usages:

     – digital signature

     – key encipherment

     – server auth

     – client auth

     issuerRef:

       name: caissuer

       kind: Issuer

   kubectl create -f mtlscerts.yml

   kubectl -n spintest get certs

   kubectl  -n spintest get secret mtlscerts-pkcs12 -o yaml should show ca.crt, tls.crt,tls.key and keystore.p12

   kubectl -n spintest get secret mtlscerts-pkcs12 -o jsonpath='{.data.ca\.crt}’ | base64 -d >ca.crt

5.  From clouddriver pod get the cacerts file:

   kubectl -n spintest cp clouddriverpod:/etc/ssl/certs/java/cacerts cacerts

   keytool -import -file ca.crt -keystore cacerts   

   kubectl -n spintest create secret generic cacerts –from-file=cacerts

6.  Make the following changes in spinnaker by exec into halyard pod: 
   1. In /home/spinnaker/.hal/default/service-settings , change svc.yaml ( example echo.yml, clouddriver.yml) to mount secret on to svc and overridebaseurl from http to https:

      kubernetes:

        volumes:

        – id: cacerts

          mountPath: /etc/ssl/certs/java

          type: secret

          readOnly: true

        – id: mtlscerts-pkcs12

          mountPath: /pkcs12

          type: secret

          readOnly: true

      overrideBaseUrl: https://spin-clouddriver.spintest.svc:7002

      # Change the service name, namespace and port accordingly

   2. In /home/spinnaker/.hal/default/profiles, change svc-local.yml   ( example echo-local.yml, clouddriver-local.yml) to add https to server and okHttpClient:

   server:

     port: 7002

     ssl:

       enabled: true

       keyStore: /pkcs12/keystore.p12

       keyStoreType: PKCS12

       keyStorePassword: changeit  # from the passphrase secret

       trustStore: /etc/ssl/certs/java/cacerts

       trustStoreType: JKS

       trustStorePassword: changeit # from the passphrase secret

       clientAuth: need

   okHttpClient:

     keyStore: /pkcs12/keystore.p12

     keyStorePassword: changeit # from he passphrase secret

     trustStore: /etc/ssl/certs/java/cacerts

     propagateSpinnakerHeaders: true

     connectTimeoutMs: 60000

     readTimeoutMs: 60000

7.  Hal deploy apply after you are done.

Conclusion: 

After applying the above configuration changes to your Spinnaker deployment, the  Mutual TLS (mTLS) Authentication for Spinnaker Services is enabled thereby making it secure to communicate securely over the network with other services.