Managing Secrets in Spinnaker – Encryption using Symmetric Key

Spinnaker provides a way to encrypt secrets used by some of its services like clouddriver and igor (till this blog is published, only clouddriver, igor and echo services support this feature) through the Spring Config Server implementation. This document guides through the process of configuring encrypted secrets in Git repository and using them in two of the Spinnaker services – clouddriver and igor.

As we know clouddriver service deals with the cloud providers (Kubernetes, AWS, Google, Cloudfoundry etc) and igor service deals with the CI systems(Jenkins, Travis, Concourse etc), our goal is to encrypt the secrets and make them available to these services. 

Configuration

We configure encryption key and Git web URL in spinnakerconfig.yml, encrypted secrets in <service>-local.yml (checked-in Git) and keys in hal config.

Encrypt Key

This is a symmetric key used for encrypting and decrypting the secrets. Making it longer and complex makes the secret more secure. Here is a random key we use for our configuration: “Q7udUkHPuA3VnNlOtksSgQ”.

Configuring spinnakerconfig.yml

Web URL of remote Git repository where encrypted secrets are stored is configured along with encryption key in spinnakerconfig.yml which is placed under ~/.hal/default/profiles/. Here is how spinnakerconfig.yml looks like after configuring Git web URL and encryption key:

spring:
  profiles:
    include: git
  cloud:
    config:
      server:
        git:
          uri: https://github.com/example/spinnaker-config
          basedir: /tmp/config-repo
          refresh-rate: 10
encrypt:
  key: Q7udUkHPuA3VnNlOtksSgQ

Git Repository

Cloud provider secrets are configured in clouddriver-local.yml and CI system secrets are configured in igor-local.yml. These two files are checked in to Git. 

The following clouddriver-local.yml configuration has encrypted secrets of only AWS but we can add that of other providers as well.

clouddriver-local.yml:

aws:
  access-key: '{cipher}0314b4e9158fb0f9a2b2658a9a6d5f3de19eee7ccf7c3a2597a03a0f04fdfb910bbfe6f58bb30e33ecd2a96043f0d9f6'
  secret-key: '{cipher}1af38ea0f92204d23524fa656e82d19321921c67b87ac101e9be41f3197a6c2361e54a151613ace26caac22b2179227f8f573f5eeb31523f2ee0fea149c18c86'

Note that a prefix {cipher} is added to the encrypted secrets so that Spinnaker knows they are encrypted.

Process of generating encrypted keys is explained in the next sections.

The following igor-local.yml configuration has encrypted Jenkins username and password which are referred in the hal config and subsequently in igor.yml.

igor-local.yml:

jenkins:
  username: '{cipher}b40c99ac62a422c6002baeadaec16d2003f2abe2edaf353a63e6de649050cc06'
  password: '{cipher}9bb1857fc248b71fc921b87eedb01d0db22bdc3c1b5223698f8646853277a907'

Hal Config

Cloud provider part of the hal config looks like this for AWS:

  providers:
    aws:
      enabled: true
      accounts:
      - name: my-aws-account
        requiredGroupMembership: []
        providerVersion: V1
        permissions: {}
        accountId: '123456789101'
        regions:
        - name: us-west-2
        assumeRole: role/spinnakerManaged
        lifecycleHooks: []
      bakeryDefaults:
        baseImages: []
        awsAccessKey: ${aws.access-key} 
        awsSecretKey: ${aws.secret-key}
      accessKeyId: ${aws.access-key}
      secretAccessKey: ${aws.secret-key}
      defaultKeyPairTemplate: '{{name}}-keypair'
      defaultRegions:
      - name: us-west-2
      defaults:
        iamRole: BaseIAMRole

And CI part looks like the following:

  ci:
    jenkins:
      enabled: true
      masters:
      - name: opsmx-jenkins-master
        permissions: {}
        address: http://opsmx-jenkins.com/jenkins
        username: ${jenkins.username}
        password: ${jenkins.password}

Encrypting and Decrypting Secrets

As mentioned at the beginning, clouddriver and igor services implemented Spring Config Server and hence capable of encrypting and decrypting secrets. 

With clouddriver and igor services running on local machine, and having encryption key(Q7udUkHPuA3VnNlOtksSgQ) set in spinnakerconfig.yml, the following commands generates encrypted text:

curl http://localhost:7002/encrypt -d <my-plain-text-secret>
curl http://localhost:8088/encrypt -d <my-plain-text-secret>

$ curl http://localhost:7002/encrypt -d APIB4VRZ6XSFMVLZKLQ9   
0314b4e9158fb0f9a2b2658a9a6d5f3de19eee7ccf7c3a2597a03a0f04fdfb910bbfe6f58bb30e33ecd2a96043f0d9f6

$ curl http://localhost:7002/encrypt -d XqgRrB8k2Zm1s7MUdcwaSEcLpUPyR8jygmiNl7dA
1af38ea0f92204d23524fa656e82d19321921c67b87ac101e9be41f3197a6c2361e54a151613ace26caac22b2179227f8f573f5eeb31523f2ee0fea149c18c86

$ curl http://localhost:8088/encrypt -d opsmx-user
b40c99ac62a422c6002baeadaec16d2003f2abe2edaf353a63e6de649050cc06

$ curl http://localhost:8088/encrypt -d opsmx-password
9bb1857fc248b71fc921b87eedb01d0db22bdc3c1b5223698f8646853277a907

Similarly the following commands decrypt the encrypted secrets:

curl http://localhost:7002/decrypt -d <encrypted-secret>
curl http://localhost:8088/decrypt -d <encrypted-secret>

$ curl http://localhost:7002/decrypt -d 0314b4e9158fb0f9a2b2658a9a6d5f3de19eee7ccf7c3a2597a03a0f04fdfb910bbfe6f58bb30e33ecd2a96043f0d9f6
APIB4VRZ6XSFMVLZKLQ9

$ curl http://localhost:8088/decrypt -d 9bb1857fc248b71fc921b87eedb01d0db22bdc3c1b5223698f8646853277a907
opsmx-password

Sequence of Steps

1. Create a new git repository with clouddriver-local.yml and/or igor-local.yml without any encrypted secrets.
2. Configure ~/.hal/default/profiles/spinnakerconfig.yml with the git repository and encryption key.
3.  Run ‘hal deploy apply’ without any keys in the hal config.
4. Now clouddriver and igor services can generate encrypted secrets. Generate and configure the encrypted secrets in <service>-local.yml and replace plain-text secrets from hal config with the corresponding property names.
5. Run ‘hal deploy apply’.