Spinnaker – Configuring Dynamic Accounts in Clouddriver for Kubernetes

Sandesh Jain September 6, 2019

One of the use-cases that advanced users of Spinnaker look for is to create a Kubernetes cluster during the pipeline deployment or create it just before the pipeline deployment as another pipeline and use the new Kubernetes cluster as the target deployment environment.

To achieve this, Spinnaker has introduced a dynamic account loading feature from Spinnaker version 1.15.x. This feature is an outcome of Spring Cloud Config integration into Cloud-driver, to add support for fetching account configuration from external sources like Git, Vault, or an S3 bucket. This feature also refreshes credentials for Kubernetes dynamically, while Clouddriver is still running.

Configure dynamic account in Spinnaker for Kubernetes

This document describes the steps for configuring the dynamic account in Spinnaker Clouddriver under the following environment,

Spinnaker 1.17.4 (However, the same can work on new versions of Spinnaker)
External configuration store as a private GitHub repository (https://github.com/sagayd/spin-dynamicaccounts.git)
Halyard’s deployment profile is the default (i.e ~/.hal/default directory)

How does Spinnaker load external Git-based Clouddriver configuration to be used for Kubernetes?

Spinnaker loads account details using the cloud driver service configuration. By default, the Clouddriver configuration of account details is retrieved from the halconfig file. We can configure Spinnaker ‘Spring Cloud Config server’ to look for Clouddriver configuration from the external Git source.

Once the Spinnaker is started by the Halyard service, the Spring Cloud server will load the Clouddriver configuration file (cloud driver-local.yml) from the Git repo. Hence, one can update the cloud driver-local.yml file on the Git repo, and the accounts will be refreshed automatically. This means you don’t have to run the “hal deploy apply” command every time you add a new account and so Spinnaker service is not disturbed.

Detailed Procedure to setup Dynamic Account in Spinnaker for Kubernetes

Instruct Spinnaker to look for external configuration from Git repo.
- Do hal deploy apply and the clouddriver service gets reloaded. This is done only for the first time when you create the spinnakerconfig.yml file. Note: Add your K8s account from halconfig also to clouddriver-local.yml, because clouddriver config is the source of truth for accounts once it is loaded.nstruct Spinnaker to look for external configuration from Git repo.
  - Create a file ~/.hal/default/profiles/spinnakerconfig.yml with the content below
```
spring:
  profiles:
    include: git
  cloud:
    config:
      server:
        git:
          uri: https://github.com/OpsMx/spin-dynamicaccounts.git
          username: opsmxdemo
          password: xxxxxxxx
          basedir: /tmp/config-repo
          refresh-rate: 10
```
Note: Change the Git repo, its credential, and basedir information as per your requirement.
Use your convenient method (Terraform/CloudFormation/any) to create a K8s cluster in EKS/GKE/AKS/Baremetal environment and store the new kubeconfig file in Git repo – spin-dynamicaccounts.git
Commit the new Cluster’s Kubeconfig file my-k8s-cluster.config under the Repo path: spin-dynamicaccounts.git/k8sconfigs/

Update clouddriver-local.yml with new account information. Account *name* should be unique and the *kubeconfigFile* path should match the relative path of the Git repo. In the code snippet below, the lines after ‘accounts:‘ line and from the ‘name:‘ line is the block to be cloned every time you add a new account.

kubernetes:
      enabled: true
      accounts:
      - name: my-k8s-account
        requiredGroupMembership: []
        providerVersion: V2
        permissions: {}
        dockerRegistries: []
        configureImagePullSecrets: true
        cacheThreads: 1
        namespaces: []
        omitNamespaces: []
        kinds: []
        omitKinds: []
        customResources: []
        cachingPolicies: []
        kubeconfigFile: configserver:k8sconfigs/my-k8s-cluster.config
        oAuthScopes: []
        onlySpinnakerManaged: false

The new account gets populated after one minute approximately.

Verify if your new accounts are loaded

Open your browser, log in to Spinnaker (if required), and access the Credentials page (e.g. https://spindd.opsmx.com:30084/credentials/“). Check if the account information is available.
Use a sample Spinnaker pipeline, and add a Deploy (Manifest) stage. Then in the stage configuration, check if the Account field is able to list your new account.
If the newly added account is not visible in your Spinnaker pipeline, check the clouddriver pod’s log of what is going wrong.

If you want to know more about the Spinnaker or request a demonstration, please book a meeting with us.

OpsMx is a leading provider of Continuous Delivery solutions that help enterprises safely deliver software at scale and without any human intervention. We help engineering teams take the risk and manual effort out of releasing innovations at the speed of modern business. For additional information, contact us

Tags:

Sandesh Jain

Sandesh is a Principal Engineer at OpsMx. He is currently leading DevOps/CICD practice and working with large multinational enterprise customers (Fortune 50) in migrating to Continuous Delivery. He has wide experience in the Development and Operations of enterprise applications spanning various domains like e-governance, e-learning, Human Resource Capital Management (HCM). His expertise in building and deploying multi-tenant applications on large SAAS deployments lead to international publications and patents in respective domains. He has filed patents in the area of effective data retrieval mechanism and self-configurable applications. He has expertise in DevOps and monitoring tools. LinkedIn: https://www.linkedin.com/in/sandesh-j-62b6a215/

29th July 2021

15th Jul 2021

01st Jul 2021

17th Jun 2021

Spinnaker – Configuring Dynamic Accounts in Clouddriver for Kubernetes

Configure dynamic account in Spinnaker for Kubernetes

How does Spinnaker load external Git-based Clouddriver configuration to be used for Kubernetes?

Detailed Procedure to setup Dynamic Account in Spinnaker for Kubernetes

Verify if your new accounts are loaded

Sandesh Jain

You May Like

Setting up Google Cloud Build (GCB) as CI System

Harness and OpsMx : Different Approaches to Modern CD (Continuous Delivery)

Spinnaker Use Cases at Cisco, Intuit and Pure Storage

3 Comments

Submit a Comment Cancel reply