How to Integrate AWS Lambda with Spinnaker

Overview

This blog explain’s how to integrate AWS Lambda function with Spinnaker 1.16.2 via Cloud-driver, Also we will be looking how to call AWS Lambda function using API calls and custom Web-Hooks.

NOTE:

This Proof Of Concept expects that you have full admin rights to AWS Console, to create AWS Lambda functions and the respective IAM Roles have been created to access the Lambda function with access key and secret key. Also, please make sure you are well aware of the API calls with IAM roles by using AWS.

How to Enable AWS Lambda in Spinnaker

To enable AWS Lambda function in Spinnaker, please create a file “clouddriver-local.yml” file under your “.hal” config profiles directory.

# vim .hal/default/profiles/clouddriver-local.yml

aws:
enabled: true
lambda:
enabled: true
accounts:
- name: aswath-aws-spinnaker-lamda #CUSTOM NAME FOR LAMBDA
lambdaEnabled: true
requiredGroupMembership: []
providerVersion: V1
permissions: {}
accountId: 'XXXXXXXXXXXXXXX' #AWS ACCOUNT ID
regions:
- name: us-west-2
assumeRole: role/spinnakerManaged #AS PER YOUR IAM ROLE
primaryAccount: aswath-aws-spinnaker-lamda
bakeryDefaults:
baseImages: []
accessKeyId: XXXXXXXXXXXXXXXXXXX #ADD AWS ACCESS KEY
secretAccessKey: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX #ADD SECRET KEY
defaultKeyPairTemplate: '{{name}}-keypair'
defaultRegions:
- name: us-west-2

After adding the above configuration to “clouddriver-local.yml” try running “# hal deploy apply” to make the changes effect.

Please check the Cloud-driver logs for more details. If you ran in to any issues. Also, please ensure after the “# hal deploy apply” the port 7002 is listening. 7002 is the port needed for cloud-driver communication with AWS Lambda.

Debugging Cloud-Driver Error

In this blog example I’ve enabled SSH Tunnelling to the Spinnaker local Debian instance, So that I can locally test the Cloud-driver logs by port forwarding. In your environment it might be different or you can connect directly via the DNS to that server.

If you are using Spinner in K8S Environment, Try #kubectl logs -f -n spinnaker spin-clouddriver-xxxxx” To get the debug logs.

Querying AWS Lambda Functions using Spinnaker Cloud-Driver using CURL

This Test will help you to get the AWS Lambda Functions from CURL CLI, we use GET method to query the AWS Lambda and retrieve the available function.

curl -X GET --header 'Accept: application/json' 'http://localhost:7002/functions?functionName=firstfunctiontest&region=us-west-2&account=aswath-aws-spinnaker-lamda'

Output

Note: Ensure you give Right Function name with respective region name and the account you try to query the AWS Lambda.

 

How To Create New Lambda Function

To create AWS Lambda Function using Cloud-driver spinnaker. Please ensure you have respective IAM Roles created. And the IAM have Valid access to create Functions in Lambda.

Please follow the below example for creating new AWS Lambda Function using spinnaker Cloud-driver.

Example:

curl -X POST   http://localhost:7002/aws/ops/createLambdaFunction   -H 'Accept: application/json'   -H 'Content-Type: application/json'   -d '{
    "region": "us-west-2",
    "functionName": "mylambdafunctiontwo",   #NEW FUNCTION NAME
    "description": "sample",                      #DESCRIPTION NAME
    "credentials": "aswath-aws-spinnaker-lamda",
    "handler": "lambda_function.lambda_handler",
    "memory": 512,
    "publish": "true",
    "role": "arn:aws:iam::XXXX:role/service-role/my-lambda-role", #VALID IAM ROLE
    "runtime": "python3.6",
    "timeout": "90",
    "tags": [{
        "key":"value"
    }
    ]
}'

Output

Since, I had enabled port forwarding I am navigating to http://localhost:7002 for verification.

AWS Console Output

How To Update the Existing Lambda Function

In this below Example I am updating the previously Created Lambda function “mylambdafunctiontwo” Timeout value from “60” to “68”

curl -X POST \
http://localhost:7002/aws/ops/updateLambdaFunctionConfiguration \
-H 'Accept: application/json' \
-H 'Content-Type: application/json' \
-d '{
"region": "us-west-2",
"functionName": "mylambdafunctionone",
"description": "sample",
"credentials": "aswath-aws-spinnaker-lamda",
"handler": "lambda_function.lambda_handler",
"memory": 512,
"role": "arn:aws:iam::XXXXXX:role/service-role/my-lambda-role",
"runtime": "python3.6",
"timeout": "68", #UPDATING TIMEOUT VALUE
"tags": [{
"key":"value"
}
]
}'

Output

AWS Console Output

 

How To Delete AWS Lambda Function

In this example I will be deleting the already created AWS Lambda Function using Cloudconfig driver.

curl -X POST \
http://localhost:7002/aws/ops/deleteLambdaFunction \
-H 'Accept: application/json' \
-H 'Content-Type: application/json' \
-d '{
"region": "us-west-2",
"functionName": "mylambdafunctiontwo", #FUNCTION TO DELETE
"credentials": "aswath-aws-spinnaker-lamda"
}'

Debug Output

 

How To Create Custom Web-Hook Using Spinnaker ORCA

To Create Custom Web-Hook in Spinnaker, Please create “orca-local.yml” under “.hal” default profile.

# vim .hal/default/profiles/orca-local.yml

Add the below content in Spinnaker “orca-local.yml” to test pipeline

webhook:
preconfigured:
- label: Lambda - Get Functions #The Name you see in Pipeline GUI
type: lambdaGetFunctions
enabled: true
description: Get Lambda Functions #ADD DESCRIPTION.
method: GET #WE USE GET FUNCTION TO GET DETAILS.
url: http://localhost:7002/functions #YOUR CLOUDDRIVER SERVER DETAILS. 
customHeaders: #IF YOU ARE USING HTTPS WE DONT NEED THIS SECTION.
Accept:
- "application/json"
- label: Lambda - Update Function Code #THE NAME TO VIEW IN PIPELINE.
type: lambdaUpdateFunctionCode
enabled: true
description: Update Lambda Function Code
method: POST #This is used to POST Function
url: http://localhost:7002/aws/ops/updateLambdaFunctionCode #Please make sure you give your Server name
customHeaders:
Accept:
- "application/json"
Content-Type:
- "application/json"
payload: |-
{
"credentials": "${#root['parameterValues']['account']}",
"region": "${#root['parameterValues']['region']}",
"functionName": "${#root['parameterValues']['functionName']}",
"s3Bucket": "${#root['parameterValues']['bucketname']}",
"s3Key": "${#root['parameterValues']['key']}",
"publish": "${#root['parameterValues']['publish']}"
}
parameters:
- label: Spinnaker Account Name
name: account
type: string
- label: Region
name: region
type: string
- label: Function Name
name: functionName
type: string
- label: S3 Bucket Name
name: bucketname
type: string
- label: S3 Key
name: key
type: string
- label: Publish
name: publish
type: string
- label: Lambda - Update Function Configuration 
type: lambdaUpdateFunctionConfig
enabled: true
description: Update Lambda Function Configuration
method: POST 
url: http://localhost:7002/aws/ops/updateLambdaFunctionConfiguration 
customHeaders:
Accept:
- "application/json"
Content-Type:
- "application/json"
payload: |-
{
"region": "${#root['parameterValues']['region']}",
"functionName": "${#root['parameterValues']['functionName']}",
"description": "${#root['parameterValues']['description']}",
"credentials": "${#root['parameterValues']['account']}",
"role": "${#root['parameterValues']['roleARN']}",
"timeout": "${#root['parameterValues']['timeout']}"
}
parameters:
- label: Region
name: region
type: string
- label: Function Name
name: functionName
type: string
- label: Description
name: description
type: string
- label: Spinnaker Account Name
name: account
type: string
- label: Role ARN
name: roleARN
type: string
- label: Timeout
name: timeout
type: string

You might notice that the parameterValues are being referenced with a #root helper function. This is to ensure that Orca can evaluate the expressions using the parameter values from within the stage.

After making the above changes, please run “hal deploy apply” and ensure the Spinnaker ports are available and listening as-well. Also please wait till the port 7002 comes up.

Since that port is mandatory to have the AWS Lambda communication via cloud-driver.

Creating PipeLine

After the changes to orca, we should now be able to see the new stages while configuring the pipeline. Please select the stage, and provide the values as shown below.

After the successful execution you can see the Timeout value changed to 1 min 8 Sec in the AWS Lambda Function.

Calling Pipeline Using Spinnaker

Pipeline To Update Existing Function

The Same is verified in the AWS – Console, the Description is updated with “Sample”

Same way, we can try adding different functions and fields and connect the Lambda function using spinnaker custom web-hook.

Conclusion

Hope this guide helps you to configure AWS Lambda function successfully using Spinnaker. For details visit https://docs.opsmx.com/