Using Ksync for file synchronization in Kubernetes clusters

Ksync is a file synchronization tool for Kubernetes-native application development. It speeds up developers who build applications for Kubernetes. It saves you time during new feature development and it allows you to iterate on the code without the overhead of running through your entire automated pipeline every single time.

Ksync transparently updates containers running on the remote Kubernetes cluster from your local checkout. This enables developers to use their favorite IDEs, to work from inside a Kubernetes cluster instead of from outside it. There is no reason to wait minutes to test code changes when you can see the results in seconds.

Advantages with Ksync 

1. Editing files can be done locally using any editor.
2. Two-way sync between the pod and local is present.
3. No need for the specific copy operation.
4. Only one local connection can be made and thereby making it edit safe.

Disadvantages

1. Syncing locally stops the application service in the pod automatically.
2. Need to close and re-establish the connection for every change made locally.

Our current development method

At OpsMx, while developing the Autopilot application, the ‘autopilot’ pod running inside any namespace will have a ‘logs’ folder which is the only write-enabled folder. Whenever the Data Science (DS) team makes the needed code changes locally, the ‘datascience’ folder modified locally will be copied into the ‘logs’ folder. Then the DS application service will be restarted and the new changes will get reflected from the next run. For every new change made locally, the above process needs to be repeated. The default editor in the pod is ‘vi’ which makes editing tougher directly in that pod. There is no ‘Git’ in the pod to pull the code changes directly.

New development method using Ksync

The ‘datascience’ folder inside the ‘logs’ folder is synced locally using Ksync. Once the sync is completed, the DS application service inside the pod gets stopped automatically. The changes can now be made locally using any editor and the same will get reflected in the pod. After making the changes, the local sync connection needs to be closed. The DS service can now be started and the changes will get reflected from the next run.

Ksync Installation Steps

Step 1: Install ksync.

$ curl https://ksync.github.io/gimme-that/gimme.sh | sudo bash

Enter the sudo password.

Output:

Checking GitHub for the latest release of ksync

Found release tag: 0.4.5

Downloading ksync_linux_amd64

No previous install found. Installing ksync to /usr/local/bin/ksync

Step 2: Initialize ksync and install the server component on your cluster. (This is a one-time operation)

$ ksync init

Output:

==== Local Environment ====

Fetching extra binaries                  ✓

==== Pre-flight checks ====

Cluster Config                              ✓

Cluster Connection                        ✓

Cluster Version                              ✓

Cluster Permissions                        ✓

==== Cluster Environment ====

Adding ksync to the cluster              ✓

Waiting for pods to be healthy         ✓

==== Post-flight checks ====

Cluster Service                                ✓

Service Health                                ✓

Service Version                                ✓

Docker Version                                ✓

Docker Storage Driver                      ✓

Docker Storage Root                        ✓

==== Initialization Complete ====

Step 3: Startup the local client.

$  ksync watch &

Output:

lingaselvan@lingaselvan-Vostro-3590:~$ INFO[0000] listening bind=127.0.0.1 port=40322

INFO[0002] syncthing listening port=8384 syncthing=localhost

INFO[0073] finished unary call with code OK grpc.code=OK grpc.method=GetSpecList grpc.service=proto.ksync.Ksync grpc.start_time=”2020-11-03T12:31:26+05:30″ grpc.time_ns=9551 peer.address=”127.0.0.1:52470″ span.kind=server system=grpc

Step 4: Create a new empty folder in the local.

$ mkdir /home/lingaselvan/Downloads/test_folder

Step 5: Create a new spec that describes a folder to sync between a local directory and a directory inside a running container on the remote cluster.

$ ksync -n <namespace> create - -selector=app=oes-autopilot /home/lingaselvan/Downloads/test_folder /home/ubuntu/datascience

Step 6: Check status

$ ksync get

Output:

 NAME              LOCAL                       REMOTE                      STATUS                POD                            CONTAINER                                                                                                                

————-+———————–+————————+—————+——————————–

firm-calf    Downloads/test_folder  /home/ubuntu/datascience   starting   oes-autopilot-5d8bff7659-7kssz   

Step 7: Close a ksync watch connection

$ ps -ef | grep ksync

$ kill -9 <ksync-id>

Step 8: Delete a container

$ ksync delete firm-calf

---

If you want to know more about the Spinnaker or request a demonstration, please book a meeting with us.

OpsMx is a leading provider of Continuous Delivery solutions that help enterprises safely deliver software at scale and without any human intervention. We help engineering teams take the risk and manual effort out of releasing innovations at the speed of modern business. For additional information, contact us