Caching for Building Spring Boot App Image with Kaniko

This paper explores the different cache options to speed up the Spring Boot app image build when using Kaniko with Tekton pipelines.

Create a sample Spring Boot app with the Sprint Initializr. Select Maven, add the Spring Web dependency, generate, and download the project zip file.

Unzip it, and create the following Dockerfile to build the image.

FROM maven:3.6.3-jdk-11 AS builder
WORKDIR /workspace
RUN mvn clean install && ls -ltr target
FROM openjdk:11
WORKDIR /app
COPY --from=builder /workspace/target/demo-0.0.1-SNAPSHOT.jar .
CMD ["java", "-jar", "demo-0.0.1-SNAPSHOT.jar"]

I am using Tekton pipelines with Kaniko to build the container image. The kubernetes environment is OpenShift 4.5.

The Tekton task to build the image is shown below,

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: build-image
namespace: springboot-app
spec:
params:
- name: pathToDockerFile
default: /workspace/Dockerfile
- name: pathToContext
default: /workspace
- name: imageUrl
default: image-registry.openshift-image-registry.svc:5000/springboot-app
- name: imageTag
default: "spring-app:v1.0"
- name: sshHost
default: 192.168.10.69
- name: sshFullPath
default: springboot-app/demo.tar.gz
steps:
- name: scp-source
image: kroniak/ssh-client
command:
- sh
- -c
- "
scp -o StrictHostKeyChecking=no -i /ssh-key/sshkey ubuntu@$(params.sshHost):$(params.sshFullPath) /workspace/src.tgz \
&& tar zxvf src.tgz
"
volumeMounts:
- name: ssh-key
mountPath: /ssh-key
- name: build-image
image: gcr.io/kaniko-project/executor:debug
securityContext:
runAsUser: 0
command:
- sh
- -c
- "
/kaniko/executor \
--dockerfile=$(params.pathToDockerFile) \
--destination=$(params.imageUrl)/$(params.imageTag) \
--context=$(params.pathToContext) \
--skip-tls-verify
"
volumeMounts:
- name: docker-config
mountPath: /kaniko/.docker
volumes:
- name: docker-config
configMap:
name: docker-config
- name: ssh-key
secret:
secretName: ssh-key
defaultMode: 0400

The first step retrieves the source file from an SSH server. The ssh key is mounted from a secret with the permission set as 0400 to satisfy SCP's requirement on the key. This step also untars the tar file, where the Dockerfile is located.

The second step is the Kaniko build. We set the destination to the OpenShift private registry by using the task parameter. Kaniko will push the image into the registry after the build. The login information for the private registry is mounted from a ConfigMap. (The details for this can be referred to my other paper on medium)

Create the following taskrun resource to run the task,

apiVersion: tekton.dev/v1beta1
kind: TaskRun
metadata:
name: build-and-push-1597572260
namespace: springboot-app
spec:
serviceAccountName: pipeline
taskRef:
name: build-image
params:
- name: pathToDockerFile
value: Dockerfile
- name: imageTag
value: springboot-app:1.0
- name: sshHost
value: 192.168.10.69
- name: sshFullPath
value: springboot-app/demo.tar.gz

The overall task takes about 2min7seconds to complete.

It is noticed that there are two places involving internet downloading

  • Images in the multi-stage dockerfile
  • Maven dependency files for Spring Boot app

As for each build, these two sets of files are relatively static, we can try to eliminate the downloading to speed up the running.

Kaniko supports the image caching. Adding the following command-line arguments to the task,

--cache=true --cache-dir=/image-cache

Then when building the container image with Dokerfile, it will look into the cache directory for the image first, if found it will load the image from there.

Let's create the following task to pre-populate the image cache, which Kaniko provides a warmer tool to do it.

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: cache-images
spec:
steps:
- name: cache
image: gcr.io/kaniko-project/warmer:latest
args:
- --cache-dir=/image-cache
- --image=maven:3.6.3-jdk-11
- --image=openjdk:11
volumeMounts:
- name: image-cache
mountPath: /image-cache
volumes:
- name: image-cache
persistentVolumeClaim:
claimName: kaniko-cache

We need to create a persistent volume through PVC, mount it to the task pod. Then use the “image” option form the Kaniko warmer to cache the images used for container image build, such as maven, OpenJDK, and so on.

Create a taskrun resource to run this task to download and cache the images.

By default, maven cache the dependency modules into $HOME/.m2 directory. Running as a container, if this directory is not persisted, each time the required dependencies will be downloaded again.

We create a persistent volume and mount it for the Kaniko container to use. This leads to the following new Tekton task with the above cache options.

The task is listed below,

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: build-image-with-cache
namespace: springboot-app
spec:
params:
- name: pathToDockerFile
default: /workspace/Dockerfile
- name: pathToContext
default: /workspace
- name: imageUrl
default: image-registry.openshift-image-registry.svc:5000/springboot-app
- name: imageTag
default: "spring-app:v1.0"
- name: sshHost
default: 192.168.10.69
- name: sshFullPath
default: springboot-app/demo.tar.gz
steps:
- name: scp-source
image: kroniak/ssh-client
command:
- sh
- -c
- "
scp -o StrictHostKeyChecking=no -i /ssh-key/sshkey ubuntu@$(params.sshHost):$(params.sshFullPath) /workspace/src.tgz \
&& tar zxvf src.tgz
"
volumeMounts:
- name: ssh-key
mountPath: /ssh-key
- name: build-image
image: gcr.io/kaniko-project/executor:debug
securityContext:
runAsUser: 0
command:
- sh
- "-c"
- "
/kaniko/executor \
--dockerfile=$(params.pathToDockerFile) \
--destination=$(params.imageUrl)/$(params.imageTag) \
--context=$(params.pathToContext) \
--skip-tls-verify \
--cache=true --cache-dir=/image-cache
"
volumeMounts:
- name: docker-config
mountPath: /kaniko/.docker
- name: image-cache
mountPath: /image-cache
- name: m2-cache
mountPath: /root/.m2

volumes:
- name: docker-config
configMap:
name: docker-config
- name: ssh-key
secret:
secretName: ssh-key
defaultMode: 0400
- name: image-cache
persistentVolumeClaim:
claimName: kaniko-cache
- name: m2-cache
persistentVolumeClaim:
claimName: maven-m2-cache

Now run the task again, check the task log,

...
INFO[0008] Resolved base name maven:3.6.3-jdk-11 to builder
INFO[0008] Retrieving image manifest maven:3.6.3-jdk-11
INFO[0009] Found sha256:8481c188daed14300c1ea13ac77953e096012f05b8234c71820647f82144ad73 in local cache
INFO[0009] Found manifest at /image-cache/sha256:8481c188daed14300c1ea13ac77953e096012f05b8234c71820647f82144ad73.json
INFO[0009] Retrieving image manifest openjdk:11
INFO[0010] Found sha256:9efbdac6886418e7c473ee4d59ff728d029fad773364cb671794333dd16e4158 in local cache
INFO[0010] Found manifest at /image-cache/sha256:9efbdac6886418e7c473ee4d59ff728d029fad773364cb671794333dd16e4158.json
...

The Kaniko builder is using the cached image.

After the first run, the m2 cache is populated. In the second run of the task, the maven will no longer download the dependencies from the Internet.

The overall time taken for the image build is now about 1min42seconds, which is about 20% faster then the previous task without any caching. When using larger images and more dependencies of maven, the time saving might be more significant.

Cloud explorer