Recipe for an HTTPS Sidecar

Image for post
Image for post
Image from epicurious.com

This paper serves the purpose of reference for the well-known sidecar pattern of Kubernetes.

Implement the HTTPS interface for a container application that doesn’t have https implemented.

Run a Nginx container beside the app in the same pod. The Nginx web server listens on the HTTPS port, and reverse proxy the request to the actual app in the same pod.

A toy app to illustrate the idea. The golang app is listed as below. Notice that no https is implemented.

package mainimport (
"fmt"
"log"
"net/http"
"os"
"time"
)
func hello(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "Hello!")
}
func date(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "time now: %s", time.Now().Format("15:04:05"))
}
func main() {
http.HandleFunc("/", hello)
http.HandleFunc("/date", date)
port := os.Getenv("LISTENING_PORT") if port == "" {
port = "8080"
}
log.Printf("listening on port:%s", port)
err := http.ListenAndServe("localhost:"+port, nil)
if err != nil {
log.Fatalf("Failed to start server:%v", err)
}
}

Build the app,

export CGO_ENABLED=0
go build -o myhandler src/*.go

Prepare the Dockerfile,

FROM alpine
RUN addgroup -S appgroup && adduser -S appuser -G appgroup && mkdir -p /app
ADD myhandler /app
RUN chmod a+rx /app/myhandler

USER appuser
WORKDIR /app
ENV LISTENING_PORT 8080
CMD ["./myhandler"]

Build the image and push it to Docker hub.

sudo docker build -t zhiminwen/hello:v1 .

Followings are the steps to implement the HTTPS interface for this app.

Use cfssl to create the required HTTPS certificate. Download the cfssl binary for your platform.

  1. Create CA

Prepare the following ca config file, say “certs/myca.json”

{
"CN": "example.net",
"hosts": [
"example.net",
"www.example.net"
],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "SG",
"ST": "SG",
"L": "Singapore"
}
]
}

Run command

cd certs
cfssl.exe gencert -initca myca.json | cfssljson -bare myca

This creates the following files of

  • myca.pem: the CA cert, public key
  • myca-key.pem: the CA cert, private key

2. Create SSL cert for Nginx

Create the following “ca-config.json” file with the profiles defined as

{
"signing": {
"default": {
"expiry": "43800h"
},
"profiles": {
"server": {
"expiry": "43800h",
"usages": [
"signing",
"key encipherment",
"server auth",
"client auth"
]
},
"client": {
"expiry": "43800h",
"usages": [
"signing",
"key encipherment",
"client auth"
]
}
}
}
}

Define the cert requirement in json format as below, save as “myrequest.json”.

{
"CN": "hello-server",
"hosts": [
""
],
"key": {
"algo": "rsa",
"size": 2048
}
}

Generate the signed certs with the command below,

cd certscfssl gencert -ca=myca.pem -ca-key=myca-key.pem -config=ca-config.json -profile=server -hostname="127.0.0.1" myrequest.json | cfssljson -bare hello-server

The following certificates are then created

  • Certificate: hello-server.pem
  • Private key: hello-server-key.pem

We will then use these certificates to configure Nginx SSL.

The configuration for nginx.conf is minimum. We define a SSL server with TLS v1.2, and redirect the traffic to the actual application.

worker_processes  1;events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
server {
listen 443 ssl;
server_name localhost;
ssl_certificate /app/cert/hello-server.pem;
ssl_certificate_key /app/cert/hello-server-key.pem;
ssl_protocols TLSv1.2;
ssl_ciphers EECDH+AES128:RSA+AES128:EECDH+AES256:RSA+AES256:!EECDH+3DES:!RSA+3DES:!MD5;
ssl_prefer_server_ciphers on;
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_pass http://127.0.0.1:8080/;
}
}
}

The ssl_certifcate and ssl_certificate_key are the public key and private key of the https server.

The location block redirects the traffic to the application container which is located at the same pod with Nginx.

Create the K8s configMap for the nginx.conf and secret for the certs.

kubectl create cm hello-sidecar-nginx-conf --from-file=nginx.conf=./nginx.confkubectl create secret generic hello-sidecar-nginx-certs --from-file=hello-server-cert=./hello-server.pem --from-file=hello-server-key=./hello-server-key.pem

We are using the “ — from-file=key=filename” format, so the configMap and secret have the key fields specified as what we have defined.

The full deployment file is listed as below,

---
apiVersion: v1
kind: Service
metadata:
name: hello
labels:
app: hello
spec:
type: NodePort
ports:
- port: 443
targetPort: 443
protocol: TCP
name: https
selector:
app: hello
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: hello
labels:
app: hello
spec:
replicas: 1
selector:
matchLabels:
app: hello
template:
metadata:
labels:
app: hello
spec:
containers:
- name: hello
image: zhiminwen/hello:v1
imagePullPolicy: IfNotPresent
env:
- name: LISTENING_PORT
value: "8080"
- name: tls-sidecar
image: nginx
imagePullPolicy: IfNotPresent
volumeMounts:
- name: secret-volume
mountPath: /app/cert
- name: config-volume
mountPath: /etc/nginx/nginx.conf
subPath: nginx.conf
volumes:
- name: secret-volume
secret:
secretName: hello-sidecar-nginx-certs
items:
- key: hello-server-cert
path: hello-server.pem
- key: hello-server-key
path: hello-server-key.pem
- name: config-volume
configMap:
name: hello-sidecar-nginx-conf

The main app is nothing special. It behaves as its normal.

Define the configMap volume and the secret volume.

For the key of “hello-server-cert”, we specify the path with the file name, so the pod will mount using the file name of “hello-server.pem”. The same goes for the other key.

Name the nginx container as “tls-sidecar”. Mount the config-volume with both mountPath and subPath so that only the file “nignx.conf” will be presented in the target directory, and avoid overwriting the whole default directory of Nginx.

Mount the secret to the directory of “/app/cert” to match the exact file name defined in the nginx.conf.

Lastly, create a service that exposes the Nginx https server port 443 as nodePort.

Apply the yaml file to deploy it.

Once the pods are running, find out the NodePort. Do a HTTPS connection to the nodePort, the https is working.

Check the certificate,

Image for post
Image for post

See the second paper of the sidecar series, where I explored the client certificate authentication and the Prometheus in IBM Cloud Private.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store