Recipe for an HTTPS Sidecar

Image from epicurious.com

This paper serves the purpose of reference for the well-known sidecar pattern of Kubernetes.

Goal

Solution

Sample App

package mainimport (
"fmt"
"log"
"net/http"
"os"
"time"
)
func hello(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "Hello!")
}
func date(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "time now: %s", time.Now().Format("15:04:05"))
}
func main() {
http.HandleFunc("/", hello)
http.HandleFunc("/date", date)
port := os.Getenv("LISTENING_PORT") if port == "" {
port = "8080"
}
log.Printf("listening on port:%s", port)
err := http.ListenAndServe("localhost:"+port, nil)
if err != nil {
log.Fatalf("Failed to start server:%v", err)
}
}

Build the app,

export CGO_ENABLED=0
go build -o myhandler src/*.go

Prepare the Dockerfile,

FROM alpine
RUN addgroup -S appgroup && adduser -S appuser -G appgroup && mkdir -p /app
ADD myhandler /app
RUN chmod a+rx /app/myhandler

USER appuser
WORKDIR /app
ENV LISTENING_PORT 8080
CMD ["./myhandler"]

Build the image and push it to Docker hub.

sudo docker build -t zhiminwen/hello:v1 .

Followings are the steps to implement the HTTPS interface for this app.

Step 1. Create the certificate

  1. Create CA

Prepare the following ca config file, say “certs/myca.json”

{
"CN": "example.net",
"hosts": [
"example.net",
"www.example.net"
],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "SG",
"ST": "SG",
"L": "Singapore"
}
]
}

Run command

cd certs
cfssl.exe gencert -initca myca.json | cfssljson -bare myca

This creates the following files of

  • myca.pem: the CA cert, public key
  • myca-key.pem: the CA cert, private key

2. Create SSL cert for Nginx

Create the following “ca-config.json” file with the profiles defined as

{
"signing": {
"default": {
"expiry": "43800h"
},
"profiles": {
"server": {
"expiry": "43800h",
"usages": [
"signing",
"key encipherment",
"server auth",
"client auth"
]
},
"client": {
"expiry": "43800h",
"usages": [
"signing",
"key encipherment",
"client auth"
]
}
}
}
}

Define the cert requirement in json format as below, save as “myrequest.json”.

{
"CN": "hello-server",
"hosts": [
""
],
"key": {
"algo": "rsa",
"size": 2048
}
}

Generate the signed certs with the command below,

cd certscfssl gencert -ca=myca.pem -ca-key=myca-key.pem -config=ca-config.json -profile=server -hostname="127.0.0.1" myrequest.json | cfssljson -bare hello-server

The following certificates are then created

  • Certificate: hello-server.pem
  • Private key: hello-server-key.pem

We will then use these certificates to configure Nginx SSL.

Step 2. Create nginx.conf

worker_processes  1;events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
server {
listen 443 ssl;
server_name localhost;
ssl_certificate /app/cert/hello-server.pem;
ssl_certificate_key /app/cert/hello-server-key.pem;
ssl_protocols TLSv1.2;
ssl_ciphers EECDH+AES128:RSA+AES128:EECDH+AES256:RSA+AES256:!EECDH+3DES:!RSA+3DES:!MD5;
ssl_prefer_server_ciphers on;
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_pass http://127.0.0.1:8080/;
}
}
}

The ssl_certifcate and ssl_certificate_key are the public key and private key of the https server.

The location block redirects the traffic to the application container which is located at the same pod with Nginx.

Step 3. Create configMap and secret

kubectl create cm hello-sidecar-nginx-conf --from-file=nginx.conf=./nginx.confkubectl create secret generic hello-sidecar-nginx-certs --from-file=hello-server-cert=./hello-server.pem --from-file=hello-server-key=./hello-server-key.pem

We are using the “ — from-file=key=filename” format, so the configMap and secret have the key fields specified as what we have defined.

Step 4. Create K8s deployment

---
apiVersion: v1
kind: Service
metadata:
name: hello
labels:
app: hello
spec:
type: NodePort
ports:
- port: 443
targetPort: 443
protocol: TCP
name: https
selector:
app: hello
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: hello
labels:
app: hello
spec:
replicas: 1
selector:
matchLabels:
app: hello
template:
metadata:
labels:
app: hello
spec:
containers:
- name: hello
image: zhiminwen/hello:v1
imagePullPolicy: IfNotPresent
env:
- name: LISTENING_PORT
value: "8080"
- name: tls-sidecar
image: nginx
imagePullPolicy: IfNotPresent
volumeMounts:
- name: secret-volume
mountPath: /app/cert
- name: config-volume
mountPath: /etc/nginx/nginx.conf
subPath: nginx.conf
volumes:
- name: secret-volume
secret:
secretName: hello-sidecar-nginx-certs
items:
- key: hello-server-cert
path: hello-server.pem
- key: hello-server-key
path: hello-server-key.pem
- name: config-volume
configMap:
name: hello-sidecar-nginx-conf

The main app is nothing special. It behaves as its normal.

Define the configMap volume and the secret volume.

For the key of “hello-server-cert”, we specify the path with the file name, so the pod will mount using the file name of “hello-server.pem”. The same goes for the other key.

Name the nginx container as “tls-sidecar”. Mount the config-volume with both mountPath and subPath so that only the file “nignx.conf” will be presented in the target directory, and avoid overwriting the whole default directory of Nginx.

Mount the secret to the directory of “/app/cert” to match the exact file name defined in the nginx.conf.

Lastly, create a service that exposes the Nginx https server port 443 as nodePort.

Apply the yaml file to deploy it.

Testing

Check the certificate,

See the second paper of the sidecar series, where I explored the client certificate authentication and the Prometheus in IBM Cloud Private.