Pod log size and rotation

Image by Oleg Mityukhin from Pixabay

I have a pod verbosely generating logs. However when I need to search the past logs, the log only starts from somewhere in the past, I am not able to see the full log.

What happens to those pod logs in a Kubernetes/OpenShift environment?

Sample program

Let’s use the following program to pumping logs every second.

package main

import (
 "log/slog"
 "sync/atomic"
 "time"

 "github.com/brianvoe/gofakeit/v7"
)

const one_mega int = 1024 * 1024

func main() {
 done := make(chan struct{})

 ticker := time.NewTicker(1 * time.Second)
 var counter uint64
 go func() {
  for {
   select {
   case <-done:
    return
   case <-ticker.C:
    atomic.AddUint64(&counter, 1)
    slog.Info("message", "counter", counter, "msg", gofakeit.LoremIpsumSentence(one_mega/10))
   }
  }
 }()
 <-done
}

Every second, a log entry is created with a counter number and about 0.1 Mb size of text.

Build the application as a container image, push it into the OpenShift private registry, and deploy it as a deployment into OCP.

Missing logs

Check the log with the following command.

oc logs pod-log-6dfdffcf9b-4spm7 | less -S