Static IP for OCP4

Though a DHCP IP address is recommended for the OpenShift on-premise setup (VM or bare metal), in practice we commonly face some restrictions such as no DHCP IP is allowed based on the network security policy, no extra resource to set up the additional infra servers… We need a static IP.

Without network booting, we have to use the OpenShift ISO image to boot up,

When the bootup menu appears, we hit the Tab key and key in the following extra Kernel parameters to define the static IP, the image URL, and the ignition URL,

ip=192.168.10.30::192.168.10.1:255.255.255.0:dev-bootstrap.ocp4.ibmcloud.io.cpak:ens3:none nameserver=192.168.10.39 coreos.inst.install_dev=vda coreos.inst.image_url=http://matchbox.ocp4.ibmcloud.io.cpak:8080/assets/rhcos-4.2.18-x86_64-metal-bios.raw.gz coreos.inst.ignition_url=http://matchbox.ocp4.ibmcloud.io.cpak:8080/ignition?name=dev-bootstrap

What a long typing! 300+ characters and need to do the same for the other three master nodes, three worker nodes. Sadly, there is no copy and paste before the OS is up.

Can we automate this tedious process?

Yes. As the RHCOS is a Linux system that follows the common Linux boot up, why not we define those kernel booting parameters into the ISO image and let it boot up accordingly?

Extract the ISO image

Given the ocp4 ISO image, let's mount it and extract out the content. On a Linux shell,

sudo mount -o loop rhcos-4.2.18-x86_64-installer.iso /mnt/iso
mkdir -p iso
cp -r /mnt/iso/* iso

For MacBook,

hdiutil mount rhcos-4.2.18-x86_64-installer.iso
mkdir -p iso
cp -r /Volumes/CDROM/* iso/

Custom Isolinux config

In the ISO images,

.
├── EFI
│   └── redhat
│       └── grub.cfg
├── README.md
├── images
│   ├── efiboot.img
│   ├── initramfs.img
│   └── vmlinuz
├── isolinux
│   ├── boot.cat
│   ├── boot.msg
│   ├── isolinux.bin
│   ├── isolinux.cfg
│   ├── ldlinux.c32
│   ├── libcom32.c32
│   ├── libutil.c32
│   └── vesamenu.c32
└── zipl.prm

The isolinux directory defined how the Linux system should be boot up. The isolinux.cfg is a text configuration file for the boot menu. Update the Kernel parameters, appending the node-specific settings, as one line.

label linux
   menu label ^Install RHEL CoreOS - Bootstrap
   kernel /images/vmlinuz
   append initrd=/images/initramfs.img nomodeset rd.neednet=1 coreos.inst=yes ip=192.168.10.30::192.168.10.1:255.255.255.0:dev-bootstrap.ocp4.ibmcloud.io.cpak:ens3:none nameserver=192.168.10.39 coreos.inst.install_dev=vda coreos.inst.image_url=http://matchbox.ocp4.ibmcloud.io.cpak:8080/assets/rhcos-4.2.18-x86_64-metal-bios.raw.gz coreos.inst.ignition_url=http://matchbox.ocp4.ibmcloud.io.cpak:8080/ignition?name=dev-bootstrap

The ignition url points to the matchbox services, where it will return the respective ignition content based on the name selector. We will create a Iso image for each of the nodes. In fact, I automate the customization of this file with the following template, joining the lines together.

ip={{ .ip }}::{{ .gateway }}:{{ .mask }}:{{ .hostname }}:{{ .iface }}:none
nameserver={{ .nameserver }}
coreos.inst.install_dev={{ .diskName }}
coreos.inst.image_url={{ .imgUrl }}
coreos.inst.ignition_url={{ .ignUrl }}

Additionally, we want the bootup to be automatic. So let's replace the original default settings default vesamenu.c32 with default linux. This allows the system to use the bootup menu, linux as the default option to boot up without any keystroke.

Re-pack Iso image

After updated the isolinux.cfg file, re-pack the ISO image. Thanks to the mkiso tools, we can easily create the iso bootable image file with command line, and therefore automate it.

Install the toolset, on Ubuntu Linux,

sudo apt install -y mkisofs

On MacBook,

brew install cdrtools

Create the ISO image file from the directory,

cd {{ .workingDir }}
mkisofs -o {{ .name }}.iso -rational-rock -J -joliet-long -eltorito-boot isolinux/isolinux.bin -eltorito-catalog isolinux/boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table {{ .dir }}

The ISO file is about 70mb (for OCP 4.2). With some further helping tools like govc or simply scp we can transfer the iso files to the VMware or to the KVM host. Then mount the iso files to the VM and boot up with the static IP and its network settings automatically, therefore overcomes the environmental restrictions of DHCP.

It is noticed that in the 4.2 release, after bootup, the static IP is written to the /etc/sysconfig network configure files. The static IP will persistent after system reboot.

Update: In real practice, on top of this, I have automated the ISO image creation based on some configuration files using the magefile utility so that different type of nodes for the different environments can have their ISO image easily.