Dual-home ENI routings for RHEL

Image Generated By DALLE 3

I have a RHEL9 EC2 instance with dual-home ENI cards. One interface is located in the data subnet which suppose to handle data related communications, the other one is in the management subnet which is for management purpose. When they talk back to the on premise network, firewall rules must be applied.

We have quickly identified, traffic is being blocked by firewall even firewall is being opened. Further investigation shows that the source of the communication doesn’t match what the fireall rules specified. Knowing that in AWS the routing table definition won’t populate into the EC2 instance, we know it must be the routing problem of the underlying Linux.

Default Route

In Linux, unless we set the specific routing, all the routes will follow what the default route defined. Check out the route,

# ip route
default via 10.10.0.1 dev eth0 proto dhcp src 10.10.0.7 metric 100
default via 10.10.3.1 dev eth1 proto dhcp src 10.10.3.7 metric 101
10.10.0.0/26 dev eth0 proto kernel scope link src 10.10.0.7 metric 100
10.10.3.0/26 dev eth1 proto kernel scope link src 10.10.3.7 metric 101

The first two are the default routes as we have two NIC cards. The default route states that the route will go out using the subnet 10.10.0.0/24’s gateway 10.10.0.1 using the 1st NIC card with the IP of…