It goes without saying that if you want to configure (virtual) network devices with any semi-sane configuration mechanism¹, the device must have a working IP address. Here’s the time-honored method² to assign an IP address to a virtual network device:

• Start the virtual machine (using a GUI)³
• Open a new window: either a telnet session to the virtual console port or a full-blown virtual console (GUI) session.
• Manually configure the IP address, the SSH server, and the user credentials on the first interface.

Some networking CLI jockeys⁴ still find that the pinnacle of evolution. Everyone else (including software developers) realized ages ago that we need something better: consistent, predictable management IP addresses assigned automatically by the orchestration system.

That’s easy to do in the pure container world. After all, the container Ethernet interfaces are just one half of the vEth virtual cable residing in a separate namespace, and the host operating system has full access to them. Configuring their IP addresses is trivial. Worst case, use ip netns exec container ip address add ip dev eth0 (note: the eth0 device that gets the IP address is within the container namespace).

What about the virtual machines? We use the same IPv4 address assignment mechanism we use elsewhere: DHCPv4; that’s why every cloud-ready device image includes a working DHCP client and a default username, or a cloud-init client that lets you specify user credentials when creating the virtual machine.

However, most networking devices don’t start with a DHCP client configured on their first interface; that’s why we have to build custom Vagrant boxes before we can use networking devices with Vagrant or netlab. The initial configuration of those custom Vagrant boxes also includes the management VRF, the SSH server, and the user credentials.

Things get a tiny bit more interesting in the IPv6 world. Most operating systems have IPv6 enabled by default and use SLAAC to get a working IPv6 address. Unfortunately, most cloud providers that have ever faced a large-enough scaling challenge⁵ hate unpredictable dynamic IP addresses and control-plane address-discovery mechanisms. Every large public cloud assigns predictable, preconfigured IPv6 addresses to virtual machines via DHCPv6, including the one built by the company otherwise engaged in a nonsense⁶ anti-DHCPv6 crusade⁷.

So far, so good:

• We can assign IP addresses to container interfaces directly
• We use DHCPv4/DHCPv6 to give IPv4/IPv6 addresses to properly-preconfigured virtual machines.

What about virtual machines running in containers? The container orchestration system assigns the IP address to the container interface, and somehow that IP address must get to the VM running inside that container. One could (theoretically) start a DHCP server inside the container and use DHCP to assign an IP address to the VM management interface; the vrnetlab project took a different approach: the container initialization script (which also starts the VM) uses a telnet session to its loopback interface to reach the virtual serial port of the virtual machine and an equivalent of an expect script to:

• Abort the auto-configuration process (if needed)
• Start a console session
• Enter configuration mode
• Download the startup configuration, including the management IP address, the SSH server configuration, and the user credentials.

It looks like we truly cannot escape the curse of the virtual console cable 🤦‍♂️