Moving from VMware to Proxmox

Why bother?

I’ve always been a fan of VMware and how everything pretty much just works. However, one of the major downsides has been the visibility of hardware monitoring. VMware used to be a RedHat derivative, so you could at least use tools built for Linux but now it is its own custom thing with its own set of command line tools and it’s just generally unpleasant to use. My primary virtualization cluster has been Proxmox with a Ceph backend for a while now and I’m loving how easy it is to migrate VMs, how fast the storage backend is, how much I don’t have to use iSCSI, and how I can upgrade the underlying hosts without any downtime for the guests… But the one thing that has been holding me back is the networking.

VMware’s networking is extremely simple and I had multiple vSwitches and virtual machines assigned to specific switches to break things on the network up, etc. I know that Proxmox supports Open vSwitch and you can get similarly crazy with it, but it involves learning another configuration DSL as opposed to VMware’s networking which was GUI-based and very easy point-and-click. That said, there are three VMware servers that I’ll be able to free up for other purposes by moving the VMs over to the Proxmox cluster, so…it’s time.

Migrating a Linux VM from VMware to Proxmox

By all accounts, it’s much easier to move Linux guests from VMware to Proxmox, so I figured that’s where I’d start. Most of the VMs that are still hosted on VMware are crucial for one reason or another, so downtime is an inconvenience. However, since I’m at home, my VPN virtual machine can probably serve as the test case.

# Shut down the VPN server.
vpn# poweroff

# Copy the VMware disk images to the Proxmox cluster.
vmware1# scp /vmfs/volumes/uuid/VPN/VPN.vmdk root@proxmox1:/tmp/
vmware1# scp /vmfs/volumes/uuid/VPN/VPN-flat.vmdk root@proxmox1:/tmp/

# Convert the disk image into RAW format so that it can be used in Proxmox.
proxmox1# qemu-img convert -f vmdk /tmp/VPN.vmdk -O raw /tmp/VPN.raw

Note that there are two VMDK files for the VMware guest. VPN-flat.vmdk is the disk image and VPN.vmdk is a disk configuration file/pointer to the flat image. If you attempt to qemu-img convert just the image, it’ll throw the following error: qemu-img: Could not open 'VPN.vmdk': invalid VMDK image descriptor

With the converted disk image, it’s time to recreate the virtual machine. I used the Proxmox web interface to create a VM with identical specs (1 core, 256 MB RAM in this case). I detached and removed the existing storage. The VM ID that Proxmox assigned–which we’ll need to put the new disk image in place–was 110.

# Move the RAW disk image into place.
proxmox1# mv /tmp/VPN.raw /var/lib/vz/images/110/vm-110-disk-1.raw

# Edit the VM config for the new VM to add the converted disk.
proxmox1# vi /etc/pve/qemu-server/110.conf

...
unused0: local:110/vm-110-disk-1.raw

Back in Proxmox, I refreshed the hardware list and saw that the disk was added. Double-clicked on the disk to add it as an in-use disk (added as SCSI), and booted the system. It came up and appeared to be fine. A few minor issues:

1. The network interface name changed, so my /etc/network/interfaces file was wrong.
2. The MAC address for the network interface changed, so my DHCP server didn’t recognize it for a lease.

Both of those were easily remedied. I tested the VPN connection from the outside world and everything was good.

Migrating a Windows VM from VMware to Proxmox

The process for migrating a Windows VM wasn’t much different except that the VM had to be prepared for the move in advance. Specifically, there’s a registry key that needs to be set to use IDE as a boot device otherwise you’ll end up with a blue screen at boot with the error “Stop 0x0000007B”. Proxmox provides a file (mergeide.zip) that you can import in advance of moving the VM.

With the IDE registry change in place, I shut down the Windows VM, moved the disk over to the Proxmox server, converted it from VMDK, and added the disk to the VMID.conf file as an unused disk. (Basically the same steps as for a Linux VM.) Back in the Proxmox web interface, double-clicked the unused disk and added it to the system. Note that if you don’t add it as an IDE disk, you will continue to have problems. You can also install virtio drivers from RedHat that will let you use paravirtualized devices on Windows (and boot from a VirtIO device instead of an IDE device). After that, the Windows VM booted up as normal. Conversion complete.