How to Clone a Live Production Linux Server with This Cool Technique

Do you find yourself stuck with some old Ubuntu Server that runs some critical application on a Physical Hardware which is running out of resources. You don’t want to waste rack space by cloning it to another physical server. You are a smart guy, you know cloning it to a VM gives you all the benefit of modern day server management.

What if the original hard drive has couple hundred Gigs of disk space and the actual data is only a few to a few dozen Gigs? You don’t want all that unused disk space to be also cloned into the VM via dd command. Is there a way to clone the server to a smaller disk that lives on a KVM VM guest? What if you can’t risk shutting down the physical server because it is in production and is a single point of failure? Look no further, here’s how you can live clone a old running Linux server to a KVM virtual machine.

Prepare New KVM Guest Image

Step 1. - Log in to your KVM host server and create a qcow2 image file with enough disk space to clone the actual data from the physical server:

qemu-img create -f qcow2 new-vm.qcow2 100G

Step 2. - Let’s attach the qcow2 file to a local device on KVM host so you can partition and format it.

mkdir /mnt/new-vm
modprobe nbd max_part=8
qemu-nbd -c /dev/nbd0 new-vm.qcow2
cfdisk /dev/nbd0
mkfs.ext3 -v /dev/nbd0p1

cfsidk - Create partition. Tailor this to your environment, instruction HERE.
mkfs.ext3 - Format the disk to ext3 filesystem. You can also use mkfs.ext4 or other type of filesystem depend on your situation.

Step 3. - Mount the drive

partprobe /dev/nbd0
mount /dev/nbd0p1 /mnt/new-vm

Clone Data from Physical Server

Step 4. - Rsync (Copy) all the files from the physical server
Do a dry run first.

rsync -e 'ssh -p 22' -avxn root@physical.server.com:/ /mnt/new-vm/

Now do it for real if you don’t see any problem with the dry run.

rsync -e 'ssh -p 22' -avx root@physical.server.com:/ /mnt/new-vm/

Step 5. - Sync data for the last time
Make sure all cached data are written to the hard drive on the physical server by running:

sync;sync;sync

Go back to the KVM host and do final rsync again

rsync -e 'ssh -p 22' -avx root@physical.server.com:/ /mnt/new-vm/

Clean up

Step 6. - Update disk UUID in GRUB bootloader
On the KVM host:

blkid

You will get a list of mounted drives UUID. Copy the one that follows /dev/nbd0. Use that value and replace all the disk UUID in /mnt/new-vm/boot/grub/grub.cfg

Step 7. - Clean up /etc/fstab
Do the same for /mnt/new-vm/etc/fstab and replace with the new UUID or just remove all UUID in the fstab file.

Step 8. - Clean up network settings

• Change IP address by editing /mnt/new-vm/etc/network/interfaces
• Remove cached mac address from udev. /mnt/new-vm/etc/udev/rules.d/70-persistent-net.rules
• Edit any application settings that might create conflict with your physical server once you stand the new VM up.

Step 9. - Un-mount the drive

umount /mnt/new-vm
qemu-nbd -d /dev/nbd0

Stand Up the New VM

virt-install \
--connect qemu:///system \
-n vm-name \
--os-type linux \
--vcpus=2 \
--ram 2048 \
--disk path=/path/to/new-vm.qcow2 \
--vnc \
--vnclisten=127.0.0.1 \
--noautoconsole \
--import

Change the vm-name, vcpus, ram, and disk location to your liking. If everything go smooth, you should see these success messages on screen:

Starting install...
Creating domain...                                                                             |    0 B     00:01
Domain creation completed. You can restart your domain by running:
  virsh --connect qemu:///system start new-vm

You can now try to ssh into the new VM using the new IP and see if things are working. To troubleshoot any issue, you will need to use VNC to connect to your VM and debug accordingly. Congratulations! You’ve now save your company from a legacy dying server to the new, shiny, scalable, replicable, and testable VM. Now this is money ~~~