Introduction

I've been a big fan of VirtualBox for years. But with the last two iterations, I have had more and more issues. So now I've been looking in using qemu directly.

Pros of VirtualBox

• Simple to Use
• Nearly all settings accessible from the interface
• Network expensions that allow for setups not possible with qemu
• Easy to open/close the UI of a VM

Cons of VirtualBox

• The mouse gets stuck once in a while1
• On Ubuntu 24.04, once all the RAM is used by the OS for caches (i.e. I have 512Gb and use between 2% and 5% of the memory, the rest are caches...), VirtualBox stops working
• When I upgraded my OS, somehow, the one drive I had encrypted cannot be read by VirtualBox anymore; it says that the drive is not encrypted

Using QEmu Instead

Now, I'll do everything on the command line. There is a GUI manager, but I don't like it because many of the options I'd like to use are not available there. Also I always wanted to use the VirtualBox command line, but never really got into it. So this is my chance to do so...

Create a VM

First, we have to create a VM. Note that I like to have what I call a "Clean Server" of each version of Ubuntu. That way, I can just upgrade that version, clone it, and start with a fresh server very quickly without having to re-install everything from a cdrom/Internet.

virt-install \
    --osinfo detect=on,require=on \
    --metadata name=ubuntu2404,title="Ubuntu 24.04" \
    --memory 8192 \
    --vcpus vcpu=4 \
    --network bridge=virbr0 \
    --disk path=/mnt/vm-storage-space/clean/ubuntu2404.img,size=50,sparse=yes,format=raw \
    --cdrom /path/to/iso/ubuntu-24.04.2-live-server-amd64.iso

The --osinfo option is used to not have to define the OS for each type of VM and the "require =on" is to make sure that if the detection fails, the whole command fails.

The --metadata defines a VM name, don't use spaces or such. The title, on the other hand can include spaces. Make sure to use quotes.

The --memory defines the memory size of the VM. 8Gb here. Note that I tried to use the memory size (i.e. "8GiB") and it did not work. You have to use a number of megabytes.

The --vcpus defines the number of CPUs to use within the VM.

The --network defines the name of the bridge to use. By default, the virtual system creates the virbr0 so we can use that here. At the moment, I'm not too sure why you'd want to use a different bridge. If you really need (can) create so many VMs that you'd need multiple briges... you probably have a large server and this may not be the docs you need to read for that...

The --disk path defines a path to a .img file. If the file does not exist yet, it will be created. The size parameter defines how many Gb the disk will be. The sparse flag defines whether to allocate the whole thing or not. Be careful since not allocating all (sparse=yes) can cause two issues: (1) installation is too slow and fails with various I/O errors; (2) you end up creating hundred of VMs and they all slowly grow their disk which in the end tries to eat up  more disk space than your physical disk offers. (i.e. the disk will first take up 4Kb, once the OS is installed, it will take around 10Gb, once you install and run services on that VM, it will add logs and data and eventually end up using the entire 50Gb or at least try to...) There is another possible drawback: the sparse capability means that blocks get allocated later and thus in a different location on the drive. For SSD, that's not an issue. For an HDD, however, fragmentation can cause slowness as reading the data requires many more head movements.

Finally, the --cdrom defines the ISO to use to boot from. In general, this is a CD image that  has some version of Linux or some other OS to be installed. Here I show Ubuntu 24.04 server. I like to install the server version because it generally installs less UI things and thus makes for a slimmer Linux base image.

Weird Bug? Unexpected Disk Size

When I created the VM, I asked the installer to use the entire drive. Only once it was all created, the drive was about 28Gb instead of nearly 50Gb. So it used rougly 50% instead of the entire drive. I have no clue what happened.

To fix the issue, I had to extend to the drive and then resize the file system:

$ sudo lvextend -l +100%FREE /dev/ubuntu-vg/ubuntu-lv
$ sudo resize2fs /dev/ubuntu-vg/ubuntu-lv

Note that since I used an LVM and ext4, I could do that on the system directly. Quite practical in this situation (note that ext4 can only enlarge a live partition, not shrink it; to do a shrink, you'd want to boot on the ISO above and do the resizing from that tools found there,)

Renaming the Drive

As a side note to myself: Keep in mind that if you want to be able to mount the image drives later, you may want to give them a name that won't clash with another drive. So the ubuntu-vg and ubuntu-lv should be changed. The LVM system uses those names in its mapper so if two distinct drives use the same name, it won't work properly.

TODO: add command(s)

Cloning

Once we have a clean server VM, we are ready to clone it for our own purposes. In my case, I'm looking at creating a build server.

But first make 100% sure that the machine being cloned is shutdown.

From a console inside the clean server to be cloned do:

$ sudo init 0

Now we're ready to run the next command, the actual cloning. The --file option is used to specify the source being cloned:

$ virt-clone \
    --original ubuntu2404 \
    --name buildserver \
    --file /mnt/vm-storage-space/servers/buildserver.img

The virt system automatically copies the image file for you. Only the destination is indicated. The source is not necessary since the --original knows where that is.

WARNING: if your source has multiple drives, then the virt-clone command needs one --file option per drive.

Note: if the source drive is sparse, then the copy will also be sparse. However, you cannot copy a drive which isn't sparse as a sparse drive with this function. You can later run a command to find empty blocks and make the file sparse if you'd like. Keep in mind that sparse file on HDD can end up being heavily fragmented.

Setup the Host with a Static IP

For my need, I want to have simple scripts and settings that allow me to access the VM with an IP address that does not change between runs. To force a specific IP we can use the command below. That command requires two parameters which we need to gather first:

1) Supported IP Addresses

TODO: how do you obtain the ip range supported by the corresponding bridge?

2) Mac Address of Interface

Start the VM if it's not already running and run:

$ ip address

This prints all the interfaces and their Mac address.

3) Actually change the IP address of that host

$ virsh net-update default add ip-dhcp-host \
      "<host mac='52:54:00:17:33:e4' name='buildserver' ip='192.168.122.224'/>" \
       --live --config

Setup Memory and CPUs

The system has access to the XML which can be tweaked with the following commands:

$ virsh setmem buildserver 16384 --config
$ virsh setvcpus buildserver 8 --config --maximum
$ virsh setvcpus buildserver 8 --config

The setmem sets the amount memory the VM will receive. Remember that one option does not change the live system. I've tried and it was not working, although  I may have needed the --live command line option.

The setvcpus can be used to change the number of CPUs in a VM. Note that the biggest it gets the more thread the qemu will be using to run all those processes.

The number of CPU must be lower or equal to the maximum. You always define the max first if you are increasing the number of CPUs. If decreasing the number of CPUs, You first want to decrease the number of CPU then the maximum.

Start Clone

Once the clone is ready, you can start the server like so:

$ virsh start buildserver

It should start pretty quickly.

Once started, verify that you have the correct amount of memory and disk space:

$ free
$ df

From the above, free should tell us we have 16Gb.

Similarly, the df command should show that the drive is around 48Gb,

View Clone Display

It is possible to check out the display. By defaul, that feature is not turned on.

$ virt-viewer buildserver

Note that this command blocks until you close the window.

List Available VMs

The VMs you create are added to a list which can be displayed using:

$ virsh list --all

The --all option is to show active (a.k.a. running) domains and also the non-active domains.

• 1. I've not worked with qemu for long enough to know whether it would also get stuck with that one... so that may change later.