FreeBSD Bhyve Companion Tools

Some time ago I wrote about how FreeBSD Bhyve can be used in the FreeBSD Bhyve Virtualization article. It was quite short after I moved from VirtualBox – and to be honest – I am really glad that I moved to Bhyve.

When I was working with VirtualBox – I was not fully happy – it just felt that FreeBSD is not its native environment. While most things worked the networking stuff sometimes required all VMs reboot and sometimes even entire FreeBSD host hard reset was needed. It was not often – but still.

Now – after almost a year with Bhyve hypervisor – and using vm-bhyve toolkit – I would like to share some tools that make everyday life easier and more productive.

VNC Connection

In the VirtualBox world You either started a VM in SDL mode – with full graphical window and interface … or you started headless w/o any graphical output at all. With Bhyve its quite different. For the UEFI boot machines you start a VM process in the text mode w/o any additional graphics – but You are allowed/able to connect to that machine graphical framebuffer using – for example – vncviewer(1) command for VNC connection.

In the past I just first listed the VMs and then copied and pasted the VNC address to a vncviewer(1) tool.

… and it made me less happy with time – especially as I got more and more VMs – so for that purpose I wrote a simple vnc.sh script that takes Bhyve VM name as argument and then using vm(8) command listing finds the needed VNC address.

I even shortened it with vnc alias.

% alias vnc                                       
vnc=vnc.sh

% vnc ghostbsd

Pause/Resume Bhyve VM

In the past I implemented the desktop-pause.sh script to pause/resume UNIX applications – usually focused on the GUI ones – Pause Any Application – described here in details.

As Bhyve is a type 2 hypervisor – just a process in a FreeBSD OS – we can also use kill(1) to pause/resume the state of each Bhyve VM.

This is where bp.sh tool comes handy (BP stands for Bhyve Pause).

I even shortened it with bp alias.

% alias bp                                       
bp=bp.sh

% bp -p poudriere

You specify a VM name and you have an option to pause/resume the VM process or to toggle the pause/resume state.

The main use case I find for that is – for example – some VM takes too much CPU/RAM resources while You are doing something else on the host desktop/laptop/server system and you want to pause it to save resources for other things – let it be interactive work or even other Poudriere based machine that needs the same resources.

It works like a charm but I would like to point one warning here – either configure some time sync inside all of Your VMs – or just remember to update time in a VM after you paused it for some time. There are no other drawbacks.

Summary

While simple and not epic – these two additions made my day to day Bhyve life a lot easier and allowed me to focus on the work and not on the internals.

I hope they will do the same to Your workloads.

UPDATE 1 – More Listing Options

After some playing with these tools I often started another terminal to check something … not anymore after these updates.

Colored vm list Output

If you have small amount of Bhyve VMs then this feature will not help you much – but the more VMs you have – the more it is useful.

Now only 3 colors are used. Bold white for the headers. Green for the Running machines. Red for the Locked machines.

The thing I did is I wrapped a ‘generic’ vm list output inside vmc.sh script and added colors to it – check for yourself how it looks.

Now compare that to the default vm(8) listing.

Night and day I would say 🙂

List Paused Machines

After I paused any machine I needed to use my other desktop-pause.sh script with -l option to list paused Bhyve VMs.

Now all this is integrated into the bp.sh tool.

Much better.

List VNC Enabled VMs

This last update is the smallest one – still sometimes usable.

While the default vm list output provides that information one may want to just have VNC information – and as we already have a dedicated vnc.sh tool for that – why not implement it here.

Summary

One can confuse vmc.sh with vnc.sh so I will try to make that logic.

The vmc.sh states for VM Listing Colored … and I really did not wanted to use vlc.sh name as someone could take it as some VLC media player wrapper 🙂

The vnc.sh is for VNC connection.

Hope that helps.

UPDATE 2 – Small Useful Addons

While using mine vmc.sh | vnc.sh | bp.sh scripts I found some things that could be improved in them – and this update details these improvements to them.

This is the GitHub commit changelog of these changes.

Different Color for Bootloader State

The vmc.sh – the colored VM list primary task – is to color everything different then Stopped state – so I also added a different color (blue) to color VMs that are in the Bootloader state.

Not sure if useful for everyone – but I found it useful at least several times.

Resume All VMs

While its possible to pause/resume any Bhyve VM – and I sometimes ‘pause’ several VMs that together combine some environment – its PITA to resume them one by one – so I added a -R flag to resume ALL paused VMs – with single command.

Remove All Locks

Useful especially after unexpected reboot or crash. You had several or more VMs perfectly running – but after reboot all of them are locked because the run.lock file was not removed in the process.

For this – I have added -l (for lock) flag – to find and delete run.lock files in the Bhyve VM directory.

For now – the so called VM_DIR is hardlocked in the scripts – set it before using – in the future I plan to make this ‘dynamic’ and being gathered from /etc/rc.conf file or using ‘default’ value.

Check Duplicated MAC Addresses

If you move/migrate VMs between hosts and environments – or clone/copy them – its possible that you will end up with one or more VMs with the same MAC address.

… and while its perfectly possible to just type small UNIX pipe train to make that check – its a lot faster to have that option in vmc.sh command.

Check Duplicated UUID for VMs

While less problematic then duplicated MAC address – I also added a check for duplicated UUID of the Bhyve VMs.

UPDATE 3 – Show VMs Sizes

Another small pack of updates to the vmc.sh script to show size of each virtual machine.

Also to destroy virtual machine status – useful if you needed to kill bhyve(8) process and the machine was still treated as running or present in the FreeBSD system.

Other updates:

• Add -s option to show VMs size.
• Read VM_DIR from /etc/rc.conf file.
• Upgrade removing lock files.
• Add -d option to reset/destroy VM state.
• Add check for doas(1) and sudo(8) existence.

Let me know if any other features would be useful.

Summary

Feel free to share your thoughts about these recent changes.

Regards.

EOF