VirtualBox KVM Public Release

Exactly what I came to comment. Same thing here, they seemed to hyperfixate on the extension pack which most VBox users would need. We had to get everyone off that as soon as possible.

Of course this isn't limited to VBox, but their database as well. Just avoid.

What if you just ignored Oracle? Do they even have teeth? If they made a download free it should be free.

You can't hand someone a banana on the street and then come back 3 months later demanding $1000 for it.

Great! What guests do you typically run where you see better performance with libvirt/kvm?

I thought virt manager was ok but honestly your complaints about it are specific and fair.

virt-manager plays fine with hidpi on Wayland. On the opposite side, VirtualBox’s GUI is super buggy in Wayland.

It’s basically the opposite for both. I use virt-manager because the GUI is simpler (and setting up virtualbox is a nightmare anyway).

Regardless, this feature is a step in the right direction. I’m wondering if distributions will pick it up or if it will ever be integrated upstream.

100% agree about the terrible GUI choices. That actually turned me away completely from libvirt for virtual machines because when I actually needed to work with virtual machines I didn't want to finick around with CLIs and I stuck to VMware until recently. (at the time I ran Windows 10 on my main PC and Windows has a pretty terrible CLI)

Have you tried Gnome Boxes?

Regarding IPv6, there is support. In fact I run IPv6 only networks including for the hypervisors.

    $ virsh net-dumpxml default6
    <network>
      <name>default6</name>
      <uuid>73590ea2-eb15-4e67-b104-319721bdf302</uuid>
      <forward mode='route'/>
      <bridge name='virbr1' stp='on' delay='0'/>
      <mac address='52:54:00:ff:a7:2d'/>
      <domain name='default6'/>
      <ip family='ipv6' address='2001:db8:ffff::1' prefix='48'>
      </ip>
    </network>

One can also use DHCPv6 if required.

If you use a network bridge there's no reason to have to configure any IP addresses at all.

Yeah this has always been a bit of a pain. Since LXD recently launched a web interface that includes graphical console support (plus VM support some time before that, it's not just a container host anymore, it can do both), I have been using that for remote VMs now. But it still has quite a few rough edges, but nicer than libvirt at least.

virt-manager is the kind of software that frustrates me, has lost me VM (due to beraking SPICE beyond it already having been broken for years). Bug reports go ignored. Yet, when it works, it works great. And KVM is great. But Christ, it's a nightmare sometimes. Features lag behind libvirt, sometimes years. Just not great. The virt-viewer port to Windows never made it past POC+1, IMO, also.

I'm honestly surprised there aren't more alternatives. Especially since SPICE has gstreamer instegration. Gstreamer can natively do webrtc video and data back channel. Someone could hack together a webrtc broweser-based VM viewer and I suspect there's a lot of interest, potentially money making things you could crate.

There is experimental support for VFIO PCI pass-through with VirtualBox-KVM, even for GPUs. Please have a look at https://www.cyberus-technology.de/products/hypervisor

Please evaluate the license of the extension package carefully before using!

I cannot even imagine using a distro that can't manage to get DKMS/kernel module updates to be reliable. I don't think I've ever had a problem with DKMS in NixOS ever; not a single time.

edit: downvoting me won't make your distro any more competent.

Doing this for FreeBSD would be a great project. Unfortunately it is also big enough that we couldn’t afford to do it without some kind of funding.

A decent UI and hopefully support to use virtualbox appliances

The out-of-the-box performance of Windows in VirtualBox is very good and usually better than virt-manager (Qemu). You can tune Qemu to great performance as well, but it takes some fiddling. VirtualBox is in general very user friendly.

Guest integration (drag'n'drop, clipboard), USB passhthrough and audio support is also top-notch in VBox.

I virtualize most of my desktop environment. I wanted to go with KVM and virt-manager initially, since I'm mostly using a Linux host and Linux guests, but there were two important features I wanted and couldn't figure out how to get that way: encryption and portability.

Most of the VMs are encrypted, so I feel safe traveling with them. Various secrets are also encrypted, but the encryption of the VMs themselves mean that I don't have to worry about losing my device at an airport and someone else potentially getting access to things they shouldn't. There are schemes that make this work in virt-manager and KVM, but I didn't like any of them as much; I didn't want to rely on the host for filesystem-level encryption (see portability), and I have previously had a bit of trouble with full disk encryption, so I wasn't comfortable relying on that. VirtualBox essentially is also doing full disk encryption, but it's invisible to the guest and seems to be reliable.

For portability, I should be able to use https://www.vbox.me/ to install the VMs and a host onto a flash drive and be able to run any of my environments from any Windows host without additional installations. Haven't actually tried this yet (happily, I no longer have easy access to Windows machines!), but it was a big point in favor.

Most of my environments now get auto-configured through Vagrant: https://github.com/robsheldon/vagrantfiles, so I get some of the benefits of virt-manager that way.

I really don't love relying on Oracle for anything

https://linux-kvm.org/page/Main_Page

It means Kernel-Based Virtual Machine, a VM engine dating back to 2007: https://en.wikipedia.org/wiki/Kernel-based_Virtual_Machine

Yes, quite different. It's basically a project that allows you to use Linux as a hypervisor. A very popular project.

https://en.m.wikipedia.org/wiki/Kernel-based_Virtual_Machine

Happy 10000 Day!

https://xkcd.com/1053/

While an over simplification, here is the context

There are two large hypervisors in the Linux world.

Xen, which extends the kernel to support virtual CPUs with time slices.

KVM, which assigns each virtual core a process that uses the Linux scheduler.

When a hardware vm vcpu core is preempted there is vmexit call that has to reset registers etc... and it is expensive.

Xen is what legacy AWS instances ran on and has advantages for being fair to guests is an easier task.

KVM has the advantage of gaining the benefits of the Linux scheduler which is red black tree based and well optimized.

When a new CPU comes out for example, KVM gains support from the upstream while Xen has to support it themselves.

Once technology like cgroups improved the benefits of letting your thread complete and not be preempted due to the time slice expiring avoided the cost of vmexit.

In theory, leveraging the inherently optimized core Linux features is what will also benefit virtualbox.

Most people who use KVM are using an abstraction layer like libvirt that hides how it is implemented.

In fact if you look at the processes you will see qemu even if KVM is how it is implemented.

It depends on your setup and workload. On a recent Intel CPU, our performance dashboard shows +10% for some benchmarks. It's hard to make a general statement though.

If you don't mind, I have a specific question regarding this setup. I've been looking into getting into pentesting, mostly for fun. I decided on messing around with HackTheBox as a starting point. Seeing as you need to connect to their VPN, it seems like a good idea to me to separate this activity from my personal network.

Which networking setup do you use for your pentesting VM? Ideally, I'd want a setup where the VM can access the internet (and therefore the HTB VPN), but not anything inside my local network. But I don't quite know how I could achieve that, at least in a way where I'd trust it to be reliable. Maybe the whole idea's a bit too paranoid to be practical in general, I don't know, so I'd love an expert opinion on this :)

Usually, I'd be using QEMU, but I'd be fine with using VirtualBox for this case if it includes something that makes this easier.

Thank you!

Yes, this is indeed nice. I have a Chromebook as well and the integration with Linux apps is super seamless. The major usecase for VBox is mostly Windows, though.

The emulated hardware is basically the same as with stock VirtualBox. Only the interrupt contoller (local APIC) is emulated by KVM.

This only affects anyone that wants to spin up VirtualBox VM's on Linux hosts.

I'm not an expert in this field, but my best TL;DR is that VirtualBox and other VMM's (virtual machine monitor) used to ship with their own hypervisors (the thing that let's you run virtual machines). However, now Linux has its own hypervisor/framework (KVM) and now VirtualBox can use KVM to do all the functionalities their own hypervisor used to do.

Someone please correct me if I'm wrong

What you can configure in the GUI is an enlightenment that the guest will see. This is usually done to improve guest performance.

You can sill select these PV interfaces with VirtualBox-KVM, but the underlying hypervisor is different (kvm vs vboxdrv).

The paravirtualization option affects the interface presented for the guest operating system for dealing with being virtualized.

This new code is about using different virtualization technology on the host system.

The name KVM is a bit confusing. It doesn't do anything fundamental different than VirtualBox. The difference is that KVM comes by default with any Linux. VirtualBox ships it's own Linux kernel module for that. That has drawbacks. You typically can't use the newest Linux or the newest features.

KVM also always has the hottest new (performance-relevant) features, because Intel and AMD will always build their hot stuff into KVM first.

You can’t use virtualbox’s kernel module and kvm at the same time. This basically means that you can’t use virtualbox and qemu at the same time.

If you use virtualbox with this new backend, you can use it concurrently with qemu (and a few other virtualisation tools).

KVM is also part of Linux itself, so there’s a lot less haste with setting it up.

I wouldn't call it kernel based. It's not like this is an in-kernel emulation. I would stick with hardware virtualization.

KVM is a userspace API, and kvm-intel/kvm-amd are the drivers for the hardware.

You will be using hardware features. That's also why it is in the kernel: nothing but the kernel should have full unlimited access to the CPU to set this up.

So you could say it must be in kernel to keep the kernel secure. And the performance benefit is "just" exposed hardware features.

The kernel does not provide additional things. As far as I understand: you set up a dedicated memory space and handle traps that halt the execution e.g. when the VM talks to the PCI bus. (It's been a while since I looked this up)

But you need the pieces, especially virtual PCI devices. That's where qemu or VirtualBox enter the scene (or minimalist systems like firecracker). They provide a repository of virtual hardware and all the auxiliary methods to boot a virtual machines. You also need to emulate something like a BIOS or UEFI.

You can think of it as your CPU removing the need to emulate the very same CPU (and a memory controller). You still need to emulate the rest though! But running on the same CPU removes most performance penalties. You run at native speed.

Newer generations can even nest this. Having virtual machines in virtual machines. That's mostly useful for cloud environments so that the cloud provider can run kvm based VMs and you are still able to run VMs inside that VM.

>How does this work in licensing terms? If VB foss enough?

It's as FOSS as the VirtualBox open source edition.

> Do you expect Oracle to merge this?

That would be nice, but I wouldn't hold my breath. Oracle gonna Oracle.

> If oracle doesn’t merge this, will you keep on maintaining it, potentially forking VirtualBox?

We don't intend to fork VirtualBox. VBox has a somewhat modular architecture where you can plug-in different hypervisor backends. That's what we did. It's not as modular, but our changes to core VirtualBox code is very small.

As far as our plans go, we are pretty open at this point. We are very interested to get to know people that find this useful!

Yes.

Thanks. You can indeed use the extension pack with KVM Virtualbox, just make sure that you don't violate any license agreements.

We're offering commercial support. We can also help with graphics virtualization and other topics (e.g. performance tuning and automated testing in real world scenarios).

Those are good questions. I don’t understand why you were downvoted.

To answer: We are offering service contracts and contract engineering services around virtualization, KVM and a couple of other topics.

The long term goal for this specific project is to support the security goals our customers have and to enable a couple of KVM features in Virtualbox as well. We plan to keep this updated with upstream Virtualbox and KVM.

It seems to be a fork of VirtualBox under the same dual license as the original project.

https://github.com/cyberus-technology/virtualbox-kvm/blob/de...

I mean it's Oracle we're talking about here.

Oracle already had an unfinished and broken KVM backend in the code that was not exposed. Whether they incorporate this polished KVM backend is anyone's guess at this point.

I’d actually much, much rather see it set up as a proper fork (rebranded etc.), and then the features that Oracle extorts people with in the “free” but not actually free extension pack (like USB pass through) re-implemented and included directly with no ‘extension’ required.

Much of the reason I refuse to use Virtualbox for anything is how scummy Oracle is.

The KVM backend doesn't have nesting enabled just yet. We're on it.

Yes, we are switching between vanilla VBox and KVM VBox during development quite often and the VMs are fine with it.

Hyper-V is a good option for VM servers but AFAIK it doesn't do user-friendly stuff like USB or graphics card pass-thru.

Yes, I recently had to compile some stuff on Windows (I'm on an AMD Linux host) and VirtualBox just wouldn't start Microsoft's Windows dev VM (the one they provide for free for Virtualbox). I ended up learning how to use qemu and it works great...and as a bonus I was able to run a hackintosh (via https://github.com/kholia/OSX-KVM) and it works near flawlessly, which was something I was never able to accomplish with Virtualbox (granted I haven't tried in a few years).

I'm pretty happy with Qemu now, even if it's jsut a CLI interface. I was tempted to try the virt-* stuff, but honestly it seems like one more thing to learn so I'm going to hold off until I need something like copy/paste between VMs and can't figure it out in qemu direct.

Is bridged networking easy to setup? It's the main reason i use VirtualBox.

Yes. Don't expect too much though. 3D acceleration with Virtualbox is rather flaky (independent of using KVM).

It does.

I do all of those things daily with virt-manager already. Except multiple monitors, although I believe it is supported.

Well, KVM is used by Google and AWS and others for their clouds. As such, there are a lot of eyes on KVM code. The vboxdrv kernel module that provides the same functionality in vanilla VBox definitely has fewer people looking at it. It also has anti-features, such as code upload from the userspace VirtualBox process to the kernel. This is also the largest security issue with vanilla VBox, because a lot of emulation code runs directly in the kernel.

From a performance perspective, it's a bit more complicated. KVM has support for modern virtualization features (Intel APICv, AMD AVIC, etc) that vanilla VBox lacks. You get these in the VirtualBox/KVM version. On the other hand, vanilla VBox emulates most devices in the kernel (see above). So SATA emulation in vanilla VBox is very fast compared to KVM/Qemu or KVM/VirtualBox for a bit unfair reasons. Modern devices, such as virtio or NVMe, are not as impacted by that.

tl;dr So the performance you get depends on your workload. If it's very interrupt heavy, VirtualBox/KVM will win. If it uses antiquated virtual devices (SATA), vanilla VirtualBox (with vboxdrv) will have an edge.

KVM is the one built into Linux. VBox has it's own module for hardware virtualization all along

Crappy? Windows 7 was the last good Windows.

Ooops. Will fix. :) Thanks!

Not yet. Nesting support is on our list. But the performance will not be great.

A lot of this has changed in recent VirtualBox versions. USB3 and Webcam support is now part of the OSE release.