State of Linux on Hyper-V

Update 02.2012 – I got some feedback on this blog post and updated some of the content – but since lot’s of things have changed since then regard this as a snapshot of the back then in Autumn 2011.

I’m still interested in the progress the Hyper-V modules from Microsoft are doing in the mainline Linux kernel. I have continued to test them with each new kernel release  as well as following what patches go in. Consider this as a outside reflection at how the drivers evolve from a sysadmin’s perspective. Here are some of the current experiences I made and notable modifications that went upstream I think are worth being mentioned. Due to being absent for a couple of weeks, lots of proposed updates have only been merged into the staging tree after came back online.

Overall stability

Starting 2.6.39 the state of the drivers have largely improved from being really annoyingly unstable to ‘usable’ – I don’t warrant their stability for or critical use –  at least I’ve not seen a single crash on my small webserver running Ubuntu lucid due to these module with custom-built kernels since then. The only notable exception were the modules worked stable  were the modules in LIC 2.1 provided by Microsoft for RHEL5 and SLES10.

Since then Microsoft has published LIC 3.1 adding official support for RHEL 6 and CentOS 6 which seem to be based on the codebase in mainline 2.6.39 / 3.0 (when comparing the code) so MS themselves believe the stability is also acceptable for them to publish supported and pre-packaged binaries.

Goodbye /dev/hdX, wellcome /dev/sdX and > 2TB  LUN support

If you looked at your Linux virtual machine running on Hyper-V and were a distribution newer than RHEL5, SLES10 or a Kernel older than 2.6.21, you may have realized the following uncommon behaviour for Hyper-V VMs with hv modules: The IDE drives still showed up as /dev/hdX instead of /dev/sdX like almost everywhere else. Additionally – when in paravirtualized mode – the IDE controller was handled by ‘hv_blkvsc’ and the paravirtualized-only SCSI controller by ‘hv_storvsc’. This willl also remain with the release of Linux 3.1.

But with the merge-windows for 3.2 you can expect to see a bunch of changes already present in staging-next that (after a bunch of other patches) will move the handling of IDE drives to hv_storvsc and drops hv_blkvsc completely. Meanwhile a patch addresses the 2TB limitation for those who need it.

Recently before writing this blog  entry I have built a 3.1-rc4 kernel out of the staging-next branch and I can confirm the results for storvsc:

sim@sojus-1:~$ uname -a
Linux sojus-1 3.1.0-rc4-stagingnext+ #3 SMP Thu Oct 20 14:44:55 CEST 2011 x86_64 GNU/Linux

sim@sojus-1:~$ lsmod
Module                  Size  Used by
xt_multiport            1925  1
iptable_filter          1578  1
ip_tables              17884  1 iptable_filter
x_tables               24072  3 xt_multiport,iptable_filter,ip_tables
sit                     9942  0
tunnel4                 2925  1 sit
joydev                 10705  0
hv_mouse                4193  0
hid                    87780  1 hv_mouse
psmouse                61215  0
serio_raw               4720  0
hv_timesource           1111  0 [permanent]
i2c_piix4               8879  0
floppy                 63708  0
hv_storvsc              9152  3
hv_utils                5405  0
hv_netvsc              15931  0
hv_vmbus               27112  4 hv_mouse,hv_storvsc,hv_utils,hv_netvsc,[permanent]

As you can see: No more hv_blkvsc. But  I consider hv_storvsc’s behaviour of attaching IDE drives to be a bit weird: Normally the first disk on Linux would be named sda and partitions mounted as /dev/sdaX. This VM has 2 IDE and 1 SCSI disk attached so I’d guess we should have sda to sdc, right?

sim@sojus-1:~$ sudo fdisk -l

Disk /dev/sda: 1073 MB, 1073479680 bytes

Disk /dev/sdb: 10.7 GB, 10737377280 bytes

Disk /dev/sdc: 1073 MB, 1073741824 bytes

Disk /dev/sdd: 10.7 GB, 10737418240 bytes

Disk /dev/sde: 48.3 GB, 48318382080 bytes

The 2 IDE disks appear twice while the SCSI-attached disks appears once. Since I use LVM on all disks here  except for /boot,  mount shows /dev/mapper for the root partition etc. but here is how  /boot gets mounted:

administrator@sojus-1:~$ mount
/dev/sdc1 on /boot type ext4 (rw)

Interesting … albeit I  haven’t found a negative impact but still I think this has to be handled differently before storvsc leaves the staging directory.

Update: It seems one can ommit this having only /boot and GRUB on the bootable IDE disk, thanks Victor – he has more details how to omit this.
Actually it’s not a technical problem yet. MS handles this via modprobe rules in their Linux ICs for sanity.

Mouse support (not yet tested)

As some might remember the satori project  at Xen provided a semi-open inputvsc module that allowed i.e. RHEL5 to have mouse support as a Hyper-V guest but it was never part of the Microsoft-provided LIC 2.1, 3.1 nor in mainline linux kernel. Now original the Citrix-owned code was renamed to hv_mouse, expanded and merged into staging fully open source and expected to be functional with 3.2. As I don’t use Linux with X on servers, didn’t really care about that, nonetheless it’s greatto have fully open source driver for this component now. If you have tested hv_mouse I’d be happy to know about!

Update: I have since then tested the dirvers, 3.2 seems to work well but 3.3 contains some memory leak bugfixes others have had their issues with them in 3.0/3.1. If you dont have on a Server VM, you can quietly ignore this module anyway. – I’ll let  Puppet  do the work or SSH to the box first 😉

Starting 3.3 the driver has been renamed once as hid_hyperv and was moved out ot staging.

Partly leaving the staging area

Greg Kroah-Hartmann has put a patch in staging-next that moves the VMBus part out of drivers/staging/hv  – read his commit message. It now means  the VMBus drivers – the paravirtualized bus of Hyper-V where paravirt devices connect to the hypervisor backend  – are considered enough cleaned up to be considered stable.

Update: With Linux 3.3 only storvsc will be / is left in staging area. This means that the kernel hackers think that the overall quality of the kernel modules are believed to be good enough for daily use. For sure there are possible bugs and will be fixed hopefully as well.

Missing features

While I don’t have an urgent need for dynamic nemory on Linux guests rather than equal performance  to Windows guests this feature available to Windows guests starting Hyper-V 2008 R2 SP1 is still missing for Linux guests. I hope Microsoft considers making Linux a first class citizen on Hyper-V – not just good second class. – That said: BSD and other UNIX-based OS’es still lack the support Linux is yet having on Hyper-V.

Update (shortly after this blog post was made)

I have been keen enough to point K.Y on this article (he’s actually the guy doing the most of the work on this code at Microsoft)  as he has once gave positive answer to an earlier feedback already. Let me write down what he answered me when asking about the current IDE behaviour:

We are committed to supporting Linux as a first class guest on Hyper-V. What you are currently  seeing is what is to be expected: What is happening is that the IDE disks are being discovered by both the native IDE driver in Linux (ata_piix) as well as the storvsc driver. The way we deal with this problem on RHEL6/6.1 and SLES is to have a modprobe rule that ensures that hyperv driver loads first and if the load succeeds, the ata_piix driver will not be loaded. This will result in the IDE disks being discovered only once. […]

Mouse driver is now functional and fully open sourced; I got it working a few weeks ago.

Later on:

The solution of using the modprobe rules is only the current solution. We are exploring other options to deal with this problem

This sounds great!

October 22, 2011

Posted In: Uncategorized


One Comment