Update 02.2012 – I got some feedback on this blog post and updated some of the content – but since lot’s of things have changed since then regard this as a snapshot of the back then in Autumn 2011.
I’m still interested in the progress the Hyper-V modules from Microsoft are doing in the mainline Linux kernel. I have continued to test them with each new kernel release as well as following what patches go in. Consider this as a outside reflection at how the drivers evolve from a sysadmin’s perspective. Here are some of the current experiences I made and notable modifications that went upstream I think are worth being mentioned. Due to kernel.org being absent for a couple of weeks, lots of proposed updates have only been merged into the staging tree after kernel.org came back online.
Starting 2.6.39 the state of the drivers have largely improved from being really annoyingly unstable to ‘usable’ – I don’t warrant their stability for or critical use – at least I’ve not seen a single crash on my small webserver running Ubuntu lucid due to these module with custom-built kernels since then. The only notable exception were the modules worked stable were the modules in LIC 2.1 provided by Microsoft for RHEL5 and SLES10.
Since then Microsoft has published LIC 3.1 adding official support for RHEL 6 and CentOS 6 which seem to be based on the codebase in mainline 2.6.39 / 3.0 (when comparing the code) so MS themselves believe the stability is also acceptable for them to publish supported and pre-packaged binaries.
Goodbye /dev/hdX, wellcome /dev/sdX and > 2TB LUN support
If you looked at your Linux virtual machine running on Hyper-V and were a distribution newer than RHEL5, SLES10 or a Kernel older than 2.6.21, you may have realized the following uncommon behaviour for Hyper-V VMs with hv modules: The IDE drives still showed up as /dev/hdX instead of /dev/sdX like almost everywhere else. Additionally – when in paravirtualized mode – the IDE controller was handled by ‘hv_blkvsc’ and the paravirtualized-only SCSI controller by ‘hv_storvsc’. This willl also remain with the release of Linux 3.1.
But with the merge-windows for 3.2 you can expect to see a bunch of changes already present in staging-next that (after a bunch of other patches) will move the handling of IDE drives to hv_storvsc and drops hv_blkvsc completely. Meanwhile a patch addresses the 2TB limitation for those who need it.
Recently before writing this blog entry I have built a 3.1-rc4 kernel out of the staging-next branch and I can confirm the results for storvsc:
sim@sojus-1:~$ uname -a Linux sojus-1 3.1.0-rc4-stagingnext+ #3 SMP Thu Oct 20 14:44:55 CEST 2011 x86_64 GNU/Linux sim@sojus-1:~$ lsmod Module Size Used by xt_multiport 1925 1 iptable_filter 1578 1 ip_tables 17884 1 iptable_filter x_tables 24072 3 xt_multiport,iptable_filter,ip_tables sit 9942 0 tunnel4 2925 1 sit joydev 10705 0 hv_mouse 4193 0 hid 87780 1 hv_mouse psmouse 61215 0 serio_raw 4720 0 hv_timesource 1111 0 [permanent] i2c_piix4 8879 0 floppy 63708 0 hv_storvsc 9152 3 hv_utils 5405 0 hv_netvsc 15931 0 hv_vmbus 27112 4 hv_mouse,hv_storvsc,hv_utils,hv_netvsc,[permanent]
As you can see: No more hv_blkvsc. But I consider hv_storvsc’s behaviour of attaching IDE drives to be a bit weird: Normally the first disk on Linux would be named sda and partitions mounted as /dev/sdaX. This VM has 2 IDE and 1 SCSI disk attached so I’d guess we should have sda to sdc, right?
sim@sojus-1:~$ sudo fdisk -l Disk /dev/sda: 1073 MB, 1073479680 bytes [...] Disk /dev/sdb: 10.7 GB, 10737377280 bytes [...] Disk /dev/sdc: 1073 MB, 1073741824 bytes [...] Disk /dev/sdd: 10.7 GB, 10737418240 bytes [...] Disk /dev/sde: 48.3 GB, 48318382080 bytes
The 2 IDE disks appear twice while the SCSI-attached disks appears once. Since I use LVM on all disks here except for /boot, mount shows /dev/mapper for the root partition etc. but here is how /boot gets mounted:
administrator@sojus-1:~$ mount [...] /dev/sdc1 on /boot type ext4 (rw)
Interesting … albeit I haven’t found a negative impact but still I think this has to be handled differently before storvsc leaves the staging directory.
Update: It seems one can ommit this having only /boot and GRUB on the bootable IDE disk, thanks Victor – he has more details how to omit this.
Actually it’s not a technical problem yet. MS handles this via modprobe rules in their Linux ICs for sanity.
Mouse support (not yet tested)
As some might remember the satori project at Xen provided a semi-open inputvsc module that allowed i.e. RHEL5 to have mouse support as a Hyper-V guest but it was never part of the Microsoft-provided LIC 2.1, 3.1 nor in mainline linux kernel. Now original the Citrix-owned code was renamed to hv_mouse, expanded and merged into staging fully open source and expected to be functional with 3.2. As I don’t use Linux with X on servers, didn’t really care about that, nonetheless it’s greatto have fully open source driver for this component now. If you have tested hv_mouse I’d be happy to know about!
Update: I have since then tested the dirvers, 3.2 seems to work well but 3.3 contains some memory leak bugfixes others have had their issues with them in 3.0/3.1. If you dont have X.org on a Server VM, you can quietly ignore this module anyway. – I’ll let Puppet do the work or SSH to the box first 😉
Starting 3.3 the driver has been renamed once as hid_hyperv and was moved out ot staging.
Partly leaving the staging area
Greg Kroah-Hartmann has put a patch in staging-next that moves the VMBus part out of drivers/staging/hv – read his commit message. It now means the VMBus drivers – the paravirtualized bus of Hyper-V where paravirt devices connect to the hypervisor backend – are considered enough cleaned up to be considered stable.
Update: With Linux 3.3 only storvsc will be / is left in staging area. This means that the kernel hackers think that the overall quality of the kernel modules are believed to be good enough for daily use. For sure there are possible bugs and will be fixed hopefully as well.
While I don’t have an urgent need for dynamic nemory on Linux guests rather than equal performance to Windows guests this feature available to Windows guests starting Hyper-V 2008 R2 SP1 is still missing for Linux guests. I hope Microsoft considers making Linux a first class citizen on Hyper-V – not just good second class. – That said: BSD and other UNIX-based OS’es still lack the support Linux is yet having on Hyper-V.
Update (shortly after this blog post was made)
I have been keen enough to point K.Y on this article (he’s actually the guy doing the most of the work on this code at Microsoft) as he has once gave positive answer to an earlier feedback already. Let me write down what he answered me when asking about the current IDE behaviour:
We are committed to supporting Linux as a first class guest on Hyper-V. What you are currently seeing is what is to be expected: What is happening is that the IDE disks are being discovered by both the native IDE driver in Linux (ata_piix) as well as the storvsc driver. The way we deal with this problem on RHEL6/6.1 and SLES is to have a modprobe rule that ensures that hyperv driver loads first and if the load succeeds, the ata_piix driver will not be loaded. This will result in the IDE disks being discovered only once. […]
Mouse driver is now functional and fully open sourced; I got it working a few weeks ago.
The solution of using the modprobe rules is only the current solution. We are exploring other options to deal with this problem
This sounds great!