Re: [PATCH RFC 0/4] tools: Upstream blktap "2.5" as an external repo

[Felipe Franciosi <felipe.franciosi@citrix.com>]

> -----Original Message-----
> From: xen-devel-bounces@lists.xen.org [mailto:xen-devel-
> bounces@lists.xen.org] On Behalf Of George Dunlap
> Sent: 26 March 2015 16:09
> To: Jonathan Ludlam
> Cc: Wei Liu; Dave Scott; Wen Congyang; Jonathan Ludlam; xen-
> devel@lists.xen.org; Ian Jackson; Yang Hongyang; Ian Campbell
> Subject: Re: [Xen-devel] [PATCH RFC 0/4] tools: Upstream blktap "2.5" as an
> external repo
> 
> On 03/26/2015 03:47 PM, Jon Ludlam wrote:
> > On Thu, Mar 26, 2015 at 12:46:06PM +0000, George Dunlap wrote:
> >> For some time, the blktap2 in-tree has bitrotted.  Many years ago the
> >> XenServer team at Citrix forked the code into a separate repository;
> >> several attempts have been made to upstream those changes back into
> >> Xen, to no avail.
> >>
> >> The blktap code at the moment is the only source of performant vhd
> >> format integration.  It's additionally in use by projects like the
> >> COLO project.
> >>
> >> This patch series removes the in-tree blktap2 code and treats the
> >> XenServer blktap tree as an upstream.  I've gotten agreement from the
> >> XenServer team to act as an upstream -- to accept patches fixing
> >> bugs, to help track down errors, and to attempt to help fix build
> >> breakages introduced by development.
> >>
> >> At the moment we're using the "blktap2" branch of XenServer's
> >> blktap.git.  (This has been sometimes known as "blktap 2.5".)  This
> >> branch is maintianed in order to provide a buildroot for OpenStack,
> >> and has also been used by the CentOS xen packages for several years
> >> now.
> >
> > It's probably worth mentioning again that there is a kernel patch
> > required. Some years ago I did some work to make the patch into a dkms
> > module, but since then the patch and the kernel have moved on and I
> > couldn't quite make it work any more; I'm afraid my kernel knowledge
> > is a bit lacking.
> >
> > The current patches used in XenServer are on github here:
> > https://github.com/xenserver/linux-3.x.pg/tree/master/master
> >
> > and the old dkms code is here:
> > https://github.com/xapi-project/blktap-dkms
> >
> > In case anyone is interested...!
> 
> Another thing I'd like to explore (since this took all of about an afternoon to get
> working) is what it would take to switch to using
> blktap3 instead.  As I understand from my conversations with the XenServer
> team, they use a kernel module in XenServer when mounting an image in dom0
> for scalability reasons; but there's no reason XenProject need to do the same
> thing.

Today, to access virtual disks in dom0 we use the blktap2 kernel module that Jonathan Ludlam mentioned. This provides a block device (in /dev/xen/blktap-2/tapdev<minor>). In the (somewhat distant) past, we actually had blkfront in dom0.

> I'd probably need some guidance from the XenServer team about an
> appropriate way to start that. :-)

What is also needed to get blktap3 working is a gntdev supporting grant copy. I believe this is the order they are applied in 3.10:
https://github.com/xenserver/linux-3.x.pg/blob/master/master/0001-xen-install-xen-gntdev.h-and-xen-gntalloc.h.patch
https://github.com/xenserver/linux-3.x.pg/blob/master/master/xen-gntdev-grant-copy.patch
https://github.com/xenserver/linux-3.x.pg/blob/master/master/0002-xen-gntdev-mark-userspace-PTEs-as-special-on-x86-PV-.patch
https://github.com/xenserver/linux-3.x.pg/blob/master/master/0003-xen-gntdev-provide-a-set-of-pages-for-the-VMA.patch

The _main_ difference between tapdisk2 and 3 is that tapdisk3 can connect directly to blkfront. It does all I/O via grant copies which has some implications in the way memory is handled in dom0.

Cheers,
Felipe



> 
>  -George
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel