From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ingo Molnar Subject: Re: [PATCH 4/6] kvm tools: Add rwlock wrapper Date: Thu, 26 May 2011 22:25:31 +0200 Message-ID: <20110526202531.GA2765@elte.hu> References: <1306419950-19064-1-git-send-email-levinsasha928@gmail.com> <1306419950-19064-4-git-send-email-levinsasha928@gmail.com> <1306426743.3065.34.camel@lappy> <20110526180518.GA3572@elte.hu> <4DDE97CE.4000302@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: QUOTED-PRINTABLE Cc: Avi Kivity , Sasha Levin , john@jfloren.net, kvm@vger.kernel.org, asias.hejun@gmail.com, gorcunov@gmail.com, prasadjoshi124@gmail.com, "Paul E. McKenney" , mathieu.desnoyers@efficios.com To: Pekka Enberg Return-path: Received: from mx2.mail.elte.hu ([157.181.151.9]:42327 "EHLO mx2.mail.elte.hu" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1758192Ab1EZUZq (ORCPT ); Thu, 26 May 2011 16:25:46 -0400 Content-Disposition: inline In-Reply-To: Sender: kvm-owner@vger.kernel.org List-ID: * Pekka Enberg wrote: > On Thu, May 26, 2011 at 9:11 PM, Avi Kivity wrote: > > On 05/26/2011 09:05 PM, Ingo Molnar wrote: > >> > >> > > >> > =A0I've added some rwlocks because of what Ingo said yesterday a= bout > >> > =A0adding/removing devices after the first initialization phase. > >> > > >> > =A0Take MMIO lock for example: Since we can now run SMP guests, = we may > >> > =A0have multiple MMIO exits (one from each VCPU thread). Each of= those > >> > =A0exits leads to searching the MMIO rbtree. > >> > > >> > =A0We can use a mutex to lock it, but it just means that those t= hreads > >> > =A0will be blocked there instead of concurrently searching the M= MIO > >> > =A0tree which makes the search linear instead of parallel. > >> > > >> > =A0It's hard to bring 'real' numbers at this stage because the o= nly > >> > =A0'real' device we have which uses MMIO is the VESA driver, and= we > >> > =A0can't really simulate many VCPUs writing to it :) > >> > >> I'd suggest keeping it simple first - rwlocks are nasty and will > >> bounce a cacheline just as much. > > > > Well, this is the first case where tools/kvm can do better than qem= u with > > its global lock, so I think it's worth it. > > > >> If lookup scalability is an issue we can extend RCU to tools/kvm/. > > > > Definitely rcu is a perfect patch for mmio dispatch. >=20 > Userspace RCU code is here, Sasha, if you feel like tackling this: >=20 > http://lttng.org/urcu >=20 > :-) >=20 > I'm CC'ing Paul and Mathieu as well for urcu. I think we should rather share some of the kernel RCU code in an=20 intelligent way instead of bringing in yet another library which is a=20 IIRC a distant copy of the kernel code to begin with. That way we could lazily benefit from all the enhancements Paul does=20 to the kernel RCU code! :-) Note that kernel/treercu.c is pretty tied to the kernel right now, so=20 a first approach could be to: - Check Paul's excellent documentation about RCU and make sure you don't miss Paul's excellent 3-part primer on LWN.net: http://lwn.net/Articles/262464/ http://lwn.net/Articles/263130/ http://lwn.net/Articles/264090/ There are also lots of very good RCU articles on the LWN Kernel=20 Index page: http://lwn.net/Kernel/Index/ - Check kernel/tinyrcu.c to see how RCU is implemented in its=20 simplest form. :) - Copy the tree-RCU code from kernel/treercu.c to tools/kvm/rcu/ - Massage it so far that it is suitable for tools/kvm/. We really only need a few core RCU facilities initially: struct rcu_head; rcu_read_lock(); rcu_read_unlock(); rcu_dereference() call_rcu(head, func); rcu_synchronize(); The rest, _bh(), etc. are all kernelisms. - Then once it's working we could look at doing the code sharing *for real*: splitting the functionality out of the original treercu.c code into kernel/treercu-lib.c and rcuupdate-lib.h or so and include that one in tools/kvm/rcu/. - [ You might also benefit from porting rcutorture code to=20 user-space. It will catch RCU bugs like nothing else. ] That way the 'core RCU' logic would be contained in treercu-lib.c,=20 all kernel glue would be in kernel/rcutree.c. Some other approach might be possible as well, this was just a first=20 rough idea :) Thanks, Ingo