Re: [PATCH 08 of 11] anon-vma-rwsem

From: Nick Piggin <npiggin@suse.de>
To: Robin Holt <holt@sgi.com>
Cc: Christoph Lameter <clameter@sgi.com>,
	Nick Piggin <nickpiggin@yahoo.com.au>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Andrea Arcangeli <andrea@qumranet.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Jack Steiner <steiner@sgi.com>,
	Peter Zijlstra <a.p.zijlstra@chello.nl>,
	kvm-devel@lists.sourceforge.net,
	Kanoj Sarcar <kanojsarcar@yahoo.com>,
	Roland Dreier <rdreier@cisco.com>,
	Steve Wise <swise@opengridcomputing.com>,
	linux-kernel@vger.kernel.org, Avi Kivity <avi@qumranet.com>,
	linux-mm@kvack.org, general@lists.openfabrics.org,
	Hugh Dickins <hugh@veritas.com>,
	Rusty Russell <rusty@rustcorp.com.au>,
	Anthony Liguori <aliguori@us.ibm.com>,
	Chris Wright <chrisw@redhat.com>,
	Marcelo Tosatti <marcelo@kvack.org>,
	Eric Dumazet <dada1@cosmosbay.com>,
	"Paul E. McKenney" <paulmck@us.ibm.com>
Subject: Re: [PATCH 08 of 11] anon-vma-rwsem
Date: Tue, 20 May 2008 13:14:24 +0200	[thread overview]
Message-ID: <20080520111424.GB25791@wotan.suse.de> (raw)
In-Reply-To: <20080520110528.GD30341@sgi.com>

On Tue, May 20, 2008 at 06:05:29AM -0500, Robin Holt wrote:
> On Tue, May 20, 2008 at 12:50:25PM +0200, Nick Piggin wrote:
> > On Tue, May 20, 2008 at 05:01:11AM -0500, Robin Holt wrote:
> > > On Tue, May 20, 2008 at 07:31:46AM +0200, Nick Piggin wrote:
> > > > 
> > > > Really? You can get the information through via a sleeping messaging API,
> > > > but not a non-sleeping one? What is the difference from the hardware POV?
> > > 
> > > That was covered in the early very long discussion about 28 seconds.
> > > The read timeout for the BTE is 28 seconds and it automatically retried
> > > for certain failures.  In interrupt context, that is 56 seconds without
> > > any subsequent interrupts of that or lower priority.
> > 
> > I thought you said it would be possible to get the required invalidate
> > information without using the BTE. Couldn't you use XPMEM pages in
> > the kernel to read the data out of, if nothing else?
> 
> I was wrong about that.  I thought it was safe to do an uncached write,
> but it turns out any processor write is uncontained and the MCA that
> surfaces would be fatal.  Likewise for the uncached read.

Oh, so the BTE transfer is purely for fault isolation. I was thinking
you guys might have sufficient control of the hardware to be able to
do it at the level of CPU memory operations, but if it is some
limitation of ia64, then I guess that's a problem.

How do you do fault isolation of userspace XPMEM accesses?