Re: [PATCH v12 10/20] dax: Replace XIP documentation with DAX documentation

From: Matthew Wilcox <willy@linux.intel.com>
To: Jared Hulbert <jaredeh@gmail.com>
Cc: "Wilcox, Matthew R" <matthew.r.wilcox@intel.com>,
	Linux FS Devel <linux-fsdevel@vger.kernel.org>,
	LKML <linux-kernel@vger.kernel.org>,
	Linux Memory Management List <linux-mm@kvack.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	Carsten Otte <cotte@de.ibm.com>,
	Chris Brandt <Chris.Brandt@renesas.com>
Subject: Re: [PATCH v12 10/20] dax: Replace XIP documentation with DAX documentation
Date: Mon, 25 Jan 2016 11:52:09 -0500	[thread overview]
Message-ID: <20160125165209.GH2948@linux.intel.com> (raw)
In-Reply-To: <CA+ZsKJ4EMKRgdFQzUjRJOE48=tTJzHf66-60PnVRj7pxvmNgVg@mail.gmail.com>

On Sun, Jan 24, 2016 at 01:03:49AM -0800, Jared Hulbert wrote:
> I our defense we didn't know we were sinning at the time.

Fair enough.  Cache flushing is Hard.

> Can you walk me through the cache flushing hole?  How is it okay on
> X86 but not VIVT archs?  I'm missing something obvious here.
> 
> I thought earlier that vm_insert_mixed() handled the necessary
> flushing.  Is that even the part you are worried about?

No, that part should be fine.  My concern is about write() calls to files
which are also mmaped.  See Documentation/cachetlb.txt around line 229,
starting with "There exists another whole class of cpu cache issues" ...

> What flushing functions would you call if you did have a cache page.

Well, that's the problem; they don't currently exist.

> There are all kinds of cache flushing functions that work without a
> struct page. If nothing else the specialized ASM instructions that do
> the various flushes don't use struct page as a parameter.  This isn't
> the first I've run into the lack of a sane cache API.  Grep for
> inval_cache in the mtd drivers, should have been much easier.  Isn't
> the proper solution to fix update_mmu_cache() or build out a pageless
> cache flushing API?
> 
> I don't get the explicit mapping solution.  What are you mapping
> where?  What addresses would be SHMLBA?  Phys, kernel, userspace?

The problem comes in dax_io() where the kernel stores to an alias of the
user address (or reads from an alias of the user address).  Theoretically,
we should flush user addresses before we read from the kernel's alias,
and flush the kernel's alias after we store to it.

But if we create a new address for the kernel to use which lands on the
same cache line as the user's address (and this is what SHMLBA is used
to indicate), there is no incoherency between the kernel's view and the
user's view.  And no new cache flushing API is needed.

Is that clearer?  I'm not always good at explaining these things in a
way which makes sense to other people :-(