Re: [RFC XEN PATCH v2 00/15] Add vNVDIMM support to HVM domains

From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
To: Dan Williams <dan.j.williams@intel.com>
Cc: Wei Liu <wei.liu2@citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Ian Jackson <ian.jackson@eu.citrix.com>,
	xen-devel@lists.xen.org, Jan Beulich <jbeulich@suse.com>,
	Konrad Rzeszutek Wilk <konrad@darnok.org>,
	Daniel De Graaf <dgdegra@tycho.nsa.gov>
Subject: Re: [RFC XEN PATCH v2 00/15] Add vNVDIMM support to HVM domains
Date: Tue, 4 Apr 2017 13:00:10 -0400	[thread overview]
Message-ID: <20170404170010.GC24507@char.us.oracle.com> (raw)
In-Reply-To: <CAPcyv4gm618KBqy6hZEEk9aP6hy_cZDhvR5=Tnd-tTci3HnDWw@mail.gmail.com>

On Sat, Apr 01, 2017 at 08:45:45AM -0700, Dan Williams wrote:
> On Sat, Apr 1, 2017 at 4:54 AM, Konrad Rzeszutek Wilk <konrad@darnok.org> wrote:
> > ..snip..
> >> >> Is there a resource I can read more about why the hypervisor needs to
> >> >> have this M2P mapping for nvdimm support?
> >> >
> >> > M2P is basically an array of frame numbers. It's indexed by the host
> >> > page frame number, or the machine frame number (MFN) in Xen's
> >> > definition. The n'th entry records the guest page frame number that is
> >> > mapped to MFN n. M2P is one of the core data structures used in Xen
> >> > memory management, and is used to convert MFN to guest PFN. A
> >> > read-only version of M2P is also exposed as part of ABI to guest. In
> >> > the previous design discussion, we decided to put the management of
> >> > NVDIMM in the existing Xen memory management as much as possible, so
> >> > we need to build M2P for NVDIMM as well.
> >> >
> >>
> >> Thanks, but what I don't understand is why this M2P lookup is needed?
> >
> > Xen uses it to construct the EPT page tables for the guests.
> >
> >> Does Xen establish this metadata for PCI mmio ranges as well? What Xen
> >
> > It doesn't have that (M2P) for PCI MMIO ranges. For those it has an
> > ranges construct (since those are usually contingous and given
> > in ranges to a guest).
> 
> So, I'm confused again. This patchset / enabling requires both M2P and
> contiguous PMEM ranges. If the PMEM is contiguous it seems you don't
> need M2P and can just reuse the MMIO enabling, or am I missing
> something?

I think I am confusing you.

The patchset (specifically 04/15] xen/x86: add XEN_SYSCTL_nvdimm_pmem_setup to setup host pmem )
adds a hypercall to tell Xen where on the NVDIMM it can put 
the M2P array and as well the frametables ('struct page').

There is no range support. The reason is that if break up
an NVDIMM in various chunks (and then put a filesystem on top of it) - and
then figure out which of the SPAs belong to the file - and then
"expose" that file to a guest as NVDIMM - it's SPAs won't
be contingous. Hence the hypervisor would need to break down
the 'ranges' structure down in either a bitmap or an M2P
and also store it. This can get quite tricky so you may
as well just start with an M2P and 'struct page'.

The placement of those datastructures is "
v2 patch
   series relies on users/admins in Dom0 instead of Dom0 driver to indicate the
   location to store the frametable and M2P of pmem.
"

Hope this helps?
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://lists.xen.org/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel