Re: [RFC Design Doc v2] Add vNVDIMM support for Xen

From: "Jan Beulich" <JBeulich@suse.com>
To: Haozhong Zhang <haozhong.zhang@intel.com>
Cc: Juergen Gross <JGross@suse.com>,
	Kevin Tian <kevin.tian@intel.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Wei Liu <wei.liu2@citrix.com>,
	George Dunlap <George.Dunlap@eu.citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Ian Jackson <ian.jackson@eu.citrix.com>,
	"xen-devel@lists.xen.org" <xen-devel@lists.xen.org>,
	Jun Nakajima <jun.nakajima@intel.com>,
	Xiao Guangrong <guangrong.xiao@linux.intel.com>
Subject: Re: [RFC Design Doc v2] Add vNVDIMM support for Xen
Date: Wed, 03 Aug 2016 02:45:34 -0600	[thread overview]
Message-ID: <57A1CB4E0200007800102110@prv-mh.provo.novell.com> (raw)
In-Reply-To: <20160803065420.fprppr2jhhxl5ehi@hz-desktop>

>>> On 03.08.16 at 08:54, <haozhong.zhang@intel.com> wrote:
> On 08/02/16 08:46, Jan Beulich wrote:
>> >>> On 18.07.16 at 02:29, <haozhong.zhang@intel.com> wrote:
>> >  (4) Because the reserved area is now used by Xen hypervisor, it
>> >      should not be accessible by Dom0 any more. Therefore, if a host
>> >      pmem device is recorded by Xen hypervisor, Xen will unmap its
>> >      reserved area from Dom0. Our design also needs to extend Linux
>> >      NVDIMM driver to "balloon out" the reserved area after it
>> >      successfully reports a pmem device to Xen hypervisor.
>> 
>> ... "balloon out" ... _after_? That'd be unsafe.
>>
> 
> Before ballooning is accomplished, the pmem driver does not create any
> device node under /dev/ and hence no one except the pmem drive can
> access the reserved area on pmem, so I think it's okey to balloon
> after reporting.

Right now Dom0 isn't allowed to access any memory in use by Xen
(and not explicitly shared), and I don't think we should deviate
from that model for pmem.

>> > 4.2.3 Get Host Machine Address (SPA) of Host pmem Files
>> > 
>> >  Before a pmem file is assigned to a domain, we need to know the host
>> >  SPA ranges that are allocated to this file. We do this work in xl.
>> > 
>> >  If a pmem device /dev/pmem0 is given, xl will read
>> >  /sys/block/pmem0/device/{resource,size} respectively for the start
>> >  SPA and size of the pmem device.
>> > 
>> >  If a pre-allocated file /mnt/dax/file is given,
>> >  (1) xl first finds the host pmem device where /mnt/dax/file is. Then
>> >      it uses the method above to get the start SPA of the host pmem
>> >      device.
>> >  (2) xl then uses fiemap ioctl to get the extend mappings of
>> >      /mnt/dax/file, and adds the corresponding physical offsets and
>> >      lengths in each mapping entries to above start SPA to get the SPA
>> >      ranges pre-allocated for this file.
>> 
>> Remind me again: These extents never change, not even across
>> reboot? I think this would be good to be written down here explicitly.
> 
> Yes
> 
>> Hadn't there been talk of using labels to be able to allow a guest to
>> own the exact same physical range again after reboot or guest or
>> host?
> 
> You mean labels in NVDIMM label storage area? As defined in Intel
> NVDIMM Namespace Specification, labels are used to specify
> namespaces. For a pmem interleave set (possible cross several dimms),
> at most one pmem namespace (and hence at most one label) is
> allowed. Therefore, labels can not be used to partition pmem.

Okay. But then how do particular ranges get associated with the
owning guest(s)? Merely by SPA would seem rather fragile to me.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel