From mboxrd@z Thu Jan  1 00:00:00 1970
From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Subject: Re: [PATCH 4/4] hvmloader: add support to load extra
 ACPI tables from qemu
Date: Wed, 20 Jan 2016 12:48:06 -0500
Message-ID: <20160120174805.GA3385@char.us.oracle.com>
References: <569F575902000078000C8EDC@prv-mh.provo.novell.com>
	<20160120110449.GD4939@hz-desktop.sh.intel.com>
	<569F7B8302000078000C8FF8@prv-mh.provo.novell.com>
	<569FA7F3.8080506@linux.intel.com>
	<20160120154749.GD1742@char.us.oracle.com>
	<569FB4E4.4040204@linux.intel.com>
	<20160120164753.GA2497@char.us.oracle.com>
	<569FBC09.80702@linux.intel.com>
	<20160120171823.GE2581@char.us.oracle.com>
	<569FC293.9030904@linux.intel.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Return-path: <xen-devel-bounces@lists.xen.org>
Content-Disposition: inline
In-Reply-To: <569FC293.9030904@linux.intel.com>
List-Unsubscribe: <http://lists.xen.org/cgi-bin/mailman/options/xen-devel>,
	<mailto:xen-devel-request@lists.xen.org?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xen.org>
List-Help: <mailto:xen-devel-request@lists.xen.org?subject=help>
List-Subscribe: <http://lists.xen.org/cgi-bin/mailman/listinfo/xen-devel>,
	<mailto:xen-devel-request@lists.xen.org?subject=subscribe>
Sender: xen-devel-bounces@lists.xen.org
Errors-To: xen-devel-bounces@lists.xen.org
To: Xiao Guangrong <guangrong.xiao@linux.intel.com>
Cc: Haozhong Zhang <haozhong.zhang@intel.com>, Kevin Tian <kevin.tian@intel.com>, feng.wu@intel.com, Ian Campbell <ian.campbell@citrix.com>, Stefano Stabellini <stefano.stabellini@eu.citrix.com>, Andrew Cooper <andrew.cooper3@citrix.com>, Ian Jackson <ian.jackson@eu.citrix.com>, xen-devel@lists.xen.org, Jan Beulich <JBeulich@suse.com>, Jun Nakajima <jun.nakajima@intel.com>, Wei Liu <wei.liu2@citrix.com>, Keir Fraser <keir@xen.org>
List-Id: xen-devel@lists.xenproject.org

On Thu, Jan 21, 2016 at 01:23:31AM +0800, Xiao Guangrong wrote:
> 
> 
> On 01/21/2016 01:18 AM, Konrad Rzeszutek Wilk wrote:
> >>>>>>c) hotplug support
> >>>>>
> >>>>>How does that work? Ah the _DSM will point to the new ACPI NFIT for the OS
> >>>>>to scan. That would require the hypervisor also reading this for it to
> >>>>>update it's data-structures.
> >>>>
> >>>>Similar as you said. The NVDIMM root device in SSDT/DSDT dedicates a new interface,
> >>>>_FIT, which return the new NFIT once new device hotplugged. And yes, domain 0 is
> >>>>the better place handing this case too.
> >>>
> >>>That one is a bit difficult. Both the OS and the hypervisor would need to know about
> >>>this (I think?). dom0 since it gets the ACPI event and needs to process it. Then
> >>>the hypervisor needs to be told so it can slurp it up.
> >>
> >>Can dom0 receive the interrupt triggered by device hotplug? If yes, we can let dom0
> >
> >Yes of course it can.
> >>handle all the things like native. If it can not, dom0 can interpret ACPI and fetch
> >>the irq info out and tell hypervior to pass the irq to dom0, it is doable?
> >>
> >>>
> >>>However I don't know if the hypervisor needs to know all the details of an
> >>>NVDIMM - or just the starting and ending ranges so that when an guest is created
> >>>and the VT-d is constructed - it can be assured that the ranges are valid.
> >>>
> >>>I am not an expert on the P2M code - but I think that would need to be looked
> >>>at to make sure it is OK with stitching an E820_NVDIMM type "MFN" into an guest PFN.
> >>
> >>We do better do not use "E820" as it lacks some advantages of ACPI, such as, NUMA, hotplug,
> >>lable support (namespace)...
> >
> ><hand-waves> I don't know what QEMU does for guests? I naively assumed it would
> >create an E820_NVDIMM along with the ACPI MADT NFIT tables (and the SSDT to have
> >the _DSM).
> 
> Ah, ACPI eliminates this E820 entry.
> 
> >
> >Either way what I think you need to investigate is what is neccessary for the
> >Xen hypervisor VT-d code (IOMMU) to have an entry which is the system address for
> >the NVDIMM. Based on that - you will know what kind of exposure the hypervisor
> >needs to the _FIT and NFIT tables.
> >
> 
> Interesting. I did not consider using NVDIMM as DMA. Do you have usecase for this
> kind of NVDIMM usage?

An easy one is iSCSI target. You could have an SR-IOV NIC that would have TCM
enabled (CONFIG_TCM_FILEIO or CONFIG_TCM_IBLOCK). Mount an file on the /dev/pmem0
(using DAX enabled FS) and export it as iSCSI LUN. The traffic would go over an SR-IOV NIC.

The DMA transactions would be SR-IOV NIC <-> NVDIMM.