From mboxrd@z Thu Jan 1 00:00:00 1970 From: Tiejun Chen Subject: [RFC][PATCH 00/13] Fix RMRR Date: Fri, 10 Apr 2015 17:21:51 +0800 Message-ID: <1428657724-3498-1-git-send-email-tiejun.chen@intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: JBeulich@suse.com, tim@xen.org, konrad.wilk@oracle.com, andrew.cooper3@citrix.com, kevin.tian@intel.com, yang.z.zhang@intel.com, ian.campbell@citrix.com, wei.liu2@citrix.com, Ian.Jackson@eu.citrix.com, stefano.stabellini@citrix.com Cc: xen-devel@lists.xen.org List-Id: xen-devel@lists.xenproject.org RMRR is an acronym for Reserved Memory Region Reporting, expected to be used for legacy usages (such as USB, UMA Graphics, etc.) requiring reserved memory. Special treatment is required in system software to setup those reserved regions in IOMMU translation structures, otherwise passing through a device with RMRR reported may not work correctly. This patch set tries to enhance existing Xen RMRR implementation to fix various reported and theoretical problems. Most noteworthy changes are to setup identity mapping in p2m layer and handle possible conflicts between reported regions and gfn space. Initial proposal can be found at: http://lists.xenproject.org/archives/html/xen-devel/2015-01/msg00524.html and after a long discussion a summarized agreement is here: http://lists.xen.org/archives/html/xen-devel/2015-01/msg01580.html Below is a key summary of this patch set according to agreed proposal: 1. Use RDM (Reserved Device Memory) name in user space as a general description instead of using ACPI RMRR name directly. 2. Introduce configuration parameters to allow user control both per-device and global RDM resources along with desired policies upon a detected conflict. 3. Introduce a new hypercall to query global and per-device RDM resources. 4. Extend libxl to be a central place to manage RDM resources and handle potential conflicts between reserved regions and gfn space. One simplification goal is made to keep existing lowmem / mmio / highmem layout which is passed around various function blocks. So a reasonable assumption is made, that conflicts falling into below areas are not re-arranged otherwise it will result in a more scattered layout: a) in highmem region (>4G) b) in lowmem region, and below a predefined boundary (default 2G) a) is a new assumption not discussed before. From VT-d spec this is possible but no such observation in real-world. So we can make this reasonable assumption until there's real usage on it. 5. Extend XENMEM_set_memory_map usable for HVM guest, and then have libxl to use that hypercall to carry RDM information to hvmloader. There is one difference from original discussion. Previously we discussed to introduce a new E820 type specifically for RDM entries. After more thought we think it's OK to just tag them as E820_reserved. Actually hvmloader doesn't need to know whether the reserved entries come from RDM or from other purposes. 6. Then in hvmloader the change is generic for XENMEM_memory_map change. Given a predefined memory layout, hvmloader should avoid allocating all reserved entries for other usages (opregion, mmio, etc.) 7. Extend existing device passthrough hypercall to carry conflict handling policy. 8. Setup identity map in p2m layer for RMRRs reported for the given device. And conflicts are handled according to specified policy in hypercall. Current patch set contains core enhancements calling for comments. There are still several tasks not implemented now. We'll include them in final version after RFC is agreed: - remove existing USB hack - detect and fail assigning device which has a shared RMRR with another device - add a config parameter to configure that memory boundary flexibly - In the case of hotplug we also need to figure out a way to fix that policy conflict between the per-pci policy and the global policy but firstly we think we'd better collect some good or correct ideas to step next in RFC. So here I made this as RFC to collect your any comments. ---------------------------------------------------------------- Jan Beulich (1): introduce XENMEM_reserved_device_memory_map Tiejun Chen (12): tools: introduce some new parameters to set rdm policy tools/libxc: Expose new hypercall xc_reserved_device_memory_map tools/libxl: detect and avoid conflicts with RDM xen/x86/p2m: introduce set_identity_p2m_entry xen:vtd: create RMRR mapping xen/passthrough: extend hypercall to support rdm reservation policy tools: extend xc_assign_device() to support rdm reservation policy xen: enable XENMEM_set_memory_map in hvm tools: extend XENMEM_set_memory_map hvmloader: get guest memory map into memory_map[] hvmloader/pci: skip reserved ranges hvmloader/e820: construct guest e820 table docs/man/xl.cfg.pod.5 | 44 +++++ docs/misc/vtd.txt | 34 ++++ tools/firmware/hvmloader/e820.c | 40 ++-- tools/firmware/hvmloader/e820.h | 7 + tools/firmware/hvmloader/hvmloader.c | 36 ++++ tools/firmware/hvmloader/pci.c | 19 +- tools/firmware/hvmloader/util.c | 25 +++ tools/firmware/hvmloader/util.h | 10 + tools/libxc/include/xenctrl.h | 19 +- tools/libxc/include/xenguest.h | 3 +- tools/libxc/xc_domain.c | 80 +++++++- tools/libxc/xc_hvm_build_x86.c | 28 +-- tools/libxl/libxl_create.c | 7 +- tools/libxl/libxl_dm.c | 195 ++++++++++++++++++++ tools/libxl/libxl_dom.c | 99 +++++++++- tools/libxl/libxl_internal.h | 11 +- tools/libxl/libxl_pci.c | 11 +- tools/libxl/libxl_types.idl | 25 +++ tools/libxl/libxlu_pci.c | 78 ++++++++ tools/libxl/libxlutil.h | 4 + tools/libxl/xl_cmdimpl.c | 44 ++++- tools/libxl/xl_cmdtable.c | 2 +- tools/ocaml/libs/xc/xenctrl_stubs.c | 17 +- tools/python/xen/lowlevel/xc/xc.c | 29 ++- xen/arch/x86/hvm/hvm.c | 2 - xen/arch/x86/mm.c | 6 - xen/arch/x86/mm/p2m.c | 30 +++ xen/common/compat/memory.c | 66 +++++++ xen/common/memory.c | 63 +++++++ xen/drivers/passthrough/amd/pci_amd_iommu.c | 3 +- xen/drivers/passthrough/iommu.c | 10 + xen/drivers/passthrough/pci.c | 10 +- xen/drivers/passthrough/vtd/dmar.c | 32 ++++ xen/drivers/passthrough/vtd/extern.h | 1 + xen/drivers/passthrough/vtd/iommu.c | 37 +++- xen/include/asm-x86/p2m.h | 4 + xen/include/public/domctl.h | 4 + xen/include/public/memory.h | 32 +++- xen/include/xen/iommu.h | 6 +- xen/include/xen/pci.h | 2 + xen/include/xlat.lst | 3 +- 41 files changed, 1101 insertions(+), 77 deletions(-) Thanks Tiejun