From: Tiejun Chen <tiejun.chen@intel.com>
To: JBeulich@suse.com, tim@xen.org, konrad.wilk@oracle.com,
andrew.cooper3@citrix.com, kevin.tian@intel.com,
yang.z.zhang@intel.com, ian.campbell@citrix.com,
wei.liu2@citrix.com, Ian.Jackson@eu.citrix.com,
stefano.stabellini@citrix.com
Cc: xen-devel@lists.xen.org
Subject: [RFC][PATCH 00/13] Fix RMRR
Date: Fri, 10 Apr 2015 17:21:51 +0800 [thread overview]
Message-ID: <1428657724-3498-1-git-send-email-tiejun.chen@intel.com> (raw)
RMRR is an acronym for Reserved Memory Region Reporting, expected to
be used for legacy usages (such as USB, UMA Graphics, etc.) requiring
reserved memory. Special treatment is required in system software to
setup those reserved regions in IOMMU translation structures, otherwise
passing through a device with RMRR reported may not work correctly.
This patch set tries to enhance existing Xen RMRR implementation to fix
various reported and theoretical problems. Most noteworthy changes are
to setup identity mapping in p2m layer and handle possible conflicts between
reported regions and gfn space. Initial proposal can be found at:
http://lists.xenproject.org/archives/html/xen-devel/2015-01/msg00524.html
and after a long discussion a summarized agreement is here:
http://lists.xen.org/archives/html/xen-devel/2015-01/msg01580.html
Below is a key summary of this patch set according to agreed proposal:
1. Use RDM (Reserved Device Memory) name in user space as a general
description instead of using ACPI RMRR name directly.
2. Introduce configuration parameters to allow user control both per-device
and global RDM resources along with desired policies upon a detected conflict.
3. Introduce a new hypercall to query global and per-device RDM resources.
4. Extend libxl to be a central place to manage RDM resources and handle
potential conflicts between reserved regions and gfn space. One simplification
goal is made to keep existing lowmem / mmio / highmem layout which is
passed around various function blocks. So a reasonable assumption
is made, that conflicts falling into below areas are not re-arranged otherwise
it will result in a more scattered layout:
a) in highmem region (>4G)
b) in lowmem region, and below a predefined boundary (default 2G)
a) is a new assumption not discussed before. From VT-d spec this is
possible but no such observation in real-world. So we can make this
reasonable assumption until there's real usage on it.
5. Extend XENMEM_set_memory_map usable for HVM guest, and then have
libxl to use that hypercall to carry RDM information to hvmloader. There
is one difference from original discussion. Previously we discussed to
introduce a new E820 type specifically for RDM entries. After more thought
we think it's OK to just tag them as E820_reserved. Actually hvmloader
doesn't need to know whether the reserved entries come from RDM or
from other purposes.
6. Then in hvmloader the change is generic for XENMEM_memory_map
change. Given a predefined memory layout, hvmloader should avoid
allocating all reserved entries for other usages (opregion, mmio, etc.)
7. Extend existing device passthrough hypercall to carry conflict handling
policy.
8. Setup identity map in p2m layer for RMRRs reported for the given
device. And conflicts are handled according to specified policy in hypercall.
Current patch set contains core enhancements calling for comments.
There are still several tasks not implemented now. We'll include them
in final version after RFC is agreed:
- remove existing USB hack
- detect and fail assigning device which has a shared RMRR with another device
- add a config parameter to configure that memory boundary flexibly
- In the case of hotplug we also need to figure out a way to fix that policy
conflict between the per-pci policy and the global policy but firstly we think
we'd better collect some good or correct ideas to step next in RFC.
So here I made this as RFC to collect your any comments.
----------------------------------------------------------------
Jan Beulich (1):
introduce XENMEM_reserved_device_memory_map
Tiejun Chen (12):
tools: introduce some new parameters to set rdm policy
tools/libxc: Expose new hypercall xc_reserved_device_memory_map
tools/libxl: detect and avoid conflicts with RDM
xen/x86/p2m: introduce set_identity_p2m_entry
xen:vtd: create RMRR mapping
xen/passthrough: extend hypercall to support rdm reservation policy
tools: extend xc_assign_device() to support rdm reservation policy
xen: enable XENMEM_set_memory_map in hvm
tools: extend XENMEM_set_memory_map
hvmloader: get guest memory map into memory_map[]
hvmloader/pci: skip reserved ranges
hvmloader/e820: construct guest e820 table
docs/man/xl.cfg.pod.5 | 44 +++++
docs/misc/vtd.txt | 34 ++++
tools/firmware/hvmloader/e820.c | 40 ++--
tools/firmware/hvmloader/e820.h | 7 +
tools/firmware/hvmloader/hvmloader.c | 36 ++++
tools/firmware/hvmloader/pci.c | 19 +-
tools/firmware/hvmloader/util.c | 25 +++
tools/firmware/hvmloader/util.h | 10 +
tools/libxc/include/xenctrl.h | 19 +-
tools/libxc/include/xenguest.h | 3 +-
tools/libxc/xc_domain.c | 80 +++++++-
tools/libxc/xc_hvm_build_x86.c | 28 +--
tools/libxl/libxl_create.c | 7 +-
tools/libxl/libxl_dm.c | 195 ++++++++++++++++++++
tools/libxl/libxl_dom.c | 99 +++++++++-
tools/libxl/libxl_internal.h | 11 +-
tools/libxl/libxl_pci.c | 11 +-
tools/libxl/libxl_types.idl | 25 +++
tools/libxl/libxlu_pci.c | 78 ++++++++
tools/libxl/libxlutil.h | 4 +
tools/libxl/xl_cmdimpl.c | 44 ++++-
tools/libxl/xl_cmdtable.c | 2 +-
tools/ocaml/libs/xc/xenctrl_stubs.c | 17 +-
tools/python/xen/lowlevel/xc/xc.c | 29 ++-
xen/arch/x86/hvm/hvm.c | 2 -
xen/arch/x86/mm.c | 6 -
xen/arch/x86/mm/p2m.c | 30 +++
xen/common/compat/memory.c | 66 +++++++
xen/common/memory.c | 63 +++++++
xen/drivers/passthrough/amd/pci_amd_iommu.c | 3 +-
xen/drivers/passthrough/iommu.c | 10 +
xen/drivers/passthrough/pci.c | 10 +-
xen/drivers/passthrough/vtd/dmar.c | 32 ++++
xen/drivers/passthrough/vtd/extern.h | 1 +
xen/drivers/passthrough/vtd/iommu.c | 37 +++-
xen/include/asm-x86/p2m.h | 4 +
xen/include/public/domctl.h | 4 +
xen/include/public/memory.h | 32 +++-
xen/include/xen/iommu.h | 6 +-
xen/include/xen/pci.h | 2 +
xen/include/xlat.lst | 3 +-
41 files changed, 1101 insertions(+), 77 deletions(-)
Thanks
Tiejun
next reply other threads:[~2015-04-10 9:21 UTC|newest]
Thread overview: 125+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-04-10 9:21 Tiejun Chen [this message]
2015-04-10 9:21 ` [RFC][PATCH 01/13] tools: introduce some new parameters to set rdm policy Tiejun Chen
2015-05-08 13:04 ` Wei Liu
2015-05-11 5:35 ` Chen, Tiejun
2015-05-11 14:54 ` Wei Liu
2015-05-15 1:52 ` Chen, Tiejun
2015-05-18 1:06 ` Chen, Tiejun
2015-05-18 19:17 ` Wei Liu
2015-05-19 3:16 ` Chen, Tiejun
2015-05-19 9:42 ` Wei Liu
2015-05-19 10:50 ` Chen, Tiejun
2015-05-19 11:00 ` Wei Liu
2015-05-20 5:27 ` Chen, Tiejun
2015-05-20 8:36 ` Wei Liu
2015-05-20 8:51 ` Chen, Tiejun
2015-05-20 9:07 ` Wei Liu
2015-04-10 9:21 ` [RFC][PATCH 02/13] introduce XENMEM_reserved_device_memory_map Tiejun Chen
2015-04-16 14:59 ` Tim Deegan
2015-04-16 15:10 ` Jan Beulich
2015-04-16 15:24 ` Tim Deegan
2015-04-16 15:40 ` Tian, Kevin
2015-04-23 12:32 ` Chen, Tiejun
2015-04-23 12:59 ` Jan Beulich
2015-04-24 1:17 ` Chen, Tiejun
2015-04-24 7:21 ` Jan Beulich
2015-04-10 9:21 ` [RFC][PATCH 03/13] tools/libxc: Expose new hypercall xc_reserved_device_memory_map Tiejun Chen
2015-05-08 13:07 ` Wei Liu
2015-05-11 5:36 ` Chen, Tiejun
2015-05-11 9:50 ` Wei Liu
2015-04-10 9:21 ` [RFC][PATCH 04/13] tools/libxl: detect and avoid conflicts with RDM Tiejun Chen
2015-04-15 13:10 ` Ian Jackson
2015-04-15 18:22 ` Tian, Kevin
2015-04-23 12:31 ` Chen, Tiejun
2015-04-20 11:13 ` Jan Beulich
2015-05-06 15:00 ` Chen, Tiejun
2015-05-06 15:34 ` Jan Beulich
2015-05-07 2:22 ` Chen, Tiejun
2015-05-07 6:04 ` Jan Beulich
2015-05-08 1:14 ` Chen, Tiejun
2015-05-08 1:24 ` Chen, Tiejun
2015-05-08 15:13 ` Wei Liu
2015-05-11 6:06 ` Chen, Tiejun
2015-05-08 14:43 ` Wei Liu
2015-05-11 8:09 ` Chen, Tiejun
2015-05-11 11:32 ` Wei Liu
2015-05-14 8:27 ` Chen, Tiejun
2015-05-18 1:06 ` Chen, Tiejun
2015-05-18 20:00 ` Wei Liu
2015-05-19 1:32 ` Tian, Kevin
2015-05-19 10:22 ` Wei Liu
2015-05-19 6:47 ` Chen, Tiejun
2015-04-10 9:21 ` [RFC][PATCH 05/13] xen/x86/p2m: introduce set_identity_p2m_entry Tiejun Chen
2015-04-16 15:05 ` Tim Deegan
2015-04-23 12:33 ` Chen, Tiejun
2015-04-10 9:21 ` [RFC][PATCH 06/13] xen:vtd: create RMRR mapping Tiejun Chen
2015-04-16 15:16 ` Tim Deegan
2015-04-16 15:50 ` Tian, Kevin
2015-04-10 9:21 ` [RFC][PATCH 07/13] xen/passthrough: extend hypercall to support rdm reservation policy Tiejun Chen
2015-04-16 15:40 ` Tim Deegan
2015-04-23 12:32 ` Chen, Tiejun
2015-04-23 13:05 ` Tim Deegan
2015-04-23 13:59 ` Jan Beulich
2015-04-23 14:26 ` Tim Deegan
2015-05-04 8:15 ` Tian, Kevin
2015-04-20 13:36 ` Jan Beulich
2015-05-11 8:37 ` Chen, Tiejun
2015-05-08 16:07 ` Julien Grall
2015-05-11 8:42 ` Chen, Tiejun
2015-05-11 9:51 ` Julien Grall
2015-05-11 10:57 ` Jan Beulich
2015-05-14 5:48 ` Chen, Tiejun
2015-05-14 20:13 ` Jan Beulich
2015-05-14 5:47 ` Chen, Tiejun
2015-05-14 10:19 ` Julien Grall
2015-04-10 9:21 ` [RFC][PATCH 08/13] tools: extend xc_assign_device() " Tiejun Chen
2015-04-20 13:39 ` Jan Beulich
2015-05-11 9:45 ` Chen, Tiejun
2015-05-11 10:53 ` Jan Beulich
2015-05-14 7:04 ` Chen, Tiejun
2015-04-10 9:22 ` [RFC][PATCH 09/13] xen: enable XENMEM_set_memory_map in hvm Tiejun Chen
2015-04-16 15:42 ` Tim Deegan
2015-04-20 13:46 ` Jan Beulich
2015-05-15 2:33 ` Chen, Tiejun
2015-05-15 6:12 ` Jan Beulich
2015-05-15 6:24 ` Chen, Tiejun
2015-05-15 6:35 ` Jan Beulich
2015-05-15 6:59 ` Chen, Tiejun
2015-04-10 9:22 ` [RFC][PATCH 10/13] tools: extend XENMEM_set_memory_map Tiejun Chen
2015-04-10 10:01 ` Wei Liu
2015-04-13 2:09 ` Chen, Tiejun
2015-04-13 11:02 ` Wei Liu
2015-04-14 0:42 ` Chen, Tiejun
2015-05-05 9:32 ` Wei Liu
2015-04-20 13:51 ` Jan Beulich
2015-05-15 2:57 ` Chen, Tiejun
2015-05-15 6:16 ` Jan Beulich
2015-05-15 7:09 ` Chen, Tiejun
2015-05-15 7:32 ` Jan Beulich
2015-05-15 7:51 ` Chen, Tiejun
2015-04-10 9:22 ` [RFC][PATCH 11/13] hvmloader: get guest memory map into memory_map[] Tiejun Chen
2015-04-20 13:57 ` Jan Beulich
2015-05-15 3:10 ` Chen, Tiejun
2015-04-10 9:22 ` [RFC][PATCH 12/13] hvmloader/pci: skip reserved ranges Tiejun Chen
2015-04-20 14:21 ` Jan Beulich
2015-05-15 3:18 ` Chen, Tiejun
2015-05-15 6:19 ` Jan Beulich
2015-05-15 7:34 ` Chen, Tiejun
2015-05-15 7:44 ` Jan Beulich
2015-05-15 8:16 ` Chen, Tiejun
2015-05-15 8:31 ` Jan Beulich
2015-05-15 9:21 ` Chen, Tiejun
2015-05-15 9:32 ` Jan Beulich
2015-04-10 9:22 ` [RFC][PATCH 13/13] hvmloader/e820: construct guest e820 table Tiejun Chen
2015-04-20 14:29 ` Jan Beulich
2015-05-15 6:11 ` Chen, Tiejun
2015-05-15 6:25 ` Jan Beulich
2015-05-15 6:39 ` Chen, Tiejun
2015-05-15 6:56 ` Jan Beulich
2015-05-15 7:11 ` Chen, Tiejun
2015-05-15 7:34 ` Jan Beulich
2015-05-15 8:00 ` Chen, Tiejun
2015-05-15 8:12 ` Jan Beulich
2015-05-15 8:47 ` Chen, Tiejun
2015-05-15 8:54 ` Jan Beulich
2015-05-15 9:18 ` Chen, Tiejun
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1428657724-3498-1-git-send-email-tiejun.chen@intel.com \
--to=tiejun.chen@intel.com \
--cc=Ian.Jackson@eu.citrix.com \
--cc=JBeulich@suse.com \
--cc=andrew.cooper3@citrix.com \
--cc=ian.campbell@citrix.com \
--cc=kevin.tian@intel.com \
--cc=konrad.wilk@oracle.com \
--cc=stefano.stabellini@citrix.com \
--cc=tim@xen.org \
--cc=wei.liu2@citrix.com \
--cc=xen-devel@lists.xen.org \
--cc=yang.z.zhang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.