From: Paul Durrant <Paul.Durrant@citrix.com>
To: Roger Pau Monne <roger.pau@citrix.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
Andrew Cooper <Andrew.Cooper3@citrix.com>,
"Tim \(Xen.org\)" <tim@xen.org>,
George Dunlap <George.Dunlap@citrix.com>,
Julien Grall <julien.grall@arm.com>,
Jan Beulich <jbeulich@suse.com>,
Ian Jackson <Ian.Jackson@citrix.com>,
Anthony Perard <anthony.perard@citrix.com>,
"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
Subject: Re: [Xen-devel] [PATCH v6 06/10] domain: introduce XEN_DOMCTL_CDF_iommu flag
Date: Thu, 29 Aug 2019 09:00:09 +0000 [thread overview]
Message-ID: <3aca89f0afb54c2f8a20b0aadf43ec1d@AMSPEX02CL03.citrite.net> (raw)
In-Reply-To: <20190823103254.juap7726l2stdkfb@Air-de-Roger>
> -----Original Message-----
> From: Roger Pau Monne <roger.pau@citrix.com>
> Sent: 23 August 2019 11:33
> To: Paul Durrant <Paul.Durrant@citrix.com>
> Cc: xen-devel@lists.xenproject.org; Ian Jackson <Ian.Jackson@citrix.com>; Wei Liu <wl@xen.org>;
> Anthony Perard <anthony.perard@citrix.com>; Andrew Cooper <Andrew.Cooper3@citrix.com>; George Dunlap
> <George.Dunlap@citrix.com>; Jan Beulich <jbeulich@suse.com>; Julien Grall <julien.grall@arm.com>;
> Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>; Stefano Stabellini <sstabellini@kernel.org>; Tim
> (Xen.org) <tim@xen.org>; Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
> Subject: Re: [PATCH v6 06/10] domain: introduce XEN_DOMCTL_CDF_iommu flag
>
> On Fri, Aug 16, 2019 at 06:19:57PM +0100, Paul Durrant wrote:
> > This patch introduces a common domain creation flag to determine whether
> > the domain is permitted to make use of the IOMMU. Currently the flag is
> > always set (for both dom0 and domU) if the IOMMU is globally enabled
> > (i.e. iommu_enabled == 1). sanitise_domain_config() is modified to reject
> > the flag if !iommu_enabled.
> >
> > A new helper function, is_iommu_enabled(), is added to test the flag and
> > iommu_domain_init() will return immediately if !is_iommu_enabled(). This is
> > slightly different to the previous behaviour based on !iommu_enabled where
> > the call to arch_iommu_domain_init() was made regardless, however it appears
> > that this call was only necessary to initialize the dt_devices list for ARM
> > such that iommu_release_dt_devices() can be called unconditionally by
> > domain_relinquish_resources(). Adding a simple check of is_iommu_enabled()
> > into iommu_release_dt_devices() keeps this unconditional call working.
> >
> > No functional change should be observed with this patch applied.
> >
> > Subsequent patches will allow the toolstack to control whether use of the
> > IOMMU is enabled for a domain.
> >
> > NOTE: The introduction of the is_iommu_enabled() helper function might
> > seem excessive but its use is expected to increase with subsequent
> > patches. Also, having iommu_domain_init() bail before calling
> > arch_iommu_domain_init() is not strictly necessary, but I think the
> > consequent addition of the call to is_iommu_enabled() in
> > iommu_release_dt_devices() makes the code clearer.
> >
> > Signed-off-by: Paul Durrant <paul.durrant@citrix.com>
>
> I have one ARM-related question and one 'nice to have', but the code
> LGTM:
>
> Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
>
> > ---
> > Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> > Cc: Wei Liu <wl@xen.org>
> > Cc: Anthony PERARD <anthony.perard@citrix.com>
> > Cc: Andrew Cooper <andrew.cooper3@citrix.com>
> > Cc: George Dunlap <George.Dunlap@eu.citrix.com>
> > Cc: Jan Beulich <jbeulich@suse.com>
> > Cc: Julien Grall <julien.grall@arm.com>
> > Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> > Cc: Stefano Stabellini <sstabellini@kernel.org>
> > Cc: Tim Deegan <tim@xen.org>
> > Cc: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
> > Cc: "Roger Pau Monné" <roger.pau@citrix.com>
> >
> > Previously part of series https://lists.xenproject.org/archives/html/xen-devel/2019-07/msg02267.html
> >
> > v6:
> > - Remove the toolstack parts as there's no nice method of testing whether
> > the IOMMU is enabled in an architecture-neutral way
> >
> > v5:
> > - Move is_iommu_enabled() check into iommu_domain_init()
> > - Reject XEN_DOMCTL_CDF_iommu in sanitise_domain_config() if !iommu_enabled
> > - Use evaluate_nospec() in defintion of is_iommu_enabled()
> > ---
> > xen/arch/arm/setup.c | 3 +++
> > xen/arch/x86/setup.c | 3 +++
> > xen/common/domain.c | 9 ++++++++-
> > xen/common/domctl.c | 8 ++++++++
> > xen/drivers/passthrough/device_tree.c | 3 +++
> > xen/drivers/passthrough/iommu.c | 6 +++---
> > xen/include/public/domctl.h | 4 ++++
> > xen/include/xen/sched.h | 5 +++++
> > 8 files changed, 37 insertions(+), 4 deletions(-)
> >
> > diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> > index 2c5d1372c0..20021ee0ca 100644
> > --- a/xen/arch/arm/setup.c
> > +++ b/xen/arch/arm/setup.c
> > @@ -914,6 +914,9 @@ void __init start_xen(unsigned long boot_phys_offset,
> > dom0_cfg.arch.tee_type = tee_get_type();
> > dom0_cfg.max_vcpus = dom0_max_vcpus();
> >
> > + if ( iommu_enabled )
> > + dom0_cfg.flags |= XEN_DOMCTL_CDF_iommu;
> > +
> > dom0 = domain_create(0, &dom0_cfg, true);
> > if ( IS_ERR(dom0) || (alloc_dom0_vcpu0(dom0) == NULL) )
> > panic("Error creating domain 0\n");
> > diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
> > index d0b35b0ce2..fa226a2bab 100644
> > --- a/xen/arch/x86/setup.c
> > +++ b/xen/arch/x86/setup.c
> > @@ -1733,6 +1733,9 @@ void __init noreturn __start_xen(unsigned long mbi_p)
> > }
> > dom0_cfg.max_vcpus = dom0_max_vcpus();
> >
> > + if ( iommu_enabled )
> > + dom0_cfg.flags |= XEN_DOMCTL_CDF_iommu;
> > +
> > /* Create initial domain 0. */
> > dom0 = domain_create(get_initial_domain_id(), &dom0_cfg, !pv_shim);
> > if ( IS_ERR(dom0) || (alloc_dom0_vcpu0(dom0) == NULL) )
> > diff --git a/xen/common/domain.c b/xen/common/domain.c
> > index 76e6976617..e832a5c4aa 100644
> > --- a/xen/common/domain.c
> > +++ b/xen/common/domain.c
> > @@ -301,7 +301,8 @@ static int sanitise_domain_config(struct xen_domctl_createdomain *config)
> > XEN_DOMCTL_CDF_hap |
> > XEN_DOMCTL_CDF_s3_integrity |
> > XEN_DOMCTL_CDF_oos_off |
> > - XEN_DOMCTL_CDF_xs_domain) )
> > + XEN_DOMCTL_CDF_xs_domain |
> > + XEN_DOMCTL_CDF_iommu) )
> > {
> > dprintk(XENLOG_INFO, "Unknown CDF flags %#x\n", config->flags);
> > return -EINVAL;
> > @@ -328,6 +329,12 @@ static int sanitise_domain_config(struct xen_domctl_createdomain *config)
> > config->flags |= XEN_DOMCTL_CDF_oos_off;
> > }
> >
> > + if ( (config->flags & XEN_DOMCTL_CDF_iommu) && !iommu_enabled )
> > + {
> > + dprintk(XENLOG_INFO, "IOMMU is not enabled\n");
> > + return -EINVAL;
> > + }
> > +
> > return arch_sanitise_domain_config(config);
> > }
> >
> > diff --git a/xen/common/domctl.c b/xen/common/domctl.c
> > index 6e6e9b9866..fddf20f1b5 100644
> > --- a/xen/common/domctl.c
> > +++ b/xen/common/domctl.c
> > @@ -515,6 +515,14 @@ long do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
> > rover = dom;
> > }
> >
> > + /*
> > + * For now, make sure the createdomain IOMMU flag is set if the
> > + * IOMMU is enabled. When the flag comes under toolstack control
> > + * this can go away.
> > + */
> > + if ( iommu_enabled )
> > + op->u.createdomain.flags |= XEN_DOMCTL_CDF_iommu;
>
> Can you add some kind of safety check here to make sure this bodge is
> removed when the toolstack takes control of the flag, ie:
>
> BUG_ON(op->u.createdomain.flags & XEN_DOMCTL_CDF_iommu);
>
> Or maybe an ASSERT_UNREACHABLE() followed by returning EINVAL?
Ok, the former is a bit severe so I think I'll go for the latter.
>
> > +
> > d = domain_create(dom, &op->u.createdomain, false);
> > if ( IS_ERR(d) )
> > {
> > diff --git a/xen/drivers/passthrough/device_tree.c b/xen/drivers/passthrough/device_tree.c
> > index b6eaae7283..d32b172664 100644
> > --- a/xen/drivers/passthrough/device_tree.c
> > +++ b/xen/drivers/passthrough/device_tree.c
> > @@ -119,6 +119,9 @@ int iommu_release_dt_devices(struct domain *d)
> > struct dt_device_node *dev, *_dev;
> > int rc;
> >
> > + if ( !is_iommu_enabled(d) )
> > + return 0;
>
> How could you get here? If the domain doesn't have an iommu how did it
> got the devices assigned in the first place?
>
As I say in the commit comment, iommu_release_dt_device() is called unconditionally from domain_relinquish_resources().
> The hardware domain on ARM would be an exception here, since it uses
> an identity second stage translation, but I don't think
> iommu_release_dt_devices is ever used against the hardware domain.
>
No, it is only called from domain_relinquish_resources() so should not apply to the h/w domain.
Paul
> Thanks, Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
next prev parent reply other threads:[~2019-08-29 9:00 UTC|newest]
Thread overview: 35+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-08-16 17:19 [Xen-devel] [PATCH v6 00/10] use stashed domain create flags Paul Durrant
2019-08-16 17:19 ` [Xen-devel] [PATCH v6 01/10] make passthrough/pci.c:deassign_device() static Paul Durrant
2019-08-23 9:51 ` Roger Pau Monné
2019-08-16 17:19 ` [Xen-devel] [PATCH v6 02/10] x86/hvm/domain: remove the 'hap_enabled' flag Paul Durrant
2019-08-23 10:05 ` Roger Pau Monné
2019-08-23 12:23 ` Andrew Cooper
2019-08-23 12:25 ` Andrew Cooper
2019-08-27 8:19 ` Paul Durrant
2019-08-16 17:19 ` [Xen-devel] [PATCH v6 03/10] x86/domain: remove the 'oos_off' flag Paul Durrant
2019-08-16 17:19 ` [Xen-devel] [PATCH v6 04/10] domain: remove the 'is_xenstore' flag Paul Durrant
2019-08-19 20:44 ` Daniel De Graaf
2019-08-16 17:19 ` [Xen-devel] [PATCH v6 05/10] x86/domain: remove the 's3_integrity' flag Paul Durrant
2019-08-16 17:19 ` [Xen-devel] [PATCH v6 06/10] domain: introduce XEN_DOMCTL_CDF_iommu flag Paul Durrant
2019-08-23 10:32 ` Roger Pau Monné
2019-08-29 9:00 ` Paul Durrant [this message]
2019-08-16 17:19 ` [Xen-devel] [PATCH v6 07/10] use is_iommu_enabled() where appropriate Paul Durrant
2019-08-19 20:55 ` Daniel De Graaf
2019-08-23 3:04 ` Tian, Kevin
2019-08-23 10:55 ` Roger Pau Monné
2019-08-29 9:17 ` Paul Durrant
2019-08-29 13:29 ` Jan Beulich
2019-08-16 17:19 ` [Xen-devel] [PATCH v6 08/10] remove late (on-demand) construction of IOMMU page tables Paul Durrant
2019-08-16 17:24 ` Razvan Cojocaru
2019-08-23 11:34 ` Roger Pau Monné
2019-08-29 9:23 ` Paul Durrant
2019-08-29 13:39 ` Jan Beulich
2019-08-29 13:44 ` Paul Durrant
2019-08-16 17:20 ` [Xen-devel] [PATCH v6 09/10] iommu: tidy up iommu_use_hap_pt() and need_iommu_pt_sync() macros Paul Durrant
2019-08-23 11:39 ` Roger Pau Monné
2019-08-29 13:50 ` Jan Beulich
2019-08-16 17:20 ` [Xen-devel] [PATCH v6 10/10] introduce a 'passthrough' configuration option to xl.cfg Paul Durrant
2019-08-23 14:16 ` Roger Pau Monné
2019-08-29 15:25 ` Paul Durrant
2019-08-29 14:07 ` Jan Beulich
2019-08-29 15:27 ` Paul Durrant
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=3aca89f0afb54c2f8a20b0aadf43ec1d@AMSPEX02CL03.citrite.net \
--to=paul.durrant@citrix.com \
--cc=Andrew.Cooper3@citrix.com \
--cc=George.Dunlap@citrix.com \
--cc=Ian.Jackson@citrix.com \
--cc=Volodymyr_Babchuk@epam.com \
--cc=anthony.perard@citrix.com \
--cc=jbeulich@suse.com \
--cc=julien.grall@arm.com \
--cc=konrad.wilk@oracle.com \
--cc=roger.pau@citrix.com \
--cc=sstabellini@kernel.org \
--cc=tim@xen.org \
--cc=wl@xen.org \
--cc=xen-devel@lists.xenproject.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).