Re: [Xen-devel] [PATCH v5 05/10] domain: introduce XEN_DOMCTL_CDF_iommu flag

From: Paul Durrant <Paul.Durrant@citrix.com>
To: 'Julien Grall' <julien.grall@arm.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Stefano Stabellini <sstabellini@kernel.org>, Wei Liu <wl@xen.org>,
	Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>,
	Andrew Cooper <Andrew.Cooper3@citrix.com>,
	"Tim \(Xen.org\)" <tim@xen.org>,
	George Dunlap <George.Dunlap@citrix.com>,
	Jan Beulich <jbeulich@suse.com>,
	Anthony Perard <anthony.perard@citrix.com>,
	Ian Jackson <Ian.Jackson@citrix.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Roger Pau Monne <roger.pau@citrix.com>
Subject: Re: [Xen-devel] [PATCH v5 05/10] domain: introduce XEN_DOMCTL_CDF_iommu flag
Date: Wed, 14 Aug 2019 14:26:21 +0000	[thread overview]
Message-ID: <ed01d1f0d22a409389c6a9f17c020e82@AMSPEX02CL03.citrite.net> (raw)
In-Reply-To: <b0af4dc1-8991-7ec5-1a2c-24ccc8f1b96f@arm.com>

> -----Original Message-----
> From: Julien Grall <julien.grall@arm.com>
> Sent: 14 August 2019 15:00
> To: Paul Durrant <Paul.Durrant@citrix.com>; xen-devel@lists.xenproject.org
> Cc: Ian Jackson <Ian.Jackson@citrix.com>; Wei Liu <wl@xen.org>; Anthony Perard
> <anthony.perard@citrix.com>; Andrew Cooper <Andrew.Cooper3@citrix.com>; George Dunlap
> <George.Dunlap@citrix.com>; Jan Beulich <jbeulich@suse.com>; Konrad Rzeszutek Wilk
> <konrad.wilk@oracle.com>; Stefano Stabellini <sstabellini@kernel.org>; Tim (Xen.org) <tim@xen.org>;
> Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>; Roger Pau Monne <roger.pau@citrix.com>
> Subject: Re: [PATCH v5 05/10] domain: introduce XEN_DOMCTL_CDF_iommu flag
> 
> Hi Paul,
> 
> On 14/08/2019 14:38, Paul Durrant wrote:
> > This patch introduces a common domain creation flag to determine whether
> > the domain is permitted to make use of the IOMMU. Currently the flag is
> > always set (for both dom0 and domU) if the IOMMU is globally enabled
> > (i.e. iommu_enabled == 1). sanitise_domain_config() is modified to reject
> > the flag if !iommu_enabled.
> >
> > A new helper function, is_iommu_enabled(), is added to test the flag and
> > iommu_domain_init() will return immediately if !is_iommu_enabled(). This is
> > slightly different to the previous behaviour based on !iommu_enabled where
> > the call to arch_iommu_domain_init() was made regardless, however it appears
> > that this call was only necessary to initialize the dt_devices list for ARM
> > such that iommu_release_dt_devices() can be called unconditionally by
> > domain_relinquish_resources(). Adding a simple check of is_iommu_enabled()
> > into iommu_release_dt_devices() keeps this unconditional call working.
> >
> > No functional change should be observed with this patch applied.
> >
> > Subsequent patches will allow the toolstack to control whether use of the
> > IOMMU is enabled for a domain.
> >
> > NOTE: The introduction of the is_iommu_enabled() helper function might
> >        seem excessive but its use is expected to increase with subsequent
> >        patches. Also, having iommu_domain_init() bail before calling
> >        arch_iommu_domain_init() is not strictly necessary, but I think the
> >        consequent addition of the call to is_iommu_enabled() in
> >        iommu_release_dt_devices() makes the code clearer.
> >
> > Signed-off-by: Paul Durrant <paul.durrant@citrix.com>
> > ---
> > Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> > Cc: Wei Liu <wl@xen.org>
> > Cc: Anthony PERARD <anthony.perard@citrix.com>
> > Cc: Andrew Cooper <andrew.cooper3@citrix.com>
> > Cc: George Dunlap <George.Dunlap@eu.citrix.com>
> > Cc: Jan Beulich <jbeulich@suse.com>
> > Cc: Julien Grall <julien.grall@arm.com>
> > Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> > Cc: Stefano Stabellini <sstabellini@kernel.org>
> > Cc: Tim Deegan <tim@xen.org>
> > Cc: Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>
> > Cc: "Roger Pau Monné" <roger.pau@citrix.com>
> >
> > Previously part of series https://lists.xenproject.org/archives/html/xen-devel/2019-07/msg02267.html
> >
> > v5:
> >   - Move is_iommu_enabled() check into iommu_domain_init()
> >   - Reject XEN_DOMCTL_CDF_iommu in sanitise_domain_config() if !iommu_enabled
> >   - Use evaluate_nospec() in defintion of is_iommu_enabled()
> > ---
> >   tools/libxl/libxl_create.c            | 8 ++++++++
> >   xen/arch/arm/domain.c                 | 1 -
> >   xen/arch/arm/setup.c                  | 3 +++
> >   xen/arch/x86/setup.c                  | 3 +++
> >   xen/common/domain.c                   | 9 ++++++++-
> >   xen/drivers/passthrough/device_tree.c | 3 +++
> >   xen/drivers/passthrough/iommu.c       | 6 +++---
> >   xen/include/public/domctl.h           | 4 ++++
> >   xen/include/xen/sched.h               | 5 +++++
> >   9 files changed, 37 insertions(+), 5 deletions(-)
> >
> > diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
> > index 03ce166f4f..050ef042cd 100644
> > --- a/tools/libxl/libxl_create.c
> > +++ b/tools/libxl/libxl_create.c
> > @@ -555,6 +555,7 @@ int libxl__domain_make(libxl__gc *gc, libxl_domain_config *d_config,
> >               .max_grant_frames = b_info->max_grant_frames,
> >               .max_maptrack_frames = b_info->max_maptrack_frames,
> >           };
> > +        libxl_physinfo physinfo;
> >
> >           if (info->type != LIBXL_DOMAIN_TYPE_PV) {
> >               create.flags |= XEN_DOMCTL_CDF_hvm_guest;
> > @@ -564,6 +565,13 @@ int libxl__domain_make(libxl__gc *gc, libxl_domain_config *d_config,
> >                   libxl_defbool_val(info->oos) ? 0 : XEN_DOMCTL_CDF_oos_off;
> >           }
> >
> > +        rc = libxl_get_physinfo(ctx, &physinfo);
> > +        if (rc < 0)
> > +            goto out;
> > +
> > +        if (physinfo.cap_hvm_directio)
> > +            create.flags |= XEN_DOMCTL_CDF_iommu;
> 
> This is not going to work well on Arm as XEN_SYSCTL_PHYSCAP_directio is never set.

Oh, that's true. I need to pull that into the common sysctl handler. It also looks like XEN_SYSCTL_PHYSCAP_hvm should always be set for too, but ARMs arch_do_physinfo() is completely empty at the moment.

> 
> > +
> >           /* Ultimately, handle is an array of 16 uint8_t, same as uuid */
> >           libxl_uuid_copy(ctx, (libxl_uuid *)&create.handle, &info->uuid);
> >
> > diff --git a/xen/arch/arm/domain.c b/xen/arch/arm/domain.c
> > index 941bbff4fe..3ff19bc1ca 100644
> > --- a/xen/arch/arm/domain.c
> > +++ b/xen/arch/arm/domain.c
> > @@ -673,7 +673,6 @@ int arch_domain_create(struct domain *d,
> >
> >       ASSERT(config != NULL);
> >
> > -    /* p2m_init relies on some value initialized by the IOMMU subsystem */
> 
> Even with this patch, I still think iommu_domain_init() is required before
> calling p2m_init(). For instance, this function will set the features flag.

Oh, yes, I see that's in the SMMU code. I'll leave the comment in place... I thought it was only there because of arch_iommu_domain_init() being called unconditionally before (regardless of iommu_enabled).

  Paul

> 
> >       if ( (rc = iommu_domain_init(d)) != 0 )
> >           goto fail;
> 
> Cheers,
> 
> --
> Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel