xen-devel.lists.xenproject.org archive mirror
 help / color / mirror / Atom feed
From: "Xu, Quan" <quan.xu@intel.com>
To: "Xu, Quan" <quan.xu@intel.com>,
	"Tian, Kevin" <kevin.tian@intel.com>,
	"xen-devel@lists.xen.org" <xen-devel@lists.xen.org>,
	Jan Beulich <jbeulich@suse.com>
Cc: "Wu, Feng" <feng.wu@intel.com>,
	"dario.faggioli@citrix.com" <dario.faggioli@citrix.com>,
	Keir Fraser <keir@xen.org>,
	Andrew Cooper <andrew.cooper3@citrix.com>
Subject: Re: [PATCH v7 09/11] vt-d: fix the IOMMU flush issue
Date: Mon, 13 Jun 2016 08:50:56 +0000	[thread overview]
Message-ID: <945CA011AD5F084CBEA3E851C0AB28894B8D9DAE@SHSMSX101.ccr.corp.intel.com> (raw)
In-Reply-To: <945CA011AD5F084CBEA3E851C0AB28894B8D9860@SHSMSX101.ccr.corp.intel.com>

On June 12, 2016 5:27 PM, Xu, Quan <quan.xu@intel.com> wrote:
> On June 12, 2016 3:33 PM, Tian, Kevin <kevin.tian@intel.com> wrote:
> > > From: Xu, Quan
> > > Sent: Wednesday, June 08, 2016 4:59 PM @@ -545,18 +549,42 @@
> static
> > > int __must_check iommu_flush_all(void)  {
> > >      struct acpi_drhd_unit *drhd;
> > >      struct iommu *iommu;
> > > -    int flush_dev_iotlb;
> > > +    int rc = 0;
> > >
> > >      flush_all_cache();
> > >      for_each_drhd_unit ( drhd )
> > >      {
> > >          iommu = drhd->iommu;
> > > -        iommu_flush_context_global(iommu, 0);
> > > -        flush_dev_iotlb = find_ats_dev_drhd(iommu) ? 1 : 0;
> > > -        iommu_flush_iotlb_global(iommu, 0, flush_dev_iotlb);
> > > +        /*
> > > +         * The current logic for rc returns:
> > > +         *   - positive  invoke iommu_flush_write_buffer to flush cache.
> > > +         *   - zero      on success.
> > > +         *   - negative  on failure. Continue to flush IOMMU IOTLB on a
> > > +         *               best effort basis.
> > > +         *
> > > +         * Moreover, IOMMU flush handlers flush_context_qi and
> > flush_iotlb_qi
> > > +         * (or flush_context_reg and flush_iotlb_reg, deep functions in the
> > > +         * call trees of iommu_flush_context_global and
> > iommu_flush_iotlb_global)
> > > +         * are with the same logic to bubble up positive return value.
> > > +         */
> > > +        rc = iommu_flush_context_global(iommu, 0);
> > > +        if ( rc <= 0 )
> > > +        {
> > > +            int flush_dev_iotlb = find_ats_dev_drhd(iommu) ? 1 : 0;
> > > +            int ret = iommu_flush_iotlb_global(iommu, 0,
> > > + flush_dev_iotlb);
> > > +
> > > +            ASSERT(ret <= 0);
> > > +            if ( !rc )
> > > +                rc = ret;
> >
> > I'm dubious about the assertion here. Why can't above call return 1
> > upon error on earlier flush? I digged back your earlier reply like:
> >
> > > Yes, the iommu_flush_iotlb_dsi() can also return 1.
> > > Look at the call tree, at the beginning of
> > > flush_context_qi()/flush_iotlb_qi(), or
> > > flush_context_reg()/flush_iotlb_reg()..
> > >
> > > If rc was negative when we call iommu_flush_context_device(), it is
> > > impossible to return 1 for iommu_flush_iotlb_dsi().
> >
> > But I don't think it a good idea of making so much assumptions about
> > internal implementations of those low level interfaces.
> > Also flush_context may fail for one specific reason which doesn't
> > block flush_iotlb which could get 1 returned when caching mode is
> > disabled. We'd better have return-1 case correctly handled here.
> >
> 
> Your comment looks reasonable here. Could I change it as below:
> 
> -static int iommu_flush_iotlb_psi(
> -    struct iommu *iommu, u16 did, u64 addr, unsigned int order,
> -    int flush_non_present_entry, int flush_dev_iotlb)
> +static int __must_check iommu_flush_iotlb_psi(struct iommu *iommu, u16
> did,
> +                                              u64 addr, unsigned int order,
> +                                              int flush_non_present_entry,
> +                                              int flush_dev_iotlb)
>  {
>      struct iommu_flush *flush = iommu_get_flush(iommu);
>      int status;
> @@ -546,17 +550,35 @@ static int __must_check iommu_flush_all(void)
>      struct acpi_drhd_unit *drhd;
>      struct iommu *iommu;
>      int flush_dev_iotlb;
> +    int rc = 0;
> 
>      flush_all_cache();
>      for_each_drhd_unit ( drhd )
>      {
> +        int ret;
> +
>          iommu = drhd->iommu;
> -        iommu_flush_context_global(iommu, 0);
> +        /*
> +         * The current logic for rc returns:
> +         *   - positive  invoke iommu_flush_write_buffer to flush cache.
> +         *   - zero      on success.
> +         *   - negative  on failure. Continue to flush IOMMU IOTLB on a
> +         *               best effort basis.
> +         */
> +        rc = iommu_flush_context_global(iommu, 0);
>          flush_dev_iotlb = find_ats_dev_drhd(iommu) ? 1 : 0;
> -        iommu_flush_iotlb_global(iommu, 0, flush_dev_iotlb);
> +        ret = iommu_flush_iotlb_global(iommu, 0, flush_dev_iotlb);
> +        if ( !rc )
> +            rc = ret;
> +
> +        if ( rc > 0 || ret > 0 )
> +            iommu_flush_write_buffer(iommu);
>      }
> 
> -    return 0;
> +    if ( rc > 0 )
> +        rc = 0;
> +
> +    return rc;
>  }
> 
> 

Ah, this change is not correct, as the previous error return
value may be erased by the later positive / zero value.

I'll highlight this change is not under Jan's R-b in next v8. 

.. I think the below is correct.



+static int __must_check iommu_flush_iotlb_psi(struct iommu *iommu, u16 did,
+                                              u64 addr, unsigned int order,
+                                              int flush_non_present_entry,
+                                              int flush_dev_iotlb)
 {
     struct iommu_flush *flush = iommu_get_flush(iommu);
     int status;
@@ -546,17 +550,37 @@ static int __must_check iommu_flush_all(void)
     struct acpi_drhd_unit *drhd;
     struct iommu *iommu;
     int flush_dev_iotlb;
+    int rc = 0;

     flush_all_cache();
     for_each_drhd_unit ( drhd )
     {
+        int iommu_rc, iommu_ret;
+
         iommu = drhd->iommu;
-        iommu_flush_context_global(iommu, 0);
+        iommu_rc = iommu_flush_context_global(iommu, 0);
         flush_dev_iotlb = find_ats_dev_drhd(iommu) ? 1 : 0;
-        iommu_flush_iotlb_global(iommu, 0, flush_dev_iotlb);
+        iommu_ret = iommu_flush_iotlb_global(iommu, 0, flush_dev_iotlb);
+
+        /*
+         * The current logic for returns:
+         *   - positive  invoke iommu_flush_write_buffer to flush cache.
+         *   - zero      on success.
+         *   - negative  on failure. Continue to flush IOMMU IOTLB on a
+         *               best effort basis.
+         */
+        if ( iommu_rc > 0 || iommu_ret > 0 )
+            iommu_flush_write_buffer(iommu);
+        if ( rc >= 0 )
+            rc = iommu_rc;
+        if ( rc >= 0 )
+            rc = iommu_ret;
     }

-    return 0;
+    if ( rc > 0 )
+        rc = 0;
+
+    return rc;
 }


> 
> Also, Jan, what's your opinion?
> 
> Quan
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

  reply	other threads:[~2016-06-13  8:50 UTC|newest]

Thread overview: 45+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-06-08  8:58 [PATCH v7 00/11] Check VT-d Device-TLB flush error Xu, Quan
2016-06-08  8:58 ` [PATCH v7 01/11] IOMMU: handle IOMMU mapping and unmapping failures Xu, Quan
2016-06-08  8:58 ` [PATCH v7 02/11] IOMMU/MMU: enhance the call trees of IOMMU unmapping and mapping Xu, Quan
2016-06-08 14:40   ` Jan Beulich
     [not found]   ` <57584D8602000078000F32D0@prv-mh.provo.novell.com>
     [not found]     ` <421857ad-4f6b-e99c-1e81-034b19effdfb@amd.com>
2016-06-10  6:56       ` Jan Beulich
2016-06-12  6:41   ` Tian, Kevin
2016-06-08  8:58 ` [PATCH v7 03/11] IOMMU: propagate IOMMU Device-TLB flush error up to IOMMU unmapping (top level ones) Xu, Quan
2016-06-08 14:42   ` Jan Beulich
2016-06-08 14:54   ` Jan Beulich
2016-06-09 19:30     ` Suravee Suthikulanit
2016-06-09 10:28   ` Julien Grall
2016-06-08  8:58 ` [PATCH v7 04/11] IOMMU: propagate IOMMU Device-TLB flush error up to IOMMU mapping " Xu, Quan
2016-06-08 14:43   ` Jan Beulich
2016-06-09 18:45     ` Suravee Suthikulanit
2016-06-09 10:30   ` Julien Grall
2016-06-08  8:58 ` [PATCH v7 05/11] IOMMU/MMU: propagate IOMMU Device-TLB flush error up to iommu_iotlb_flush{, _all} " Xu, Quan
2016-06-09 11:12   ` Julien Grall
2016-06-09 11:45     ` Jan Beulich
2016-06-09 12:03       ` Julien Grall
2016-06-09 12:08         ` Julien Grall
2016-06-09 12:15           ` Jan Beulich
2016-06-09 12:24             ` Julien Grall
2016-06-09 12:32               ` Jan Beulich
2016-06-09 12:39                 ` Julien Grall
2016-06-12  6:55                   ` Tian, Kevin
2016-06-12  7:02                     ` Xu, Quan
2016-06-12 15:35                     ` Julien Grall
2016-06-09 12:14         ` Jan Beulich
2016-06-12  6:46   ` Tian, Kevin
2016-06-08  8:58 ` [PATCH v7 06/11] propagate IOMMU Device-TLB flush error up to EPT update " Xu, Quan
2016-06-08  8:59 ` [PATCH v7 07/11] IOMMU: propagate IOMMU Device-TLB flush error up to IOMMU suspending " Xu, Quan
2016-06-08 14:51   ` Jan Beulich
2016-06-12  7:42     ` Xu, Quan
2016-06-13  9:25       ` Jan Beulich
2016-06-09 18:58   ` Suravee Suthikulanit
2016-06-12  6:58   ` Tian, Kevin
2016-06-08  8:59 ` [PATCH v7 08/11] IOMMU: propagate IOMMU Device-TLB flush error (leaf ones) Xu, Quan
2016-06-08  8:59 ` [PATCH v7 09/11] vt-d: fix the IOMMU flush issue Xu, Quan
2016-06-12  7:32   ` Tian, Kevin
2016-06-12  9:27     ` Xu, Quan
2016-06-13  8:50       ` Xu, Quan [this message]
2016-06-08  8:59 ` [PATCH v7 10/11] vt-d: propagate the IOMMU Device-TLB flush error up to ME phantom functions Xu, Quan
2016-06-12  7:34   ` Tian, Kevin
2016-06-08  8:59 ` [PATCH v7 11/11] vt-d: add __must_check annotation to IOMMU flush pointers and handlers Xu, Quan
2016-06-12  7:35   ` Tian, Kevin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=945CA011AD5F084CBEA3E851C0AB28894B8D9DAE@SHSMSX101.ccr.corp.intel.com \
    --to=quan.xu@intel.com \
    --cc=andrew.cooper3@citrix.com \
    --cc=dario.faggioli@citrix.com \
    --cc=feng.wu@intel.com \
    --cc=jbeulich@suse.com \
    --cc=keir@xen.org \
    --cc=kevin.tian@intel.com \
    --cc=xen-devel@lists.xen.org \
    --subject='Re: [PATCH v7 09/11] vt-d: fix the IOMMU flush issue' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).