xen-devel.lists.xenproject.org archive mirror
 help / color / mirror / Atom feed
From: "Xu, Quan" <quan.xu@intel.com>
To: Jan Beulich <JBeulich@suse.com>
Cc: "Tian, Kevin" <kevin.tian@intel.com>,
	"Wu, Feng" <feng.wu@intel.com>,
	"Nakajima, Jun" <jun.nakajima@intel.com>,
	George Dunlap <george.dunlap@eu.citrix.com>,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Dario Faggioli <dario.faggioli@citrix.com>,
	"xen-devel@lists.xen.org" <xen-devel@lists.xen.org>,
	Julien Grall <julien.grall@arm.com>,
	Stefano Stabellini <stefano.stabellini@citrix.com>,
	Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>,
	Keir Fraser <keir@xen.org>
Subject: Re: [PATCH 2/2] IOMMU/MMU: Adjust low level functions for VT-d Device-TLB flush error.
Date: Fri, 25 Mar 2016 09:27:09 +0000	[thread overview]
Message-ID: <945CA011AD5F084CBEA3E851C0AB28894B86C482@SHSMSX101.ccr.corp.intel.com> (raw)
In-Reply-To: <56EBE46102000078000DE46B@prv-mh.provo.novell.com>

On March 18, 2016 6:20pm, <JBeulich@suse.com> wrote:
> >>> On 17.03.16 at 07:54, <quan.xu@intel.com> wrote:
> > --- a/xen/drivers/passthrough/amd/iommu_init.c
> > +++ b/xen/drivers/passthrough/amd/iommu_init.c
> > @@ -1339,12 +1339,14 @@ static void invalidate_all_devices(void)
> >      iterate_ivrs_mappings(_invalidate_all_devices);
> >  }
> >
> > -void amd_iommu_suspend(void)
> > +int amd_iommu_suspend(void)
> >  {
> >      struct amd_iommu *iommu;
> >
> >      for_each_amd_iommu ( iommu )
> >          disable_iommu(iommu);
> > +
> > +    return 0;
> >  }
> >
> >  void amd_iommu_resume(void)
> > @@ -1368,3 +1370,11 @@ void amd_iommu_resume(void)
> >          invalidate_all_domain_pages();
> >      }
> >  }
> > +
> > +void amd_iommu_crash_shutdown(void)
> > +{
> > +    struct amd_iommu *iommu;
> > +
> > +    for_each_amd_iommu ( iommu )
> > +        disable_iommu(iommu);
> > +}
> 
> One of the two should clearly call the other - no need to have the same code
> twice.
> 

Good idea.

> > --- a/xen/drivers/passthrough/iommu.c
> > +++ b/xen/drivers/passthrough/iommu.c
> > @@ -182,7 +182,11 @@ void __hwdom_init iommu_hwdom_init(struct
> domain *d)
> >                   ((page->u.inuse.type_info & PGT_type_mask)
> >                    == PGT_writable_page) )
> >                  mapping |= IOMMUF_writable;
> > -            hd->platform_ops->map_page(d, gfn, mfn, mapping);
> > +            if ( hd->platform_ops->map_page(d, gfn, mfn, mapping) )
> > +                printk(XENLOG_G_ERR
> > +                       "IOMMU: Map page gfn: 0x%lx(mfn: 0x%lx)
> failed.\n",
> > +                       gfn, mfn);
> > +
> 
> Printing one message here is certainly necessary, but what if the failure repeats
> for very many pages? 

Yes, to me, it is ok, but I am open to your suggestion.

> Also %#lx instead of 0x%lx please, and a blank before the
> opening parenthesis.
> 
OK, just check it:

..
"IOMMU: Map page gfn: %#lx (mfn: %#lx) failed.\n"
..

Right?


> > @@ -554,11 +555,24 @@ static void iommu_flush_all(void)
> >          iommu = drhd->iommu;
> >          iommu_flush_context_global(iommu, 0);
> >          flush_dev_iotlb = find_ats_dev_drhd(iommu) ? 1 : 0;
> > -        iommu_flush_iotlb_global(iommu, 0, flush_dev_iotlb);
> > +        rc = iommu_flush_iotlb_global(iommu, 0, flush_dev_iotlb);
> > +
> > +        if ( rc > 0 )
> > +        {
> > +            iommu_flush_write_buffer(iommu);
> 
> Why is this needed all of the sudden?

As there may be multiple IOMMUs. .e.g, there are 2 IOMMUs in my machine, and I can find the following log message:
"""
(XEN) Intel VT-d iommu 0 supported page sizes: 4kB, 2MB, 1GB.
(XEN) Intel VT-d iommu 1 supported page sizes: 4kB, 2MB, 1GB.
"""
__iiuc__, iommu_flush_write_buffer() is per IOMMU, so It should be called to flush every IOMMU.



> (Note that if you did a more fine grained
> split, it might also be easier for you to note/ explain all the not directly related
> changes in the respective commit messages. Unless of course they fix actual
> bugs, in which case they should be split out anyway; such individual fixes would
> also likely have a much faster route to commit, relieving you earlier from the
> burden of at least some of the changes you have to carry and re-base.)
> 
> > +            rc = 0;
> > +        }
> > +        else if ( rc < 0 )
> > +        {
> > +            printk(XENLOG_G_ERR "IOMMU: IOMMU flush all failed.\n");
> > +            break;
> > +        }
> 
> Is a log message really advisable here?
> 

To me, It looks tricky too. I was struggling to make decision. For scheme B, I would try to do as below:

if ( iommu_flush_all() )
    printk("... nnn ...");

but there are 4 function calls, if so, to me, it looks redundant.

Or, could I ignore the print out for iommu_flush_all() failed?



> > -static void __intel_iommu_iotlb_flush(struct domain *d, unsigned long
> > gfn,
> > +static int __intel_iommu_iotlb_flush(struct domain *d, unsigned long
> > +gfn,
> 
> While I'm not VT-d maintainer, I think changes like this would be a good
> opportunity to also drop the stray double underscores: You need to touch all
> callers anyway.
> 

I think this is optional.


> > @@ -584,37 +599,40 @@ static void __intel_iommu_iotlb_flush(struct
> > domain *d, unsigned long gfn,
> >              continue;
> >
> >          if ( page_count != 1 || gfn == INVALID_GFN )
> > -        {
> > -            if ( iommu_flush_iotlb_dsi(iommu, iommu_domid,
> > -                        0, flush_dev_iotlb) )
> > -                iommu_flush_write_buffer(iommu);
> > -        }
> > +            rc = iommu_flush_iotlb_dsi(iommu, iommu_domid,
> > +                                       0, flush_dev_iotlb);
> >          else
> > +            rc = iommu_flush_iotlb_psi(iommu, iommu_domid,
> > +                                       (paddr_t)gfn <<
> PAGE_SHIFT_4K, 0,
> > +                                       !dma_old_pte_present,
> > +                                       flush_dev_iotlb);
> > +        if ( rc > 0 )
> >          {
> > -            if ( iommu_flush_iotlb_psi(iommu, iommu_domid,
> > -                        (paddr_t)gfn << PAGE_SHIFT_4K,
> PAGE_ORDER_4K,
> 
> Note how this used PAGE_ORDER_4K so far?

Sorry, this is a rebasing mistake.

> 
> > -                        !dma_old_pte_present, flush_dev_iotlb) )
> > -                iommu_flush_write_buffer(iommu);
> > +            iommu_flush_write_buffer(iommu);
> 
> Same question again: Why is this all of the sudden needed on both paths?
> 

The same as above question. Hold on first.


> > @@ -622,7 +640,7 @@ static void dma_pte_clear_one(struct domain
> *domain, u64 addr)
> >      if ( pg_maddr == 0 )
> >      {
> >          spin_unlock(&hd->arch.mapping_lock);
> > -        return;
> > +        return -ENOMEM;
> >      }
> 
> addr_to_dma_page_maddr() gets called with "alloc" being false, so there can't
> be any memory allocation failure here. There simply is nothing to do in this
> case.
> 

I copy it from iommu_map_page().

Good, then the error of iommu_unmap_page() looks only from flush (the crash is at least obvious), then error handling can be lighter weight--
We may return an error, but don't roll back the failed operation.
Right?

> > -void me_wifi_quirk(struct domain *domain, u8 bus, u8 devfn, int map)
> > +int me_wifi_quirk(struct domain *domain, u8 bus, u8 devfn, int map)
> >  {
> >      u32 id;
> > +    int rc = 0;
> >
> >      id = pci_conf_read32(0, 0, 0, 0, 0);
> >      if ( IS_CTG(id) )
> >      {
> >          /* quit if ME does not exist */
> >          if ( pci_conf_read32(0, 0, 3, 0, 0) == 0xffffffff )
> > -            return;
> > +            return -ENOENT;
> 
> Is this really an error? IOW, do all systems which satisfy IS_CTG() have such a
> device?
> 
To be honest, I didn't know much about me_wifi_quirk.
Now, IMO I don't need to deal with me_wifi_quirk().

Quan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

  reply	other threads:[~2016-03-25  9:27 UTC|newest]

Thread overview: 42+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-03-17  6:54 [PATCH 0/2] Check " Quan Xu
2016-03-17  6:54 ` [PATCH 1/2] IOMMU/MMU: Adjust top level functions for " Quan Xu
2016-03-17  7:32   ` Tian, Kevin
2016-03-17  7:58     ` Jan Beulich
2016-03-17  8:00       ` Tian, Kevin
2016-03-17 12:30   ` George Dunlap
2016-03-17 12:33     ` George Dunlap
2016-03-18  3:19       ` Xu, Quan
2016-03-18  8:09         ` Jan Beulich
2016-03-24  6:45           ` Xu, Quan
2016-03-18  7:54     ` Xu, Quan
2016-03-18  8:19       ` Jan Beulich
2016-03-18  9:09         ` Xu, Quan
2016-03-18  9:29           ` Jan Beulich
2016-03-18  9:38             ` Dario Faggioli
2016-03-18  9:48               ` Jan Beulich
2016-03-21  6:18                 ` Tian, Kevin
2016-03-21 12:22                   ` Jan Beulich
2016-03-24  9:02                 ` Xu, Quan
2016-03-24  9:58                   ` Jan Beulich
2016-03-24 14:12                     ` Xu, Quan
2016-03-24 14:37                       ` Jan Beulich
2016-03-17 17:14   ` Jan Beulich
2016-03-28  3:33     ` Xu, Quan
2016-03-29  7:20       ` Jan Beulich
2016-03-30  2:28         ` Xu, Quan
2016-03-30  2:35           ` Xu, Quan
2016-03-30  8:05           ` Jan Beulich
2016-03-17  6:54 ` [PATCH 2/2] IOMMU/MMU: Adjust low " Quan Xu
2016-03-17  7:37   ` Tian, Kevin
2016-03-18  2:30     ` Xu, Quan
2016-03-18  8:06       ` Jan Beulich
2016-03-21  5:01         ` Tian, Kevin
2016-03-17 15:31   ` George Dunlap
2016-03-18  6:57     ` Xu, Quan
2016-03-18 10:20   ` Jan Beulich
2016-03-25  9:27     ` Xu, Quan [this message]
2016-03-29  7:36       ` Jan Beulich
2016-04-11  3:09         ` Xu, Quan
2016-04-11  3:27           ` Xu, Quan
2016-04-11 16:34             ` Jan Beulich
2016-04-12  1:09               ` Xu, Quan

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=945CA011AD5F084CBEA3E851C0AB28894B86C482@SHSMSX101.ccr.corp.intel.com \
    --to=quan.xu@intel.com \
    --cc=JBeulich@suse.com \
    --cc=andrew.cooper3@citrix.com \
    --cc=dario.faggioli@citrix.com \
    --cc=feng.wu@intel.com \
    --cc=george.dunlap@eu.citrix.com \
    --cc=julien.grall@arm.com \
    --cc=jun.nakajima@intel.com \
    --cc=keir@xen.org \
    --cc=kevin.tian@intel.com \
    --cc=stefano.stabellini@citrix.com \
    --cc=suravee.suthikulpanit@amd.com \
    --cc=xen-devel@lists.xen.org \
    --subject='Re: [PATCH 2/2] IOMMU/MMU: Adjust low level functions for VT-d Device-TLB flush error.' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).