[PATCH v9 0/3] VT-d Device-TLB flush issue

* [PATCH v9 0/3] VT-d Device-TLB flush issue
@ 2016-04-01 14:47 Quan Xu
  2016-04-01 14:47 ` [PATCH v9 1/3] VT-d: add a command line parameter for Queued Invalidation Quan Xu
                   ` (2 more replies)
  0 siblings, 3 replies; 18+ messages in thread
From: Quan Xu @ 2016-04-01 14:47 UTC (permalink / raw)
  To: xen-devel; +Cc: kevin.tian, feng.wu, jbeulich, dario.faggioli, Quan Xu

This patches fix current timeout concern and also allow limited ATS support:

1. Add a command line parameter for Queued Invalidation

The command line parameter 'vtd_qi_timeout' specifies the
timeout of the VT-d Queued Invalidation in milliseconds. By
default, the timeout is 1ms, which can be boot-time changed.

Add a __must_check annotation. The followup patch titled
'VT-d IOTLB/Context/IEC flush issue' addresses the __mustcheck.
That is the other callers of this routine (two or three
levels up) ignore the return code. This patch does not
address this but the other does.

2. wrap a _sync version for all VT-d flush interfaces

The dev_invalidate_iotlb() scans ats_devices list to flush ATS devices,
and the invalidate_sync() is put after dev_invalidate_iotlb() to
synchronize with hardware for flush status. If we assign multiple
ATS devices to a domain, the flush status is about all these multiple
ATS devices. Once flush timeout expires, we couldn't find out which
one is the buggy ATS device.

Then, The invalidate_sync() variant (We need to pass down the device's
SBDF to hide the ATS device) is put within dev_invalidate_iotlb() to
synchronize for the flush status one by one. If flush timeout expires,
we could find out the buggy ATS device and hide it. However, for other
VT-d flush interfaces, the invalidate_sync() is still put after at present.
This is inconsistent.

So we wrap a _sync version for all VT-d flush interfaces. It simplifies
caller logic and makes code more readable as well.

3. Fix vt-d Device-TLB flush timeout issue

If Device-TLB flush timed out, we would hide the target ATS
device and crash the domain owning this ATS device. If impacted
domain is hardware domain, just throw out a warning (done in
queue_invalidate_wait).

By hiding the device, we make sure it can't be assigned to
any domain any longer (see device_assigned).

**NOTE**
   This patch set should base on one prereq patch sets:
    a). Check VT-d Device-TLB flush error.

----
Not covered in this series:

    a) Eliminate the panic() in IOMMU_WAIT_OP, used only in VT-d register read/write.
       Further discussion is required on whether and how to improve it.
    b) Handle IOTLB/Context/IEC flush timeout.

--Changes in v9:
#patch 1
   * Enhance the commit message and docs/misc/xen-command-line.markdown.
#patch 2
   * Enhance the commit message.
   * Add function declaration at the head of file, and then we don't need pure code movement.
#patch 3
   * Enhance the commit message.
   * 'ASSERT ( pdev->domain )' to 'ASSERT(pdev->domain)'

Quan Xu (3):
  VT-d: add a command line parameter for Queued Invalidation
  VT-d: wrap a _sync version for all VT-d flush interfaces
  VT-d: Fix vt-d Device-TLB flush timeout issue

 docs/misc/xen-command-line.markdown   |  18 +++++
 xen/drivers/passthrough/pci.c         |   6 +-
 xen/drivers/passthrough/vtd/extern.h  |   3 +
 xen/drivers/passthrough/vtd/qinval.c  | 131 ++++++++++++++++++++++++++++------
 xen/drivers/passthrough/vtd/x86/ats.c |  15 ++--
 xen/include/xen/pci.h                 |   1 +
 6 files changed, 141 insertions(+), 33 deletions(-)

-- 
1.9.1

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 18+ messages in thread