Linux-PCI Archive on lore.kernel.org
 help / color / Atom feed
* [RFC V1 0/7] Add support for a new IMS interrupt mechanism
@ 2019-09-13  1:32 Megha Dey
  2019-09-13  1:32 ` [RFC V1 1/7] genirq/msi: Differentiate between various MSI based interrupts Megha Dey
                   ` (7 more replies)
  0 siblings, 8 replies; 21+ messages in thread
From: Megha Dey @ 2019-09-13  1:32 UTC (permalink / raw)
  To: linux-kernel, x86, linux-pci, maz, bhelgaas, rafael, gregkh,
	tglx, hpa, alex.williamson, jgg
  Cc: ashok.raj, megha.dey, jacob.jun.pan, Megha Dey

Currently, MSI (Message signaled interrupts) and MSI-X are the de facto
standard for device interrupt mechanism. MSI-X supports up to 2048
interrupts per device while MSI supports 32, which seems more than enough
for current devices. However, the introduction of SIOV (Scalable IO
virtualization) shifts the creation of assignable virtual devices from
hardware to a more software assisted approach. This flexible composition
of direct assignable devices, a.k.a. assignable device interfaces (ADIs)
unchains hardware from costly PCI standard. Under SIOV, device resource
can now be mapped directly to a guest or other user space drivers for
near native DMA performance. To complete functionality of ADIs, a matching
interrupt resource must also be introduced which will be scalable.

Interrupt message storage (IMS) is conceived  as a scalable albeit device
specific interrupt mechanism to meet such a demand. With IMS, there is
theoretically no upper bound on the number of interrupts which a device
can support. The size and location of IMS is device-specific; some devices
may implement IMS as on-device storage which are memory-mapped, others may
opt to implement IMS in system memory. IMS stores each interrupt message as
a DWORD size data payload and a 64-bit address(same as MSI-X). Access to
the IMS is through the host driver due to the non-architectural nature of
device IMS unlike the architectural MSI-X table which are accessed through
PCI drivers.

In this patchset, we introduce generic IMS APIs that fits the Linux IRQ
subsystem, supports IMS IRQ chip and domains that can be used by drivers
which are capable of generating IMS interrupts.

The IMS has been introduced as part of Intel's Scalable I/O virtualization
specification:
https://software.intel.com/en-us/download/intel-scalable-io-virtualization-technical-specification

This patchset is based on Linux 5.3-rc8.

Currently there is no device out in the market which supports SIOV (Hence no
device supports IMS).

This series is a basic patchset to get the ball rolling and receive some
inital comments. As per my discussion with Marc Zyngier and Thomas Gleixner
at the Linux Plumbers, I need to do the following:
1. Since a device can support MSI-X and IMS simultaneously, ensure proper
   locking mechanism for the 'msi_list' in the device structure.
2. Introduce dynamic allocation of IMS vectors perhaps by using a group ID
3. IMS support of a device needs to be discoverable. A bit in the vendor
   specific capability in the PCI config is to be added rather than getting
   this information from each device driver.

Jason Gunthorpe of Mellanox technologies is looking to do something similar
on ARM platforms and was wondering why IMS is x86 sepcific. Perhaps we can
use this thread to discuss further on this. 

Megha Dey (7):
  genirq/msi: Differentiate between various MSI based interrupts
  drivers/base: Introduce callbacks for IMS interrupt domain
  x86/ims: Add support for a new IMS irq domain
  irq_remapping: New interfaces to support IMS irqdomain
  x86/ims: Introduce x86_ims_ops
  ims-msi: Add APIs to allocate/free IMS interrupts
  ims: Add the set_desc callback

 arch/mips/pci/msi-xlp.c              |   2 +-
 arch/s390/pci/pci_irq.c              |   2 +-
 arch/x86/include/asm/irq_remapping.h |  13 ++
 arch/x86/include/asm/msi.h           |   4 +
 arch/x86/include/asm/pci.h           |   4 +
 arch/x86/include/asm/x86_init.h      |  10 +
 arch/x86/kernel/apic/Makefile        |   1 +
 arch/x86/kernel/apic/ims.c           | 118 ++++++++++++
 arch/x86/kernel/apic/msi.c           |   6 +-
 arch/x86/kernel/x86_init.c           |  23 +++
 arch/x86/pci/xen.c                   |   2 +-
 drivers/base/Kconfig                 |   7 +
 drivers/base/Makefile                |   1 +
 drivers/base/ims-msi.c               | 353 +++++++++++++++++++++++++++++++++++
 drivers/iommu/intel_irq_remapping.c  |  30 +++
 drivers/iommu/irq_remapping.c        |   9 +
 drivers/iommu/irq_remapping.h        |   3 +
 drivers/pci/msi.c                    |  19 +-
 drivers/vfio/mdev/mdev_core.c        |   6 +
 drivers/vfio/mdev/mdev_private.h     |   1 -
 include/linux/intel-iommu.h          |   1 +
 include/linux/mdev.h                 |   2 +
 include/linux/msi.h                  |  55 +++++-
 kernel/irq/msi.c                     |   2 +-
 24 files changed, 655 insertions(+), 19 deletions(-)
 create mode 100644 arch/x86/kernel/apic/ims.c
 create mode 100644 drivers/base/ims-msi.c

-- 
2.7.4


^ permalink raw reply	[flat|nested] 21+ messages in thread

end of thread, back to index

Thread overview: 21+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-09-13  1:32 [RFC V1 0/7] Add support for a new IMS interrupt mechanism Megha Dey
2019-09-13  1:32 ` [RFC V1 1/7] genirq/msi: Differentiate between various MSI based interrupts Megha Dey
2019-09-13  4:40   ` Greg KH
2019-09-13 14:40   ` Jason Gunthorpe
2019-09-26 21:50   ` Bjorn Helgaas
2019-09-13  1:32 ` [RFC V1 2/7] drivers/base: Introduce callbacks for IMS interrupt domain Megha Dey
2019-09-13  4:39   ` Greg KH
2019-09-13 14:50   ` Jason Gunthorpe
2019-09-13  1:32 ` [RFC V1 3/7] x86/ims: Add support for a new IMS irq domain Megha Dey
2019-09-13 14:52   ` Jason Gunthorpe
2019-09-13 22:07   ` Alex Williamson
2019-09-13  1:32 ` [RFC V1 4/7] irq_remapping: New interfaces to support IMS irqdomain Megha Dey
2019-09-13  1:32 ` [RFC V1 5/7] x86/ims: Introduce x86_ims_ops Megha Dey
2019-09-13 14:54   ` Jason Gunthorpe
2019-09-13  1:32 ` [RFC V1 6/7] ims-msi: Add APIs to allocate/free IMS interrupts Megha Dey
2019-09-26 21:44   ` Bjorn Helgaas
2019-09-13  1:32 ` [RFC V1 7/7] ims: Add the set_desc callback Megha Dey
2019-09-26 21:47   ` Bjorn Helgaas
2019-09-13 19:50 ` [RFC V1 0/7] Add support for a new IMS interrupt mechanism Jason Gunthorpe
2019-09-13 20:27   ` Raj, Ashok
2019-09-19 18:25     ` Jason Gunthorpe

Linux-PCI Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-pci/0 linux-pci/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-pci linux-pci/ https://lore.kernel.org/linux-pci \
		linux-pci@vger.kernel.org linux-pci@archiver.kernel.org
	public-inbox-index linux-pci

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-pci


AGPL code for this site: git clone https://public-inbox.org/ public-inbox