From: "Dey, Megha" <megha.dey@intel.com>
To: Jason Gunthorpe <jgg@mellanox.com>, Marc Zyngier <maz@kernel.org>
Cc: "Jiang, Dave" <dave.jiang@intel.com>,
"vkoul@kernel.org" <vkoul@kernel.org>,
"bhelgaas@google.com" <bhelgaas@google.com>,
"rafael@kernel.org" <rafael@kernel.org>,
"gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>,
"tglx@linutronix.de" <tglx@linutronix.de>,
"hpa@zytor.com" <hpa@zytor.com>,
"alex.williamson@redhat.com" <alex.williamson@redhat.com>,
"Pan, Jacob jun" <jacob.jun.pan@intel.com>,
"Raj, Ashok" <ashok.raj@intel.com>,
"Liu, Yi L" <yi.l.liu@intel.com>,
"Lu, Baolu" <baolu.lu@intel.com>,
"Tian, Kevin" <kevin.tian@intel.com>,
"Kumar, Sanjay K" <sanjay.k.kumar@intel.com>,
"Luck, Tony" <tony.luck@intel.com>,
"Lin, Jing" <jing.lin@intel.com>,
"Williams, Dan J" <dan.j.williams@intel.com>,
"kwankhede@nvidia.com" <kwankhede@nvidia.com>,
"eric.auger@redhat.com" <eric.auger@redhat.com>,
"parav@mellanox.com" <parav@mellanox.com>,
"Hansen, Dave" <dave.hansen@intel.com>,
"netanelg@mellanox.com" <netanelg@mellanox.com>,
"shahafs@mellanox.com" <shahafs@mellanox.com>,
"yan.y.zhao@linux.intel.com" <yan.y.zhao@linux.intel.com>,
"pbonzini@redhat.com" <pbonzini@redhat.com>,
"Ortiz, Samuel" <samuel.ortiz@intel.com>,
"Hossain, Mona" <mona.hossain@intel.com>,
"dmaengine@vger.kernel.org" <dmaengine@vger.kernel.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"x86@kernel.org" <x86@kernel.org>,
"linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>,
"kvm@vger.kernel.org" <kvm@vger.kernel.org>
Subject: RE: [PATCH RFC v2 02/18] irq/dev-msi: Add support for a new DEV_MSI irq domain
Date: Wed, 5 Aug 2020 19:18:39 +0000 [thread overview]
Message-ID: <96a1eb5ccc724790b5404a642583919d@intel.com> (raw)
In-Reply-To: <20200722195928.GN2021248@mellanox.com>
Hi Jason,
> -----Original Message-----
> From: Jason Gunthorpe <jgg@mellanox.com>
> Sent: Wednesday, July 22, 2020 12:59 PM
> To: Marc Zyngier <maz@kernel.org>
> Cc: Jiang, Dave <dave.jiang@intel.com>; vkoul@kernel.org; Dey, Megha
> <megha.dey@intel.com>; bhelgaas@google.com; rafael@kernel.org;
> gregkh@linuxfoundation.org; tglx@linutronix.de; hpa@zytor.com;
> alex.williamson@redhat.com; Pan, Jacob jun <jacob.jun.pan@intel.com>; Raj,
> Ashok <ashok.raj@intel.com>; Liu, Yi L <yi.l.liu@intel.com>; Lu, Baolu
> <baolu.lu@intel.com>; Tian, Kevin <kevin.tian@intel.com>; Kumar, Sanjay K
> <sanjay.k.kumar@intel.com>; Luck, Tony <tony.luck@intel.com>; Lin, Jing
> <jing.lin@intel.com>; Williams, Dan J <dan.j.williams@intel.com>;
> kwankhede@nvidia.com; eric.auger@redhat.com; parav@mellanox.com;
> Hansen, Dave <dave.hansen@intel.com>; netanelg@mellanox.com;
> shahafs@mellanox.com; yan.y.zhao@linux.intel.com; pbonzini@redhat.com;
> Ortiz, Samuel <samuel.ortiz@intel.com>; Hossain, Mona
> <mona.hossain@intel.com>; dmaengine@vger.kernel.org; linux-
> kernel@vger.kernel.org; x86@kernel.org; linux-pci@vger.kernel.org;
> kvm@vger.kernel.org
> Subject: Re: [PATCH RFC v2 02/18] irq/dev-msi: Add support for a new DEV_MSI
> irq domain
>
> On Wed, Jul 22, 2020 at 07:52:33PM +0100, Marc Zyngier wrote:
>
> > Which is exactly what platform-MSI already does. Why do we need
> > something else?
>
> It looks to me like all the code is around managing the
> dev->msi_domain of the devices.
>
> The intended use would have PCI drivers create children devices using mdev or
> virtbus and those devices wouldn't have a msi_domain from the platform. Looks
> like platform_msi_alloc_priv_data() fails immediately because dev->msi_domain
> will be NULL for these kinds of devices.
>
> Maybe that issue should be handled directly instead of wrappering
> platform_msi_*?
>
> For instance a trivial addition to the platform_msi API:
>
> platform_msi_assign_domain(struct_device *newly_created_virtual_device,
> struct device *physical_device);
>
> Which could set the msi_domain of new device using the topology of
> physical_device to deduce the correct domain?
>
> Then the question is how to properly create a domain within the hardware
> topology of physical_device with the correct parameters for the platform.
>
> Why do we need a dummy msi_domain anyhow? Can this just use
> physical_device->msi_domain directly? (I'm at my limit here of how much of this
> I remember, sorry)
>
> If you solve that it should solve the remapping problem too, as the
> physical_device is already assigned by the platform to a remapping irq domain if
> that is what the platform wants.
Yeah most of what you said is right. For the most part, we are simply introducing a new IRQ domain
which provides specific domain info ops for the classes of devices which want to provide custom
mask/unmask callbacks..
Also, from your other comments, I've realized the same IRQ domain can be used when interrupt
remapping is enabled/disabled.
Hence we will only have one create_dev_msi_domain which can be called by any device driver that
wants to use the dev-msi IRQ domain to alloc/free IRQs. It would be the responsibility of the device
driver to provide the correct device and update the dev->msi_domain.
>
> >> + parent = irq_get_default_host();
> > Really? How is it going to work once you have devices sending their
> > MSIs to two different downstream blocks? This looks rather
> > short-sighted.
>
> .. and fix this too, the parent domain should be derived from the topology of the
> physical_device which is originating the interrupt messages.
>
Yes
> > On the other hand, masking an interrupt is an irqchip operation, and
> > only concerns the irqchip level. Here, you seem to be making it an
> > end-point operation, which doesn't really make sense to me. Or is this
> > device its own interrupt controller as well? That would be extremely
> > surprising, and I'd expect some block downstream of the device to be
> > able to control the masking of the interrupt.
>
> These are message interrupts so they originate directly from the device and
> generally travel directly to the CPU APIC. On the wire there is no difference
> between a MSI, MSI-X and a device using the dev-msi approach.
>
> IIRC on Intel/AMD at least once a MSI is launched it is not maskable.
>
> So the model for MSI is always "mask at source". The closest mapping to the
> Linux IRQ model is to say the end device has a irqchip that encapsulates the
> ability of the device to generate the MSI in the first place.
>
> It looks like existing platform_msi drivers deal with "masking"
> implicitly by halting the device interrupt generation before releasing the
> interrupt and have no way for the generic irqchip layer to mask the interrupt.
>
> I suppose the motivation to make it explicit is related to vfio using the generic
> mask/unmask functionality?
>
> Explicit seems better, IMHO.
I don't think I understand this fully, ive still kept the device specific mask/unmask calls in the next
patch series, please let me know if it needs further modifications.
>
> Jason
next prev parent reply other threads:[~2020-08-05 19:19 UTC|newest]
Thread overview: 97+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-07-21 16:02 [PATCH RFC v2 00/18] Add VFIO mediated device support and DEV-MSI support for the idxd driver Dave Jiang
2020-07-21 16:02 ` [PATCH RFC v2 01/18] platform-msi: Introduce platform_msi_ops Dave Jiang
2020-07-21 16:02 ` [PATCH RFC v2 02/18] irq/dev-msi: Add support for a new DEV_MSI irq domain Dave Jiang
2020-07-21 16:13 ` Jason Gunthorpe
2020-07-22 16:50 ` Dey, Megha
2020-07-22 18:52 ` Marc Zyngier
2020-07-22 19:59 ` Jason Gunthorpe
2020-07-23 8:51 ` Marc Zyngier
2020-07-24 0:16 ` Jason Gunthorpe
2020-07-24 0:36 ` Thomas Gleixner
2020-08-05 19:18 ` Dey, Megha [this message]
2020-08-05 22:15 ` Jason Gunthorpe
2020-08-05 22:36 ` Dey, Megha
2020-08-05 22:53 ` Jason Gunthorpe
2020-08-06 0:13 ` Dey, Megha
2020-08-06 0:19 ` Jason Gunthorpe
2020-08-06 0:32 ` Dey, Megha
2020-08-06 0:46 ` Jason Gunthorpe
2020-08-06 17:10 ` Thomas Gleixner
2020-08-06 17:58 ` Dey, Megha
2020-08-06 20:21 ` Thomas Gleixner
2020-08-06 22:27 ` Dey, Megha
2020-08-07 8:48 ` Thomas Gleixner
2020-08-07 12:06 ` Jason Gunthorpe
2020-08-07 12:38 ` gregkh
2020-08-07 13:34 ` Jason Gunthorpe
2020-08-07 16:47 ` Thomas Gleixner
2020-08-07 17:54 ` Dey, Megha
2020-08-07 18:39 ` Jason Gunthorpe
2020-08-07 20:31 ` Dey, Megha
2020-08-08 19:47 ` Thomas Gleixner
2020-08-10 21:46 ` Thomas Gleixner
2020-08-11 9:53 ` Thomas Gleixner
2020-08-11 18:46 ` Dey, Megha
2020-08-11 21:25 ` Thomas Gleixner
2020-08-11 18:39 ` Dey, Megha
2020-08-11 22:39 ` Thomas Gleixner
2020-08-07 15:22 ` Thomas Gleixner
2020-08-05 18:55 ` Dey, Megha
2020-07-21 16:02 ` [PATCH RFC v2 03/18] irq/dev-msi: Create IR-DEV-MSI " Dave Jiang
2020-07-21 16:21 ` Jason Gunthorpe
2020-07-22 17:03 ` Dey, Megha
2020-07-22 17:33 ` Jason Gunthorpe
2020-07-22 20:44 ` Thomas Gleixner
2020-08-05 19:02 ` Dey, Megha
2020-07-21 16:02 ` [PATCH RFC v2 04/18] irq/dev-msi: Introduce APIs to allocate/free dev-msi interrupts Dave Jiang
2020-07-21 16:25 ` Jason Gunthorpe
2020-07-22 17:05 ` Dey, Megha
2020-07-22 17:35 ` Jason Gunthorpe
2020-08-05 20:19 ` Dey, Megha
2020-07-21 16:02 ` [PATCH RFC v2 05/18] dmaengine: idxd: add support for readonly config devices Dave Jiang
2020-07-21 16:02 ` [PATCH RFC v2 06/18] dmaengine: idxd: add interrupt handle request support Dave Jiang
2020-07-21 16:03 ` [PATCH RFC v2 07/18] dmaengine: idxd: add DEV-MSI support in base driver Dave Jiang
2020-07-21 16:03 ` [PATCH RFC v2 08/18] dmaengine: idxd: add device support functions in prep for mdev Dave Jiang
2020-07-21 16:03 ` [PATCH RFC v2 09/18] dmaengine: idxd: add basic mdev registration and helper functions Dave Jiang
2020-07-21 16:03 ` [PATCH RFC v2 10/18] dmaengine: idxd: add emulation rw routines Dave Jiang
2020-07-21 16:03 ` [PATCH RFC v2 11/18] dmaengine: idxd: prep for virtual device commands Dave Jiang
2020-07-21 16:03 ` [PATCH RFC v2 12/18] dmaengine: idxd: virtual device commands emulation Dave Jiang
2020-07-21 16:03 ` [PATCH RFC v2 13/18] dmaengine: idxd: ims setup for the vdcm Dave Jiang
2020-07-21 16:03 ` [PATCH RFC v2 14/18] dmaengine: idxd: add mdev type as a new wq type Dave Jiang
2020-07-21 16:03 ` [PATCH RFC v2 15/18] dmaengine: idxd: add dedicated wq mdev type Dave Jiang
2020-07-21 16:04 ` [PATCH RFC v2 16/18] dmaengine: idxd: add new wq state for mdev Dave Jiang
2020-07-21 16:04 ` [PATCH RFC v2 17/18] dmaengine: idxd: add error notification from host driver to mediated device Dave Jiang
2020-07-21 16:04 ` [PATCH RFC v2 18/18] dmaengine: idxd: add ABI documentation for mediated device support Dave Jiang
2020-07-21 16:28 ` [PATCH RFC v2 00/18] Add VFIO mediated device support and DEV-MSI support for the idxd driver Greg KH
2020-07-21 17:17 ` Dave Jiang
2020-07-21 21:35 ` Dan Williams
2020-07-21 16:45 ` Jason Gunthorpe
2020-07-21 18:00 ` Dave Jiang
2020-07-22 17:31 ` Dey, Megha
2020-07-22 18:16 ` Jason Gunthorpe
2020-07-21 23:54 ` Tian, Kevin
2020-07-24 0:19 ` Jason Gunthorpe
2020-08-06 1:22 ` Alex Williamson
2020-08-07 12:19 ` Jason Gunthorpe
2020-08-10 7:32 ` Tian, Kevin
2020-08-11 17:00 ` Alex Williamson
2020-08-12 1:58 ` Tian, Kevin
2020-08-12 2:36 ` Alex Williamson
2020-08-12 3:35 ` Tian, Kevin
2020-08-12 3:28 ` Jason Wang
2020-08-12 4:05 ` Tian, Kevin
2020-08-13 4:33 ` Jason Wang
2020-08-13 5:26 ` Tian, Kevin
2020-08-13 6:01 ` Jason Wang
2020-08-14 13:23 ` Jason Gunthorpe
2020-08-17 2:24 ` Tian, Kevin
2020-08-14 13:35 ` Jason Gunthorpe
2020-08-17 2:12 ` Tian, Kevin
2020-08-18 0:43 ` Jason Gunthorpe
2020-08-18 1:09 ` Tian, Kevin
2020-08-18 11:50 ` Jason Gunthorpe
2020-08-18 16:27 ` Paolo Bonzini
2020-08-18 16:49 ` Jason Gunthorpe
2020-08-18 17:05 ` Paolo Bonzini
2020-08-18 17:18 ` Jason Gunthorpe
2020-08-19 7:29 ` Tian, Kevin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=96a1eb5ccc724790b5404a642583919d@intel.com \
--to=megha.dey@intel.com \
--cc=alex.williamson@redhat.com \
--cc=ashok.raj@intel.com \
--cc=baolu.lu@intel.com \
--cc=bhelgaas@google.com \
--cc=dan.j.williams@intel.com \
--cc=dave.hansen@intel.com \
--cc=dave.jiang@intel.com \
--cc=dmaengine@vger.kernel.org \
--cc=eric.auger@redhat.com \
--cc=gregkh@linuxfoundation.org \
--cc=hpa@zytor.com \
--cc=jacob.jun.pan@intel.com \
--cc=jgg@mellanox.com \
--cc=jing.lin@intel.com \
--cc=kevin.tian@intel.com \
--cc=kvm@vger.kernel.org \
--cc=kwankhede@nvidia.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=maz@kernel.org \
--cc=mona.hossain@intel.com \
--cc=netanelg@mellanox.com \
--cc=parav@mellanox.com \
--cc=pbonzini@redhat.com \
--cc=rafael@kernel.org \
--cc=samuel.ortiz@intel.com \
--cc=sanjay.k.kumar@intel.com \
--cc=shahafs@mellanox.com \
--cc=tglx@linutronix.de \
--cc=tony.luck@intel.com \
--cc=vkoul@kernel.org \
--cc=x86@kernel.org \
--cc=yan.y.zhao@linux.intel.com \
--cc=yi.l.liu@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).