From: Jason Gunthorpe <jgg@ziepe.ca>
To: "Radovanovic, Aleksandar" <aleksandar.radovanovic@amd.com>
Cc: "Gupta, Nipun" <Nipun.Gupta@amd.com>,
Marc Zyngier <maz@kernel.org>,
Robin Murphy <robin.murphy@arm.com>,
"robh+dt@kernel.org" <robh+dt@kernel.org>,
"krzysztof.kozlowski+dt@linaro.org"
<krzysztof.kozlowski+dt@linaro.org>,
"gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>,
"rafael@kernel.org" <rafael@kernel.org>,
"eric.auger@redhat.com" <eric.auger@redhat.com>,
"alex.williamson@redhat.com" <alex.williamson@redhat.com>,
"cohuck@redhat.com" <cohuck@redhat.com>,
"Gupta, Puneet (DCG-ENG)" <puneet.gupta@amd.com>,
"song.bao.hua@hisilicon.com" <song.bao.hua@hisilicon.com>,
"mchehab+huawei@kernel.org" <mchehab+huawei@kernel.org>,
"f.fainelli@gmail.com" <f.fainelli@gmail.com>,
"jeffrey.l.hugo@gmail.com" <jeffrey.l.hugo@gmail.com>,
"saravanak@google.com" <saravanak@google.com>,
"Michael.Srba@seznam.cz" <Michael.Srba@seznam.cz>,
"mani@kernel.org" <mani@kernel.org>,
"yishaih@nvidia.com" <yishaih@nvidia.com>,
"will@kernel.org" <will@kernel.org>,
"joro@8bytes.org" <joro@8bytes.org>,
"masahiroy@kernel.org" <masahiroy@kernel.org>,
"ndesaulniers@google.com" <ndesaulniers@google.com>,
"linux-arm-kernel@lists.infradead.org"
<linux-arm-kernel@lists.infradead.org>,
"linux-kbuild@vger.kernel.org" <linux-kbuild@vger.kernel.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"devicetree@vger.kernel.org" <devicetree@vger.kernel.org>,
"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
"okaya@kernel.org" <okaya@kernel.org>,
"Anand, Harpreet" <harpreet.anand@amd.com>,
"Agarwal, Nikhil" <nikhil.agarwal@amd.com>,
"Simek, Michal" <michal.simek@amd.com>,
"git (AMD-Xilinx)" <git@amd.com>
Subject: Re: [RFC PATCH v3 4/7] bus/cdx: add cdx-MSI domain with gic-its domain as parent
Date: Fri, 14 Oct 2022 10:58:10 -0300 [thread overview]
Message-ID: <Y0lq8vB2PT0zKUAQ@ziepe.ca> (raw)
In-Reply-To: <MN2PR12MB4358CF6B6D2576B35E39328A89249@MN2PR12MB4358.namprd12.prod.outlook.com>
On Fri, Oct 14, 2022 at 11:18:36AM +0000, Radovanovic, Aleksandar wrote:
> And that still does not imply lack of ordering or sharing of MSI
> target addresses between devices.
Either the end point generates the MSI, and maybe the bridge mangles
it, or it raises a lot of suspicion that this is not right. If the end
point generates the MSI then it raises the question why do we need to
tolerate these limits?
> This is a highly programmable IP block, at the core of which is an
> interconnect interfacing to programmable logic (PL), a number of
> PCIe controllers (either endpoint or root-port), DMA engines,
> offload engines, the embedded processor subsystem (PSX), etc. DMA
> and interrupts can be routed across it in almost any (meaningful)
> direction. The datapath 'endpoints' request DMA and interrupts, but
> don't concern themselves with the mechanics of delivering that in
> the target domain. It is the responsibility of the egress bridges to
> the target domains to convert the interconnect interrupt
> transactions to whatever the interrupt delivery mechanism for that
> domain is. E.g. for PCIe controllers in endpoint mode, that would be
> through PCIe MSI-X tables internal to the controller (and managed by
> the PCIe host), for PSX that would be the PSX bridge (partially
> managed by the PSX OS, mediated through firmware, i.e. through CDX
> bus driver) and so on. It is the responsibility of the interconnect
> to maintain transaction ordering (including DMA vs. interrupts). It
> is the responsibility of the firmware to manage the bridges
> according to the implemented use-case, so everything works as
> expected.
Again, this all just seems wrongly designed. MSI should not be part
of an interconnect bridge. We did that already 20 years ago, it was
called IOAPICs on x86 and I think everyone is happy to see it gone.
If you want to build IOAPICs again, I guess you can, but that is a
slightly different SW setup than the MSI you are trying to use here,
and even that didn't have the same limitations you are proposing.
> So, yes, the hardware that translates interrupt transactions to GIC
> AXI writes is shared between endpoints, but what I said above still
> applies. And that doesn't necessarily make it weird/wrong, it's just
> more complex than you might think.
If it doesn't fit the architecture, then I think it must be considered
wrong. Mis-using platform architected components like MSI in HW is
problematic.
You should design the HW properly so you don't have these
problems. Involving FW in the MSI setup is also a bad idea, POWER did
this and it made a big mess of their arch code :(
Jason
next prev parent reply other threads:[~2022-10-14 14:00 UTC|newest]
Thread overview: 113+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-08-03 12:26 [RFC PATCH 0/2] add support for CDX bus MSI domain Nipun Gupta
2022-08-03 12:26 ` [RFC PATCH 1/2] irqchip: cdx-bus: add cdx-MSI domain with gic-its domain as parent Nipun Gupta
2022-08-03 12:33 ` Greg KH
2022-08-03 12:37 ` Gupta, Nipun
2022-08-04 8:49 ` Marc Zyngier
2022-08-04 9:18 ` Gupta, Nipun
[not found] ` <87mtckwaxa.wl-maz@kernel.org>
2022-08-04 12:11 ` Gupta, Nipun
2022-08-03 12:26 ` [RFC PATCH 2/2] driver core: add compatible string in sysfs for platform devices Nipun Gupta
2022-08-03 12:31 ` Greg KH
2022-08-03 12:46 ` Gupta, Nipun
2022-08-03 14:16 ` [RFC PATCH 0/2] add support for CDX bus MSI domain Robin Murphy
2022-08-04 4:23 ` Gupta, Nipun
2022-08-17 16:00 ` Jason Gunthorpe
2022-08-17 15:05 ` [RFC PATCH v2 0/6] add support for CDX bus controller Nipun Gupta
2022-08-17 15:05 ` [RFC PATCH v2 1/6] Documentation: DT: Add entry for CDX controller Nipun Gupta
2022-08-18 9:54 ` Krzysztof Kozlowski
2022-08-18 9:59 ` Krzysztof Kozlowski
2022-09-05 14:05 ` Gupta, Nipun
2022-09-06 6:55 ` Krzysztof Kozlowski
2022-09-06 7:03 ` Gupta, Nipun
2022-09-06 7:20 ` Krzysztof Kozlowski
2022-08-17 15:05 ` [RFC PATCH v2 2/6] bus/cdx: add the cdx bus driver Nipun Gupta
2022-08-17 15:32 ` Greg KH
2022-08-17 15:46 ` Marc Zyngier
2022-08-22 13:21 ` Gupta, Nipun
2022-08-22 13:29 ` Greg KH
2022-08-24 8:50 ` Gupta, Nipun
2022-08-24 12:11 ` Greg KH
2022-08-24 23:31 ` Jason Gunthorpe
2022-08-25 18:38 ` Saravana Kannan
2022-08-25 19:57 ` Robin Murphy
2022-08-26 0:08 ` Jason Gunthorpe
2022-08-29 4:49 ` Gupta, Nipun
2022-08-29 4:49 ` Gupta, Nipun
2022-08-29 15:31 ` Jason Gunthorpe
2022-08-30 7:06 ` Gupta, Nipun
2022-08-30 11:25 ` Robin Murphy
2022-08-30 13:01 ` Jason Gunthorpe
2022-08-30 13:12 ` Gupta, Nipun
2022-08-17 15:05 ` [RFC PATCH v2 3/6] bus/cdx: add cdx-MSI domain with gic-its domain as parent Nipun Gupta
2022-08-17 15:33 ` Greg KH
2022-08-17 15:05 ` [RFC PATCH v2 4/6] bus/cdx: add rescan and reset support Nipun Gupta
2022-08-17 15:05 ` [RFC PATCH v2 5/6] vfio: platform: reset: add reset for cdx devices Nipun Gupta
2022-08-17 15:05 ` [RFC PATCH v2 6/6] driver core: add compatible string in sysfs for platform devices Nipun Gupta
2022-08-17 15:30 ` Greg KH
2022-08-17 16:04 ` Saravana Kannan
2022-09-05 13:54 ` Gupta, Nipun
2022-09-06 13:47 ` [RFC PATCH v3 0/7] add support for CDX bus Nipun Gupta
2022-09-06 13:47 ` [RFC PATCH v3 1/7] dt-bindings: bus: add CDX bus device tree bindings Nipun Gupta
2022-09-06 17:46 ` Rob Herring
2022-09-07 3:13 ` Gupta, Nipun
2022-09-08 10:51 ` Krzysztof Kozlowski
2022-09-06 13:47 ` [RFC PATCH v3 2/7] bus/cdx: add the cdx bus driver Nipun Gupta
2022-09-07 0:32 ` Saravana Kannan
2022-09-07 3:21 ` Gupta, Nipun
2022-09-07 18:06 ` Saravana Kannan
2022-09-07 12:32 ` Greg KH
2022-09-08 13:29 ` Gupta, Nipun
2022-09-06 13:47 ` [RFC PATCH v3 3/7] iommu/arm-smmu-v3: support ops registration for CDX bus Nipun Gupta
2022-09-07 0:10 ` Saravana Kannan
2022-09-07 3:17 ` Gupta, Nipun
2022-09-07 8:27 ` Robin Murphy
2022-09-07 18:24 ` Saravana Kannan
2022-09-07 20:40 ` Robin Murphy
2022-09-08 0:14 ` Saravana Kannan
2022-09-06 13:47 ` [RFC PATCH v3 4/7] bus/cdx: add cdx-MSI domain with gic-its domain as parent Nipun Gupta
2022-09-06 17:19 ` Jason Gunthorpe
2022-09-07 11:17 ` Marc Zyngier
2022-09-07 11:33 ` Robin Murphy
2022-09-07 12:14 ` Marc Zyngier
2022-09-07 11:35 ` Radovanovic, Aleksandar
[not found] ` <87illzuzyw.wl-maz@kernel.org>
2022-09-07 13:18 ` Radovanovic, Aleksandar
[not found] ` <87edwmuw4f.wl-maz@kernel.org>
2022-09-08 9:51 ` Radovanovic, Aleksandar
2022-09-08 11:49 ` Robin Murphy
2022-09-07 13:18 ` Marc Zyngier
2022-09-08 14:13 ` Gupta, Nipun
[not found] ` <87a67aueg7.wl-maz@kernel.org>
2022-09-09 6:32 ` Gupta, Nipun
2022-10-12 10:04 ` Gupta, Nipun
2022-10-12 10:34 ` Radovanovic, Aleksandar
2022-10-12 13:02 ` Jason Gunthorpe
2022-10-12 13:37 ` Radovanovic, Aleksandar
2022-10-12 14:38 ` Jason Gunthorpe
2022-10-12 15:09 ` Radovanovic, Aleksandar
2022-10-13 12:43 ` Jason Gunthorpe
2022-10-14 11:18 ` Radovanovic, Aleksandar
2022-10-14 11:54 ` gregkh
2022-10-14 12:13 ` Radovanovic, Aleksandar
2022-10-14 13:46 ` gregkh
2022-10-14 13:58 ` Jason Gunthorpe [this message]
2022-09-06 13:47 ` [RFC PATCH v3 5/7] bus/cdx: add bus and device attributes Nipun Gupta
2022-09-06 13:48 ` [RFC PATCH v3 6/7] vfio/cdx: add support for CDX bus Nipun Gupta
2022-09-06 17:20 ` Jason Gunthorpe
2022-09-06 17:23 ` Gupta, Nipun
2022-09-06 13:48 ` [RFC PATCH v3 7/7] vfio/cdx: add interrupt support Nipun Gupta
2022-10-14 4:40 ` [RFC PATCH v4 0/8] add support for CDX bus Nipun Gupta
2022-10-14 4:40 ` [RFC PATCH v4 1/8] dt-bindings: bus: add CDX bus device tree bindings Nipun Gupta
2022-10-14 14:17 ` Rob Herring
2022-10-17 10:18 ` Gupta, Nipun
2022-10-14 4:40 ` [RFC PATCH v4 2/8] bus/cdx: add the cdx bus driver Nipun Gupta
2022-10-14 7:15 ` Greg KH
2022-10-14 8:12 ` Gupta, Nipun
2022-10-14 7:18 ` Greg KH
2022-10-14 8:20 ` Gupta, Nipun
2022-10-14 4:40 ` [RFC PATCH v4 3/8] iommu/arm-smmu-v3: support ops registration for CDX bus Nipun Gupta
2022-10-14 4:51 ` Gupta, Nipun
2022-10-14 4:40 ` [RFC PATCH v4 4/8] bux/cdx: support dma configuration for CDX devices Nipun Gupta
2022-10-14 4:40 ` [RFC PATCH v4 5/8] bus/cdx: add bus and device attributes Nipun Gupta
2022-10-14 4:40 ` [RFC PATCH v4 6/8] irq/msi: use implicit msi domain for alloc and free Nipun Gupta
2022-10-14 4:40 ` [RFC PATCH v4 7/8] bus/cdx: add cdx-MSI domain with gic-its domain as parent Nipun Gupta
2022-11-17 19:10 ` Thomas Gleixner
2022-10-14 4:40 ` [RFC PATCH v4 8/8] bus/cdx: add cdx controller Nipun Gupta
2022-10-14 14:10 ` [RFC PATCH v4 0/8] add support for CDX bus Rob Herring
2022-10-17 10:08 ` Gupta, Nipun
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Y0lq8vB2PT0zKUAQ@ziepe.ca \
--to=jgg@ziepe.ca \
--cc=Michael.Srba@seznam.cz \
--cc=Nipun.Gupta@amd.com \
--cc=aleksandar.radovanovic@amd.com \
--cc=alex.williamson@redhat.com \
--cc=cohuck@redhat.com \
--cc=devicetree@vger.kernel.org \
--cc=eric.auger@redhat.com \
--cc=f.fainelli@gmail.com \
--cc=git@amd.com \
--cc=gregkh@linuxfoundation.org \
--cc=harpreet.anand@amd.com \
--cc=jeffrey.l.hugo@gmail.com \
--cc=joro@8bytes.org \
--cc=krzysztof.kozlowski+dt@linaro.org \
--cc=kvm@vger.kernel.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kbuild@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mani@kernel.org \
--cc=masahiroy@kernel.org \
--cc=maz@kernel.org \
--cc=mchehab+huawei@kernel.org \
--cc=michal.simek@amd.com \
--cc=ndesaulniers@google.com \
--cc=nikhil.agarwal@amd.com \
--cc=okaya@kernel.org \
--cc=puneet.gupta@amd.com \
--cc=rafael@kernel.org \
--cc=robh+dt@kernel.org \
--cc=robin.murphy@arm.com \
--cc=saravanak@google.com \
--cc=song.bao.hua@hisilicon.com \
--cc=will@kernel.org \
--cc=yishaih@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).