linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "Radovanovic, Aleksandar" <aleksandar.radovanovic@amd.com>
To: Marc Zyngier <maz@kernel.org>, Jason Gunthorpe <jgg@nvidia.com>
Cc: "Gupta, Nipun" <Nipun.Gupta@amd.com>,
	"robh+dt@kernel.org" <robh+dt@kernel.org>,
	"krzysztof.kozlowski+dt@linaro.org" 
	<krzysztof.kozlowski+dt@linaro.org>,
	"gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>,
	"rafael@kernel.org" <rafael@kernel.org>,
	"eric.auger@redhat.com" <eric.auger@redhat.com>,
	"alex.williamson@redhat.com" <alex.williamson@redhat.com>,
	"cohuck@redhat.com" <cohuck@redhat.com>,
	"Gupta, Puneet (DCG-ENG)" <puneet.gupta@amd.com>,
	"song.bao.hua@hisilicon.com" <song.bao.hua@hisilicon.com>,
	"mchehab+huawei@kernel.org" <mchehab+huawei@kernel.org>,
	"f.fainelli@gmail.com" <f.fainelli@gmail.com>,
	"jeffrey.l.hugo@gmail.com" <jeffrey.l.hugo@gmail.com>,
	"saravanak@google.com" <saravanak@google.com>,
	"Michael.Srba@seznam.cz" <Michael.Srba@seznam.cz>,
	"mani@kernel.org" <mani@kernel.org>,
	"yishaih@nvidia.com" <yishaih@nvidia.com>,
	"robin.murphy@arm.com" <robin.murphy@arm.com>,
	"will@kernel.org" <will@kernel.org>,
	"joro@8bytes.org" <joro@8bytes.org>,
	"masahiroy@kernel.org" <masahiroy@kernel.org>,
	"ndesaulniers@google.com" <ndesaulniers@google.com>,
	"linux-arm-kernel@lists.infradead.org" 
	<linux-arm-kernel@lists.infradead.org>,
	"linux-kbuild@vger.kernel.org" <linux-kbuild@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"devicetree@vger.kernel.org" <devicetree@vger.kernel.org>,
	"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
	"okaya@kernel.org" <okaya@kernel.org>,
	"Anand, Harpreet" <harpreet.anand@amd.com>,
	"Agarwal, Nikhil" <nikhil.agarwal@amd.com>,
	"Simek, Michal" <michal.simek@amd.com>,
	"git (AMD-Xilinx)" <git@amd.com>
Subject: RE: [RFC PATCH v3 4/7] bus/cdx: add cdx-MSI domain with gic-its domain as parent
Date: Wed, 7 Sep 2022 11:35:54 +0000	[thread overview]
Message-ID: <MN2PR12MB4358E3CFD2E3ECECC14471F489419@MN2PR12MB4358.namprd12.prod.outlook.com> (raw)
In-Reply-To: <87leqvv3g7.wl-maz@kernel.org>

[AMD Official Use Only - General]



> -----Original Message-----
> From: Marc Zyngier <maz@kernel.org>
> Sent: 07 September 2022 12:17
> To: Jason Gunthorpe <jgg@nvidia.com>
> Cc: Gupta, Nipun <Nipun.Gupta@amd.com>; robh+dt@kernel.org;
> krzysztof.kozlowski+dt@linaro.org; gregkh@linuxfoundation.org;
> rafael@kernel.org; eric.auger@redhat.com; alex.williamson@redhat.com;
> cohuck@redhat.com; Gupta, Puneet (DCG-ENG)
> <puneet.gupta@amd.com>; song.bao.hua@hisilicon.com;
> mchehab+huawei@kernel.org; f.fainelli@gmail.com;
> jeffrey.l.hugo@gmail.com; saravanak@google.com;
> Michael.Srba@seznam.cz; mani@kernel.org; yishaih@nvidia.com;
> robin.murphy@arm.com; will@kernel.org; joro@8bytes.org;
> masahiroy@kernel.org; ndesaulniers@google.com; linux-arm-
> kernel@lists.infradead.org; linux-kbuild@vger.kernel.org; linux-
> kernel@vger.kernel.org; devicetree@vger.kernel.org; kvm@vger.kernel.org;
> okaya@kernel.org; Anand, Harpreet <harpreet.anand@amd.com>; Agarwal,
> Nikhil <nikhil.agarwal@amd.com>; Simek, Michal <michal.simek@amd.com>;
> Radovanovic, Aleksandar <aleksandar.radovanovic@amd.com>; git (AMD-
> Xilinx) <git@amd.com>
> Subject: Re: [RFC PATCH v3 4/7] bus/cdx: add cdx-MSI domain with gic-its
> domain as parent
> 
> [CAUTION: External Email]
> 
> On Tue, 06 Sep 2022 18:19:06 +0100,
> Jason Gunthorpe <jgg@nvidia.com> wrote:
> >
> > On Tue, Sep 06, 2022 at 07:17:58PM +0530, Nipun Gupta wrote:
> >
> > > +static void cdx_msi_write_msg(struct irq_data *irq_data,
> > > +                         struct msi_msg *msg) {
> > > +   /*
> > > +    * Do nothing as CDX devices have these pre-populated
> > > +    * in the hardware itself.
> > > +    */
> > > +}
> >
> > Huh?
> >
> > There is no way it can be pre-populated, the addr/data pair,
> > especially on ARM, is completely under SW control.
> 
> There is nothing in the GIC spec that says that.
> 
> > There is some commonly used IOVA base in Linux for the ITS page, but
> > no HW should hardwire that.
> 
> That's not strictly true. It really depends on how this block is integrated, and
> there is a number of existing blocks that know *in HW* how to signal an LPI.
> 
> See, as the canonical example, how the mbigen driver doesn't need to know
> about the address of GITS_TRANSLATER.
> 
> Yes, this messes with translation (the access is downstream of the
> SMMU) if you relied on it to have some isolation, and it has a "black hole"
> effect as nobody can have an IOVA that overlaps with the physical address of
> the GITS_TRANSLATER register.
> 
> But is it illegal as per the architecture? No. It's just stupid.
> 
>         M.
> 
> --
> Without deviation from the norm, progress is not possible.

To give some context, CDX devices are specific to embedded ARM CPUs on the FPGA and a lot of the CDX hardware core is under the control of the system firmware, not the application CPUs. 

That being said, the MSI address is always going to be the GIC GITS_TRANSLATER, which is known to the system firmware, as it is fixed per FPGA platform. At present, we do not allow the application CPU OS to change this - I believe this is for security reasons, but this may or may not be a good idea in general. As Marc mentions, CDX MSI writes are downstream of the SMMU and, if SMMU does not provide identity mapping for GITS_TRANSLATER, then we have a problem and may need to allow the OS to write the address part. However, even if we did, the CDX hardware is limited in that it can only take one GITS_TRANSLATER register target address per system, not per CDX device, nor per MSI vector.

As for the data part (EventID in GIC parlance), this is always going to be the CDX device-relative vector number - I believe this can't be changed, it is a hardware limitation (but I need to double-check). That should be OK, though, as I believe this is exactly what Linux would write anyway, as each CDX device should be in its own IRQ domain (i.e. have its own ITS device table).

The best I can propose is to pass the addr/data info to firmware here, which will then decide what to do with it. At least, it can assert that the values are what the hardware expects and fail loudly if not, rather than having a silently misconfigured system.

Aleksandar

  parent reply	other threads:[~2022-09-07 11:37 UTC|newest]

Thread overview: 113+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-08-03 12:26 [RFC PATCH 0/2] add support for CDX bus MSI domain Nipun Gupta
2022-08-03 12:26 ` [RFC PATCH 1/2] irqchip: cdx-bus: add cdx-MSI domain with gic-its domain as parent Nipun Gupta
2022-08-03 12:33   ` Greg KH
2022-08-03 12:37     ` Gupta, Nipun
2022-08-04  8:49   ` Marc Zyngier
2022-08-04  9:18     ` Gupta, Nipun
     [not found]       ` <87mtckwaxa.wl-maz@kernel.org>
2022-08-04 12:11         ` Gupta, Nipun
2022-08-03 12:26 ` [RFC PATCH 2/2] driver core: add compatible string in sysfs for platform devices Nipun Gupta
2022-08-03 12:31   ` Greg KH
2022-08-03 12:46     ` Gupta, Nipun
2022-08-03 14:16 ` [RFC PATCH 0/2] add support for CDX bus MSI domain Robin Murphy
2022-08-04  4:23   ` Gupta, Nipun
2022-08-17 16:00   ` Jason Gunthorpe
2022-08-17 15:05 ` [RFC PATCH v2 0/6] add support for CDX bus controller Nipun Gupta
2022-08-17 15:05   ` [RFC PATCH v2 1/6] Documentation: DT: Add entry for CDX controller Nipun Gupta
2022-08-18  9:54     ` Krzysztof Kozlowski
2022-08-18  9:59       ` Krzysztof Kozlowski
2022-09-05 14:05       ` Gupta, Nipun
2022-09-06  6:55         ` Krzysztof Kozlowski
2022-09-06  7:03           ` Gupta, Nipun
2022-09-06  7:20             ` Krzysztof Kozlowski
2022-08-17 15:05   ` [RFC PATCH v2 2/6] bus/cdx: add the cdx bus driver Nipun Gupta
2022-08-17 15:32     ` Greg KH
2022-08-17 15:46       ` Marc Zyngier
2022-08-22 13:21       ` Gupta, Nipun
2022-08-22 13:29         ` Greg KH
2022-08-24  8:50           ` Gupta, Nipun
2022-08-24 12:11             ` Greg KH
2022-08-24 23:31               ` Jason Gunthorpe
2022-08-25 18:38                 ` Saravana Kannan
2022-08-25 19:57                   ` Robin Murphy
2022-08-26  0:08                     ` Jason Gunthorpe
2022-08-29  4:49                       ` Gupta, Nipun
2022-08-29  4:49                     ` Gupta, Nipun
2022-08-29 15:31                       ` Jason Gunthorpe
2022-08-30  7:06                         ` Gupta, Nipun
2022-08-30 11:25                           ` Robin Murphy
2022-08-30 13:01                           ` Jason Gunthorpe
2022-08-30 13:12                             ` Gupta, Nipun
2022-08-17 15:05   ` [RFC PATCH v2 3/6] bus/cdx: add cdx-MSI domain with gic-its domain as parent Nipun Gupta
2022-08-17 15:33     ` Greg KH
2022-08-17 15:05   ` [RFC PATCH v2 4/6] bus/cdx: add rescan and reset support Nipun Gupta
2022-08-17 15:05   ` [RFC PATCH v2 5/6] vfio: platform: reset: add reset for cdx devices Nipun Gupta
2022-08-17 15:05   ` [RFC PATCH v2 6/6] driver core: add compatible string in sysfs for platform devices Nipun Gupta
2022-08-17 15:30     ` Greg KH
2022-08-17 16:04       ` Saravana Kannan
2022-09-05 13:54         ` Gupta, Nipun
2022-09-06 13:47 ` [RFC PATCH v3 0/7] add support for CDX bus Nipun Gupta
2022-09-06 13:47   ` [RFC PATCH v3 1/7] dt-bindings: bus: add CDX bus device tree bindings Nipun Gupta
2022-09-06 17:46     ` Rob Herring
2022-09-07  3:13       ` Gupta, Nipun
2022-09-08 10:51         ` Krzysztof Kozlowski
2022-09-06 13:47   ` [RFC PATCH v3 2/7] bus/cdx: add the cdx bus driver Nipun Gupta
2022-09-07  0:32     ` Saravana Kannan
2022-09-07  3:21       ` Gupta, Nipun
2022-09-07 18:06         ` Saravana Kannan
2022-09-07 12:32     ` Greg KH
2022-09-08 13:29       ` Gupta, Nipun
2022-09-06 13:47   ` [RFC PATCH v3 3/7] iommu/arm-smmu-v3: support ops registration for CDX bus Nipun Gupta
2022-09-07  0:10     ` Saravana Kannan
2022-09-07  3:17       ` Gupta, Nipun
2022-09-07  8:27         ` Robin Murphy
2022-09-07 18:24           ` Saravana Kannan
2022-09-07 20:40             ` Robin Murphy
2022-09-08  0:14               ` Saravana Kannan
2022-09-06 13:47   ` [RFC PATCH v3 4/7] bus/cdx: add cdx-MSI domain with gic-its domain as parent Nipun Gupta
2022-09-06 17:19     ` Jason Gunthorpe
2022-09-07 11:17       ` Marc Zyngier
2022-09-07 11:33         ` Robin Murphy
2022-09-07 12:14           ` Marc Zyngier
2022-09-07 11:35         ` Radovanovic, Aleksandar [this message]
     [not found]           ` <87illzuzyw.wl-maz@kernel.org>
2022-09-07 13:18             ` Radovanovic, Aleksandar
     [not found]               ` <87edwmuw4f.wl-maz@kernel.org>
2022-09-08  9:51                 ` Radovanovic, Aleksandar
2022-09-08 11:49                   ` Robin Murphy
2022-09-07 13:18     ` Marc Zyngier
2022-09-08 14:13       ` Gupta, Nipun
     [not found]         ` <87a67aueg7.wl-maz@kernel.org>
2022-09-09  6:32           ` Gupta, Nipun
2022-10-12 10:04       ` Gupta, Nipun
2022-10-12 10:34         ` Radovanovic, Aleksandar
2022-10-12 13:02           ` Jason Gunthorpe
2022-10-12 13:37             ` Radovanovic, Aleksandar
2022-10-12 14:38               ` Jason Gunthorpe
2022-10-12 15:09                 ` Radovanovic, Aleksandar
2022-10-13 12:43                   ` Jason Gunthorpe
2022-10-14 11:18                     ` Radovanovic, Aleksandar
2022-10-14 11:54                       ` gregkh
2022-10-14 12:13                         ` Radovanovic, Aleksandar
2022-10-14 13:46                           ` gregkh
2022-10-14 13:58                       ` Jason Gunthorpe
2022-09-06 13:47   ` [RFC PATCH v3 5/7] bus/cdx: add bus and device attributes Nipun Gupta
2022-09-06 13:48   ` [RFC PATCH v3 6/7] vfio/cdx: add support for CDX bus Nipun Gupta
2022-09-06 17:20     ` Jason Gunthorpe
2022-09-06 17:23       ` Gupta, Nipun
2022-09-06 13:48   ` [RFC PATCH v3 7/7] vfio/cdx: add interrupt support Nipun Gupta
2022-10-14  4:40 ` [RFC PATCH v4 0/8] add support for CDX bus Nipun Gupta
2022-10-14  4:40   ` [RFC PATCH v4 1/8] dt-bindings: bus: add CDX bus device tree bindings Nipun Gupta
2022-10-14 14:17     ` Rob Herring
2022-10-17 10:18       ` Gupta, Nipun
2022-10-14  4:40   ` [RFC PATCH v4 2/8] bus/cdx: add the cdx bus driver Nipun Gupta
2022-10-14  7:15     ` Greg KH
2022-10-14  8:12       ` Gupta, Nipun
2022-10-14  7:18     ` Greg KH
2022-10-14  8:20       ` Gupta, Nipun
2022-10-14  4:40   ` [RFC PATCH v4 3/8] iommu/arm-smmu-v3: support ops registration for CDX bus Nipun Gupta
2022-10-14  4:51     ` Gupta, Nipun
2022-10-14  4:40   ` [RFC PATCH v4 4/8] bux/cdx: support dma configuration for CDX devices Nipun Gupta
2022-10-14  4:40   ` [RFC PATCH v4 5/8] bus/cdx: add bus and device attributes Nipun Gupta
2022-10-14  4:40   ` [RFC PATCH v4 6/8] irq/msi: use implicit msi domain for alloc and free Nipun Gupta
2022-10-14  4:40   ` [RFC PATCH v4 7/8] bus/cdx: add cdx-MSI domain with gic-its domain as parent Nipun Gupta
2022-11-17 19:10     ` Thomas Gleixner
2022-10-14  4:40   ` [RFC PATCH v4 8/8] bus/cdx: add cdx controller Nipun Gupta
2022-10-14 14:10   ` [RFC PATCH v4 0/8] add support for CDX bus Rob Herring
2022-10-17 10:08     ` Gupta, Nipun

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=MN2PR12MB4358E3CFD2E3ECECC14471F489419@MN2PR12MB4358.namprd12.prod.outlook.com \
    --to=aleksandar.radovanovic@amd.com \
    --cc=Michael.Srba@seznam.cz \
    --cc=Nipun.Gupta@amd.com \
    --cc=alex.williamson@redhat.com \
    --cc=cohuck@redhat.com \
    --cc=devicetree@vger.kernel.org \
    --cc=eric.auger@redhat.com \
    --cc=f.fainelli@gmail.com \
    --cc=git@amd.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=harpreet.anand@amd.com \
    --cc=jeffrey.l.hugo@gmail.com \
    --cc=jgg@nvidia.com \
    --cc=joro@8bytes.org \
    --cc=krzysztof.kozlowski+dt@linaro.org \
    --cc=kvm@vger.kernel.org \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kbuild@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mani@kernel.org \
    --cc=masahiroy@kernel.org \
    --cc=maz@kernel.org \
    --cc=mchehab+huawei@kernel.org \
    --cc=michal.simek@amd.com \
    --cc=ndesaulniers@google.com \
    --cc=nikhil.agarwal@amd.com \
    --cc=okaya@kernel.org \
    --cc=puneet.gupta@amd.com \
    --cc=rafael@kernel.org \
    --cc=robh+dt@kernel.org \
    --cc=robin.murphy@arm.com \
    --cc=saravanak@google.com \
    --cc=song.bao.hua@hisilicon.com \
    --cc=will@kernel.org \
    --cc=yishaih@nvidia.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).