Re: Programming PASID in IMS entries

From: Thomas Gleixner <tglx@linutronix.de>
To: "Raj\, Ashok" <ashok.raj@intel.com>, Jason Gunthorpe <jgg@nvidia.com>
Cc: "Dey\, Megha" <megha.dey@intel.com>,
	linux-kernel@vger.kernel.org, "Jiang\,
	Dave" <dave.jiang@intel.com>, "Tian\,
	Kevin" <kevin.tian@intel.com>, "Pan\,
	Jacob jun" <jacob.jun.pan@intel.com>, "Liu\,
	Yi L" <yi.l.liu@intel.com>, "Kumar\,
	Sanjay K" <sanjay.k.kumar@intel.com>, "Van De Ven\,
	Arjan" <arjan.van.de.ven@intel.com>, "Williams\,
	Dan J" <dan.j.williams@intel.com>, "Shankar\,
	Ravi V" <ravi.v.shankar@intel.com>,
	Ashok Raj <ashok.raj@intel.com>
Subject: Re: Programming PASID in IMS entries
Date: Thu, 08 Jul 2021 20:45:48 +0200	[thread overview]
Message-ID: <87v95kod9f.ffs@nanos.tec.linutronix.de> (raw)
In-Reply-To: <20210708143657.GA70042@otc-nc-03>

Ashok,

On Thu, Jul 08 2021 at 07:36, Ashok Raj wrote:
> On Thu, Jul 08, 2021 at 09:08:46AM -0300, Jason Gunthorpe wrote:
>> On Wed, Jul 07, 2021 at 05:33:35PM -0700, Raj, Ashok wrote:
>> > On Wed, Jul 07, 2021 at 08:58:22PM -0300, Jason Gunthorpe wrote:
>> > > > Using default PASID in struct device will work for sub-devices until the
>> > > > guest needs to enable ENQCMD support. Since the guest kernel can ask for an
>> > > > interrupt by specifying something in the descriptor submitted via ENQCMD.
>> > > > Using the PASID in struct device won't be sufficient.
>> > > 
>> > > Could you could store a pasid table in the struct device and index it
>> > > by vector?
>> > 
>> > Possibly... what ever Thomas things is clean. The device specific driver
>> > would have this already. So providing some call to get this filled in vs
>> > storing that in struct device. Someone close at heart to the driver model
>> > is best to comment :-)
>> > 
>> > IMS core owns the format of the entries right now vs device specific driver. 
>> > I suppose your use case requiring a vm_id might have a different format. 
>> > So this is yet another one the core needs to learn and adapt?
>> 
>> All entry format stuff is device specific, it shouldn't be in "core"
>> code.
>
> Well, this is how it started way back last year. 
>
> https://lore.kernel.org/lkml/158751209583.36773.15917761221672315662.stgit@djiang5-desk3.ch.intel.com/

Which is wrong on so many levels as we all know.

> Where the driver functions for mask/unmask/write_msg etc. So the core
> needs

Needs what?

> So the format or layout is device specific, but core can dictate the exact
> message that needs to be written.

Sorry, I don't grok what you want to say here.

>> It is is the same reason that the IRQ chip driver for IDXD should have
>> IDXD in the name, it is not a generic "IMS core" thing.
>> 
>> The question mark is probably the locking model, but if IDXD
>> guarentees the pasid table doesn't change while the irq is active then
>> maybe it works out well enough.
>
> I think this must be gauranteed at a min? changing things underneath when
> the interrupts are unmasked would be bad usage.

That's one way to look at it. OTOH, _if_ the association of some
arbitrary information to interrupts becomes a common scheme, then we are
surely better off to have some enforcement at the irq core level.

Thanks,

        tglx