Re: [patch 33/33] irqchip: Add IDXD Interrupt Message Store driver

From: Thomas Gleixner <tglx@linutronix.de>
To: Reinette Chatre <reinette.chatre@intel.com>,
	LKML <linux-kernel@vger.kernel.org>
Cc: x86@kernel.org, Joerg Roedel <joro@8bytes.org>,
	Will Deacon <will@kernel.org>,
	linux-pci@vger.kernel.org, Bjorn Helgaas <bhelgaas@google.com>,
	Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>,
	Marc Zyngier <maz@kernel.org>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Jason Gunthorpe <jgg@mellanox.com>,
	Dave Jiang <dave.jiang@intel.com>,
	Alex Williamson <alex.williamson@redhat.com>,
	Kevin Tian <kevin.tian@intel.com>,
	Dan Williams <dan.j.williams@intel.com>,
	Logan Gunthorpe <logang@deltatee.com>,
	Ashok Raj <ashok.raj@intel.com>, Jon Mason <jdmason@kudzu.us>,
	Allen Hubbe <allenbh@gmail.com>,
	"Ahmed S. Darwish" <darwi@linutronix.de>
Subject: Re: [patch 33/33] irqchip: Add IDXD Interrupt Message Store driver
Date: Fri, 02 Dec 2022 20:51:45 +0100	[thread overview]
Message-ID: <87sfhxoa7i.ffs@tglx> (raw)
In-Reply-To: <4a15c569-0545-20ac-e74c-ae17f7eb067d@intel.com>

Reinette!

On Fri, Dec 02 2022 at 09:55, Reinette Chatre wrote:
> On 11/11/2022 5:59 AM, Thomas Gleixner wrote:
>> The necessary iobase is stored in the irqdomain and the PASID which is
>> required for operation is handed in via msi_dev_cookie in the allocation
>> function.
>
> The use of PASID is optional for dedicated workqueues. Could this be
> supported to let the irqchip support all scenarios?

Sure. I wrote this thing mostly out of thin air based on some ancient
PoC code. :)

> Since the cookie is always provided I was wondering if an invalid
> PASID can be used to let the driver disable PASID? Please see the
> delta snippet below in which I primarily made such a change, but added
> a few more changes for consideration.

Let me check.

> With the first change I am able to test IMS on the host using devmsi-v2-part3
> of the development branch. I did try to update to the most recent development
> to confirm all is well but version devmsi-v3.1-part3 behaves differently
> in that pci_ims_alloc_irq() returns successfully but the returned
> virq is 0. This triggers a problem when request_threaded_irq() runs and
> reports:
> genirq: Flags mismatch irq 0. 00000000 (idxd-portal) vs. 00015a00 (timer)

Bah. Let me figure out what I fat-fingered there.

> @@ -33,6 +34,8 @@ struct ims_slot {
>  #define CTRL_PASID_ENABLE	BIT(3)
>  /* Position of PASID.LSB in the control word */
>  #define CTRL_PASID_SHIFT	12
> +/* Valid PASID is 20 bits */
> +#define CTRL_PASID_VALID	GENMASK(19, 0)
>  
>  static inline void iowrite32_and_flush(u32 value, void __iomem *addr)
>  {
> @@ -93,12 +96,17 @@ static void idxd_prepare_desc(struct irq_domain *domain, msi_alloc_info_t *arg,
>  	/* Mask the interrupt for paranoia sake */
>  	iowrite32_and_flush(CTRL_VECTOR_MASKBIT, &slot->ctrl);
>  
> -	/*
> -	 * The caller provided PASID. Shift it to the proper position
> -	 * and set the PASID enable bit.
> -	 */
> -	desc->data.icookie.value <<= CTRL_PASID_SHIFT;
> -	desc->data.icookie.value |= CTRL_PASID_ENABLE;
> +	if (pasid_valid((ioasid_t)desc->data.icookie.value)) {
> +		/*
> +		 * The caller provided PASID. Shift it to the proper position
> +		 * and set the PASID enable bit.
> +		 */
> +		desc->data.icookie.value &= CTRL_PASID_VALID;
> +		desc->data.icookie.value <<= CTRL_PASID_SHIFT;
> +		desc->data.icookie.value |= CTRL_PASID_ENABLE;
> +	} else {
> +		desc->data.icookie.value = 0;
> +	}

Looks about right. But that needs some sanity measures at the call sites
so that we don't end up with an invalid PASID in cases where a valid
PASID is truly required.

Thanks,

        tglx