From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D25A2C07E95 for ; Thu, 8 Jul 2021 00:34:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B2BFA61CB9 for ; Thu, 8 Jul 2021 00:34:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230150AbhGHAg4 (ORCPT ); Wed, 7 Jul 2021 20:36:56 -0400 Received: from mga02.intel.com ([134.134.136.20]:40961 "EHLO mga02.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230000AbhGHAgz (ORCPT ); Wed, 7 Jul 2021 20:36:55 -0400 X-IronPort-AV: E=McAfee;i="6200,9189,10038"; a="196587808" X-IronPort-AV: E=Sophos;i="5.84,222,1620716400"; d="scan'208";a="196587808" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Jul 2021 17:34:02 -0700 X-IronPort-AV: E=Sophos;i="5.84,222,1620716400"; d="scan'208";a="498174309" Received: from otc-nc-03.jf.intel.com (HELO otc-nc-03) ([10.54.39.36]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Jul 2021 17:34:01 -0700 Date: Wed, 7 Jul 2021 17:33:35 -0700 From: "Raj, Ashok" To: Jason Gunthorpe Cc: Thomas Gleixner , "Dey, Megha" , linux-kernel@vger.kernel.org, "Jiang, Dave" , "Tian, Kevin" , "Pan, Jacob jun" , "Liu, Yi L" , "Kumar, Sanjay K" , "Van De Ven, Arjan" , "Williams, Dan J" , "Shankar, Ravi V" , Ashok Raj Subject: Re: Programming PASID in IMS entries Message-ID: <20210708003335.GC56594@otc-nc-03> References: <87k0m2qzgz.ffs@nanos.tec.linutronix.de> <20210707221216.GA56594@otc-nc-03> <20210707235822.GB4459@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20210707235822.GB4459@nvidia.com> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Jul 07, 2021 at 08:58:22PM -0300, Jason Gunthorpe wrote: > On Wed, Jul 07, 2021 at 03:12:16PM -0700, Raj, Ashok wrote: > > Hi Thomas > > > > On Wed, Jul 07, 2021 at 10:50:52AM +0200, Thomas Gleixner wrote: > > > Megha, > > > > > > On Wed, Jul 07 2021 at 09:49, Megha Dey wrote: > > > > Per your suggestions during the last meeting, we wanted to confirm the > > > > sequence to program the PASID into the IMS entries: > > > > > > > > 1. Add a PASID member to struct msi_desc (Add as part of a union. Other > > > > source-id's such as Jason's vm-id can be added to it) > > > > > > Yes. Though we also discussed storing the default PASID in struct device > > > to begin with which is then copied to the msi_desc entries during > > > allocation. > > > > Using default PASID in struct device will work for sub-devices until the > > guest needs to enable ENQCMD support. Since the guest kernel can ask for an > > interrupt by specifying something in the descriptor submitted via ENQCMD. > > Using the PASID in struct device won't be sufficient. > > Could you could store a pasid table in the struct device and index it > by vector? Possibly... what ever Thomas things is clean. The device specific driver would have this already. So providing some call to get this filled in vs storing that in struct device. Someone close at heart to the driver model is best to comment :-) IMS core owns the format of the entries right now vs device specific driver. I suppose your use case requiring a vm_id might have a different format. So this is yet another one the core needs to learn and adapt? It seems we have conceptually 3 types of IMS already brewing up. 1. IDXD has legacy MSIx + MSix permission table to hold PASID. 2. IMS in IDXD has PASID in the IMS entry itself. 3. Jason's vm_id instead of the PASID use case. So we can keep IMS core about these types, another way is to allow device specific implementations provide the write handlers with the index, and let the driver handle the location of the write. IMS core will provide the format and data. Cheers, Ashok