From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 96652C433EF for ; Mon, 28 Mar 2022 21:38:38 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 33F6360F4C; Mon, 28 Mar 2022 21:38:38 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id nXmO_Yult95F; Mon, 28 Mar 2022 21:38:37 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id EC6D160FBF; Mon, 28 Mar 2022 21:38:36 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id BEF62C001D; Mon, 28 Mar 2022 21:38:36 +0000 (UTC) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 9149EC0012 for ; Mon, 28 Mar 2022 21:38:35 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 7025240213 for ; Mon, 28 Mar 2022 21:38:35 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp4.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=intel.com Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KNBQV_Qg35Xv for ; Mon, 28 Mar 2022 21:38:34 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mga14.intel.com (mga14.intel.com [192.55.52.115]) by smtp4.osuosl.org (Postfix) with ESMTPS id 4BAC34014A for ; Mon, 28 Mar 2022 21:38:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1648503514; x=1680039514; h=date:from:to:cc:subject:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=MoJJ7PpyY30zJiWm4s27HVjf2QmYd2IbqH48aFV+r4o=; b=K2r/qAXzWarpVx8KS+lKtIkwnGZjziZwkFKg8tGErJKFkW4SppVlBG/a 2WH+y7/XmDUcypwtHS8pGV15VcFjX4dt2gkUhP7Y2gkHAUsFeTOhln3Sy pEyMKyorF0ikiWFjYVUv2uwqC1DJEOIsnTocRdnURFZbAsdwd4c7l7xXO tVgauMqYwvTJHlrbCsChWxu2jj/OF+9O8PWrzLQvmHn+grj9u75yPKofc GPxb9MfvVs9S6/IV7KLAUhfsovJDJY7j71cqS2rxzt626wCRFKR+xXsxl ClJ+opMx156fWUinWseyK73ipZKaouBEFsGjZ4+bV+Sj1VzOkJUs6b6D+ A==; X-IronPort-AV: E=McAfee;i="6200,9189,10300"; a="259295124" X-IronPort-AV: E=Sophos;i="5.90,218,1643702400"; d="scan'208";a="259295124" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga103.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Mar 2022 14:38:33 -0700 X-IronPort-AV: E=Sophos;i="5.90,218,1643702400"; d="scan'208";a="564026275" Received: from jacob-builder.jf.intel.com (HELO jacob-builder) ([10.7.198.157]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Mar 2022 14:38:33 -0700 Date: Mon, 28 Mar 2022 14:41:56 -0700 From: Jacob Pan To: "Tian, Kevin" Subject: Re: [PATCH v2 3/8] iommu/vt-d: Implement device_pasid domain attach ops Message-ID: <20220328144156.66ba6f39@jacob-builder> In-Reply-To: References: <20220315050713.2000518-1-jacob.jun.pan@linux.intel.com> <20220315050713.2000518-4-jacob.jun.pan@linux.intel.com> <20220315143322.GW11336@nvidia.com> <20220316140140.76bb24c6@jacob-builder> Organization: OTC X-Mailer: Claws Mail 3.17.5 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Cc: "Luck, Tony" , "Jiang, Dave" , "Raj, Ashok" , "Zanussi, Tom" , "Kumar, Sanjay K" , LKML , Christoph Hellwig , "iommu@lists.linux-foundation.org" , "Pan, Jacob jun" , Jason Gunthorpe , "Williams, Dan J" , Jean-Philippe Brucker X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" Hi Kevin, On Fri, 18 Mar 2022 05:33:38 +0000, "Tian, Kevin" wrote: > > From: Jacob Pan > > Sent: Thursday, March 17, 2022 5:02 AM > > > > Hi Kevin, > > > > On Wed, 16 Mar 2022 07:41:34 +0000, "Tian, Kevin" > > wrote: > > > > > > From: Jason Gunthorpe > > > > Sent: Tuesday, March 15, 2022 10:33 PM > > > > > > > > On Mon, Mar 14, 2022 at 10:07:07PM -0700, Jacob Pan wrote: > > > > > + /* > > > > > + * Each domain could have multiple devices attached with > > > > > shared or > > > > per > > > > > + * device PASIDs. At the domain level, we keep track of > > > > > unique PASIDs > > > > and > > > > > + * device user count. > > > > > + * E.g. If a domain has two devices attached, device A > > > > > has PASID 0, 1; > > > > > + * device B has PASID 0, 2. Then the domain would have > > > > > PASID 0, 1, 2. > > > > > + */ > > > > > > > > A 2d array of xarray's seems like a poor data structure for this > > > > task. > > > > > Perhaps i mis-presented here, I am not using 2D array. It is an 1D > > xarray for domain PASIDs only. Then I use the existing device list in > > each domain, adding another xa to track per-device-domain PASIDs. > > > besides that it also doesn't work when we support per-device PASID > > > allocation in the future. In that case merging device PASIDs together > > > is conceptually wrong. > > > > > Sorry, could you elaborate? If we do per-dev PASID allocation, we could > > use the ioasid_set for each pdev, right? > > My point is simply about the comment above which says the domain > will have PASID 0, 1, 2 when there is [devA, PASID0] and [devB, PASID0]. > You can maintain a single PASID list only when it's globally allocated > cross devices. otherwise this has to be a tuple including device and > PASID. > Got you, you are right we don't want to limit to globally allocated scheme. Thanks, Jacob _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E459EC433EF for ; Mon, 28 Mar 2022 22:08:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229464AbiC1WJv (ORCPT ); Mon, 28 Mar 2022 18:09:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50174 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229500AbiC1WHv (ORCPT ); Mon, 28 Mar 2022 18:07:51 -0400 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B122A13CED5 for ; Mon, 28 Mar 2022 15:01:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1648504875; x=1680040875; h=date:from:to:cc:subject:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=MoJJ7PpyY30zJiWm4s27HVjf2QmYd2IbqH48aFV+r4o=; b=ezbr+1WecsUVnzOlGib5YzuCjxLK7by4BGLw3kkIw38OmXDUdTUcfB9+ K41itfz+pkuEn9UaFs6tjR2uz7rcLF1HeSqek3TZWbiwsGGWy6PwTU10s wxATxupkB4UsTzWMlsNfaiWvjQ6NKW/ZeyvLsFKRtPWnxPyXP8uxma1oK WWzamK0lKBt/zEcyjO1zKvFfd8fMSTt1mz8pP7muPRG6LrL7diRGO+VwM ECuElNRCPmeQDAePmaLDyRLKak2zELujnAfmig48y+q8cwX8O6Kt7ktQL qAEVKzHGkCRcQQMg5QI9/YWK5DH5imNY5UuSrOTpRajozHvQ+LrtlHFJK Q==; X-IronPort-AV: E=McAfee;i="6200,9189,10300"; a="257936479" X-IronPort-AV: E=Sophos;i="5.90,218,1643702400"; d="scan'208";a="257936479" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Mar 2022 14:38:33 -0700 X-IronPort-AV: E=Sophos;i="5.90,218,1643702400"; d="scan'208";a="564026275" Received: from jacob-builder.jf.intel.com (HELO jacob-builder) ([10.7.198.157]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Mar 2022 14:38:33 -0700 Date: Mon, 28 Mar 2022 14:41:56 -0700 From: Jacob Pan To: "Tian, Kevin" Cc: "Luck, Tony" , "Jiang, Dave" , "Raj, Ashok" , "Zanussi, Tom" , "Kumar, Sanjay K" , LKML , Christoph Hellwig , "iommu@lists.linux-foundation.org" , "Pan, Jacob jun" , Jason Gunthorpe , "Williams, Dan J" , Jean-Philippe Brucker , jacob.jun.pan@linux.intel.com Subject: Re: [PATCH v2 3/8] iommu/vt-d: Implement device_pasid domain attach ops Message-ID: <20220328144156.66ba6f39@jacob-builder> In-Reply-To: References: <20220315050713.2000518-1-jacob.jun.pan@linux.intel.com> <20220315050713.2000518-4-jacob.jun.pan@linux.intel.com> <20220315143322.GW11336@nvidia.com> <20220316140140.76bb24c6@jacob-builder> Organization: OTC X-Mailer: Claws Mail 3.17.5 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Kevin, On Fri, 18 Mar 2022 05:33:38 +0000, "Tian, Kevin" wrote: > > From: Jacob Pan > > Sent: Thursday, March 17, 2022 5:02 AM > > > > Hi Kevin, > > > > On Wed, 16 Mar 2022 07:41:34 +0000, "Tian, Kevin" > > wrote: > > > > > > From: Jason Gunthorpe > > > > Sent: Tuesday, March 15, 2022 10:33 PM > > > > > > > > On Mon, Mar 14, 2022 at 10:07:07PM -0700, Jacob Pan wrote: > > > > > + /* > > > > > + * Each domain could have multiple devices attached with > > > > > shared or > > > > per > > > > > + * device PASIDs. At the domain level, we keep track of > > > > > unique PASIDs > > > > and > > > > > + * device user count. > > > > > + * E.g. If a domain has two devices attached, device A > > > > > has PASID 0, 1; > > > > > + * device B has PASID 0, 2. Then the domain would have > > > > > PASID 0, 1, 2. > > > > > + */ > > > > > > > > A 2d array of xarray's seems like a poor data structure for this > > > > task. > > > > > Perhaps i mis-presented here, I am not using 2D array. It is an 1D > > xarray for domain PASIDs only. Then I use the existing device list in > > each domain, adding another xa to track per-device-domain PASIDs. > > > besides that it also doesn't work when we support per-device PASID > > > allocation in the future. In that case merging device PASIDs together > > > is conceptually wrong. > > > > > Sorry, could you elaborate? If we do per-dev PASID allocation, we could > > use the ioasid_set for each pdev, right? > > My point is simply about the comment above which says the domain > will have PASID 0, 1, 2 when there is [devA, PASID0] and [devB, PASID0]. > You can maintain a single PASID list only when it's globally allocated > cross devices. otherwise this has to be a tuple including device and > PASID. > Got you, you are right we don't want to limit to globally allocated scheme. Thanks, Jacob