From: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> To: "Tian, Kevin" <kevin.tian@intel.com>, Jacob Pan <jacob.jun.pan@linux.intel.com> Cc: "alex.williamson@redhat.com" <alex.williamson@redhat.com>, "robin.murphy@arm.com" <robin.murphy@arm.com>, "Raj, Ashok" <ashok.raj@intel.com>, "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, "eric.auger@redhat.com" <eric.auger@redhat.com> Subject: Re: [PATCH v2 2/4] iommu: Introduce device fault data Date: Wed, 5 Jun 2019 12:24:09 +0100 [thread overview] Message-ID: <50dc3cc5-6019-ad42-6aba-d84fab4020f9@arm.com> (raw) In-Reply-To: <AADFC41AFE54684AB9EE6CBC0274A5D19CA6A9EE@SHSMSX104.ccr.corp.intel.com> On 05/06/2019 09:51, Tian, Kevin wrote: >> From: Jacob Pan >> Sent: Tuesday, June 4, 2019 6:09 AM >> >> On Mon, 3 Jun 2019 15:57:47 +0100 >> Jean-Philippe Brucker <jean-philippe.brucker@arm.com> wrote: >> >>> +/** >>> + * struct iommu_fault_page_request - Page Request data >>> + * @flags: encodes whether the corresponding fields are valid and >>> whether this >>> + * is the last page in group (IOMMU_FAULT_PAGE_REQUEST_* >>> values) >>> + * @pasid: Process Address Space ID >>> + * @grpid: Page Request Group Index >>> + * @perm: requested page permissions (IOMMU_FAULT_PERM_* values) >>> + * @addr: page address >>> + * @private_data: device-specific private information >>> + */ >>> +struct iommu_fault_page_request { >>> +#define IOMMU_FAULT_PAGE_REQUEST_PASID_VALID (1 << 0) >>> +#define IOMMU_FAULT_PAGE_REQUEST_LAST_PAGE (1 << 1) >>> +#define IOMMU_FAULT_PAGE_REQUEST_PRIV_DATA (1 << 2) >>> + __u32 flags; >>> + __u32 pasid; >>> + __u32 grpid; >>> + __u32 perm; >>> + __u64 addr; >>> + __u64 private_data[2]; >>> +}; >>> + >> >> Just a thought, for non-identity G-H PASID management. We could pass on >> guest PASID in PRQ to save a lookup in QEMU. In this case, QEMU >> allocate a GPASID for vIOMMU then a host PASID for pIOMMU. QEMU has a >> G->H lookup. When PRQ comes in to the pIOMMU with HPASID, IOMMU >> driver >> can retrieve GPASID from the bind data then report to the guest via >> VFIO. In this case QEMU does not need to do a H->G PASID lookup. >> >> Should we add a gpasid field here? or we can add a flag and field >> later, up to you. >> > > Can private_data serve this purpose? Isn't private_data already used for VT-d's Private Data field? > It's better not introducing > gpasid awareness within host IOMMU driver. It is just a user-level > data associated with a PASID when binding happens. Kernel doesn't > care the actual meaning, simply record it and then return back to user > space later upon device fault. Qemu interprets the meaning as gpasid > in its own context. otherwise usages may use it for other purpose. Regarding a gpasid field I don't mind either way, but extending the iommu_fault structure later won't be completely straightforward so we could add some padding now. Userspace negotiate the iommu_fault struct format with VFIO, before allocating a circular buffer of N fault structures (https://lore.kernel.org/lkml/20190526161004.25232-26-eric.auger@redhat.com/). So adding new fields requires introducing a new ABI version and a struct iommu_fault_v2. That may be OK for disruptive changes, but just adding a new field indicated by a flag shouldn't have to be that complicated. How about setting the iommu_fault structure to 128 bytes? struct iommu_fault { __u32 type; __u32 padding; union { struct iommu_fault_unrecoverable event; struct iommu_fault_page_request prm; __u8 padding2[120]; }; }; Given that @prm is currently 40 bytes and @event 32 bytes, the padding allows either of them to grow 10 new 64-bit fields (or 20 new 32-bit fields, which is still representable with new flags) before we have to upgrade the ABI version. A 4kB and a 64kB queue can hold respectively: * 85 and 1365 records when iommu_fault is 48 bytes (current format). * 64 and 1024 records when iommu_fault is 64 bytes (but allows to grow only 2 new 64-bit fields). * 32 and 512 records when iommu_fault is 128 bytes. In comparison, * the SMMU even queue can hold 128 and 2048 events respectively at those sizes (and is allowed to grow up to 524k entries) * the SMMU PRI queue can hold 256 and 4096 PR. But the SMMU queues have to be physically contiguous, whereas our fault queues are in userspace memory which is less expensive. So 128-byte records might be reasonable. What do you think? The iommu_fault_response (patch 4/4) is a bit easier to extend because it's userspace->kernel and userspace can just declare the size it's using. I did add a version field in case we run out of flags or want to change the whole thing, but I think I was being overly cautious and it might just be a waste of space. Thanks, Jean
WARNING: multiple messages have this Message-ID (diff)
From: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> To: "Tian, Kevin" <kevin.tian@intel.com>, Jacob Pan <jacob.jun.pan@linux.intel.com> Cc: "Raj, Ashok" <ashok.raj@intel.com>, "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, "alex.williamson@redhat.com" <alex.williamson@redhat.com>, "robin.murphy@arm.com" <robin.murphy@arm.com> Subject: Re: [PATCH v2 2/4] iommu: Introduce device fault data Date: Wed, 5 Jun 2019 12:24:09 +0100 [thread overview] Message-ID: <50dc3cc5-6019-ad42-6aba-d84fab4020f9@arm.com> (raw) In-Reply-To: <AADFC41AFE54684AB9EE6CBC0274A5D19CA6A9EE@SHSMSX104.ccr.corp.intel.com> On 05/06/2019 09:51, Tian, Kevin wrote: >> From: Jacob Pan >> Sent: Tuesday, June 4, 2019 6:09 AM >> >> On Mon, 3 Jun 2019 15:57:47 +0100 >> Jean-Philippe Brucker <jean-philippe.brucker@arm.com> wrote: >> >>> +/** >>> + * struct iommu_fault_page_request - Page Request data >>> + * @flags: encodes whether the corresponding fields are valid and >>> whether this >>> + * is the last page in group (IOMMU_FAULT_PAGE_REQUEST_* >>> values) >>> + * @pasid: Process Address Space ID >>> + * @grpid: Page Request Group Index >>> + * @perm: requested page permissions (IOMMU_FAULT_PERM_* values) >>> + * @addr: page address >>> + * @private_data: device-specific private information >>> + */ >>> +struct iommu_fault_page_request { >>> +#define IOMMU_FAULT_PAGE_REQUEST_PASID_VALID (1 << 0) >>> +#define IOMMU_FAULT_PAGE_REQUEST_LAST_PAGE (1 << 1) >>> +#define IOMMU_FAULT_PAGE_REQUEST_PRIV_DATA (1 << 2) >>> + __u32 flags; >>> + __u32 pasid; >>> + __u32 grpid; >>> + __u32 perm; >>> + __u64 addr; >>> + __u64 private_data[2]; >>> +}; >>> + >> >> Just a thought, for non-identity G-H PASID management. We could pass on >> guest PASID in PRQ to save a lookup in QEMU. In this case, QEMU >> allocate a GPASID for vIOMMU then a host PASID for pIOMMU. QEMU has a >> G->H lookup. When PRQ comes in to the pIOMMU with HPASID, IOMMU >> driver >> can retrieve GPASID from the bind data then report to the guest via >> VFIO. In this case QEMU does not need to do a H->G PASID lookup. >> >> Should we add a gpasid field here? or we can add a flag and field >> later, up to you. >> > > Can private_data serve this purpose? Isn't private_data already used for VT-d's Private Data field? > It's better not introducing > gpasid awareness within host IOMMU driver. It is just a user-level > data associated with a PASID when binding happens. Kernel doesn't > care the actual meaning, simply record it and then return back to user > space later upon device fault. Qemu interprets the meaning as gpasid > in its own context. otherwise usages may use it for other purpose. Regarding a gpasid field I don't mind either way, but extending the iommu_fault structure later won't be completely straightforward so we could add some padding now. Userspace negotiate the iommu_fault struct format with VFIO, before allocating a circular buffer of N fault structures (https://lore.kernel.org/lkml/20190526161004.25232-26-eric.auger@redhat.com/). So adding new fields requires introducing a new ABI version and a struct iommu_fault_v2. That may be OK for disruptive changes, but just adding a new field indicated by a flag shouldn't have to be that complicated. How about setting the iommu_fault structure to 128 bytes? struct iommu_fault { __u32 type; __u32 padding; union { struct iommu_fault_unrecoverable event; struct iommu_fault_page_request prm; __u8 padding2[120]; }; }; Given that @prm is currently 40 bytes and @event 32 bytes, the padding allows either of them to grow 10 new 64-bit fields (or 20 new 32-bit fields, which is still representable with new flags) before we have to upgrade the ABI version. A 4kB and a 64kB queue can hold respectively: * 85 and 1365 records when iommu_fault is 48 bytes (current format). * 64 and 1024 records when iommu_fault is 64 bytes (but allows to grow only 2 new 64-bit fields). * 32 and 512 records when iommu_fault is 128 bytes. In comparison, * the SMMU even queue can hold 128 and 2048 events respectively at those sizes (and is allowed to grow up to 524k entries) * the SMMU PRI queue can hold 256 and 4096 PR. But the SMMU queues have to be physically contiguous, whereas our fault queues are in userspace memory which is less expensive. So 128-byte records might be reasonable. What do you think? The iommu_fault_response (patch 4/4) is a bit easier to extend because it's userspace->kernel and userspace can just declare the size it's using. I did add a version field in case we run out of flags or want to change the whole thing, but I think I was being overly cautious and it might just be a waste of space. Thanks, Jean _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
next prev parent reply other threads:[~2019-06-05 11:24 UTC|newest] Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-06-03 14:57 [PATCH v2 0/4] iommu: Add device fault reporting API Jean-Philippe Brucker 2019-06-03 14:57 ` Jean-Philippe Brucker 2019-06-03 14:57 ` [PATCH v2 1/4] driver core: Add per device iommu param Jean-Philippe Brucker 2019-06-03 14:57 ` Jean-Philippe Brucker 2019-06-03 14:57 ` [PATCH v2 2/4] iommu: Introduce device fault data Jean-Philippe Brucker 2019-06-03 14:57 ` Jean-Philippe Brucker 2019-06-03 22:08 ` Jacob Pan 2019-06-03 22:08 ` Jacob Pan 2019-06-05 8:51 ` Tian, Kevin 2019-06-05 8:51 ` Tian, Kevin 2019-06-05 11:24 ` Jean-Philippe Brucker [this message] 2019-06-05 11:24 ` Jean-Philippe Brucker 2019-06-05 21:58 ` Jacob Pan 2019-06-05 21:58 ` Jacob Pan 2019-06-05 17:37 ` Jacob Pan 2019-06-05 17:37 ` Jacob Pan 2019-06-06 6:54 ` Tian, Kevin 2019-06-06 6:54 ` Tian, Kevin 2019-06-03 14:57 ` [PATCH v2 3/4] iommu: Introduce device fault report API Jean-Philippe Brucker 2019-06-03 14:57 ` Jean-Philippe Brucker 2019-06-03 14:57 ` [PATCH v2 4/4] iommu: Add recoverable fault reporting Jean-Philippe Brucker 2019-06-03 14:57 ` Jean-Philippe Brucker 2019-06-03 21:59 ` [PATCH v2 0/4] iommu: Add device fault reporting API Jacob Pan 2019-06-03 21:59 ` Jacob Pan 2019-06-05 11:26 ` Jean-Philippe Brucker 2019-06-05 11:26 ` Jean-Philippe Brucker 2019-06-12 8:19 ` Joerg Roedel 2019-06-12 8:19 ` Joerg Roedel 2019-06-12 11:54 ` Jean-Philippe Brucker 2019-06-12 11:54 ` Jean-Philippe Brucker 2019-06-12 13:11 ` Joerg Roedel 2019-06-12 13:11 ` Joerg Roedel 2019-06-12 17:59 ` [PATCH] iommu: Add padding to struct iommu_fault Jean-Philippe Brucker 2019-06-12 19:02 ` Jacob Pan 2019-06-12 19:19 ` Auger Eric 2019-06-18 15:15 ` Joerg Roedel 2019-06-12 18:58 ` [PATCH v2 0/4] iommu: Add device fault reporting API Jacob Pan 2019-06-12 18:58 ` Jacob Pan
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=50dc3cc5-6019-ad42-6aba-d84fab4020f9@arm.com \ --to=jean-philippe.brucker@arm.com \ --cc=alex.williamson@redhat.com \ --cc=ashok.raj@intel.com \ --cc=eric.auger@redhat.com \ --cc=iommu@lists.linux-foundation.org \ --cc=jacob.jun.pan@linux.intel.com \ --cc=kevin.tian@intel.com \ --cc=linux-kernel@vger.kernel.org \ --cc=robin.murphy@arm.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.