From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-io0-x241.google.com (mail-io0-x241.google.com [IPv6:2607:f8b0:4001:c06::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 23C712111FE4D for ; Wed, 5 Sep 2018 13:37:00 -0700 (PDT) Received: by mail-io0-x241.google.com with SMTP id 75-v6so7070083iou.11 for ; Wed, 05 Sep 2018 13:37:00 -0700 (PDT) Subject: Re: [PATCH v5 07/13] block: Add PCI P2P flag for request queue and check support for requests References: <20180830185352.3369-1-logang@deltatee.com> <20180830185352.3369-8-logang@deltatee.com> <20180901082812.GB670@lst.de> <5f79c012-c6e1-56bb-62fd-0689181fb2c9@deltatee.com> <59b28977-8f2a-6228-2050-03fae6bdbedd@kernel.dk> <1b4283da-44df-4a02-3167-e295243cef78@deltatee.com> <09258b9b-3aed-9890-b31a-bd70a133966c@kernel.dk> <20180905195647.GA1626@lst.de> <20180905201152.GA1893@lst.de> <2a3394bd-5f13-4818-43f4-dfc61f501e05@kernel.dk> <3af4d1d4-da07-c0a6-8464-9ddc1378f2f4@kernel.dk> From: Jens Axboe Message-ID: <34d9b2f7-9e6d-4a0a-77e5-ec0e99610f5c@kernel.dk> Date: Wed, 5 Sep 2018 14:36:56 -0600 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" To: Logan Gunthorpe , Christoph Hellwig Cc: Alex Williamson , linux-nvdimm@lists.01.org, linux-rdma@vger.kernel.org, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, =?UTF-8?B?SsOpcsO0bWUgR2xpc3Nl?= , Jason Gunthorpe , Benjamin Herrenschmidt , Bjorn Helgaas , Max Gurtovoy , =?UTF-8?Q?Christian_K=c3=b6nig?= List-ID: On 9/5/18 2:32 PM, Logan Gunthorpe wrote: > > > On 05/09/18 02:19 PM, Jens Axboe wrote: >> On 9/5/18 2:18 PM, Logan Gunthorpe wrote: >>> >>> >>> On 05/09/18 02:14 PM, Jens Axboe wrote: >>>> But if the caller must absolutely know where the bio will end up, then >>>> it seems super redundant. So I'd vote for killing this check, it buys >>>> us absolutely nothing and isn't even exhaustive in its current form. >>> >>> >>> Ok, I'll remove it for v6. >> >> Since the drivers needs to know it's doing it right, it might not >> hurt to add a sanity check helper for that. Just have the driver >> call it, and don't add it in the normal IO submission path. > > I'm not sure I really see the value in that. It's the same principle in > asking the driver to do the WARN: if the developer knew enough to use > the special helper, they probably knew well enough to do the rest correctly. I don't agree with that at all. It's a "is my request valid" helper, it's not some obscure and rarely used functionality. You're making up this API right now, if you really want it done for every IO, make it part of the p2p submission process. You could even hide it behind a debug thing, if you like. > I guess one other thing to point out is that, on x86, if a driver > submits P2P pages to a PCI device that doesn't have kernel support, > everything will likely just work. Even though the driver isn't doing any > of the work correctly and the requests are not being mapped with > pci_p2pdma_map() functions. Such code on other arches would likely > break. So developers may be lulled into thinking they're doing the > correct thing when in fact they are not and the WARN in the common code > would prevent that. If you're adamant about having it in common code, put it in your common submission code. Most folks aren't going to care about P2P, let the ones that do have the checks. -- Jens Axboe _______________________________________________ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Subject: Re: [PATCH v5 07/13] block: Add PCI P2P flag for request queue and check support for requests To: Logan Gunthorpe , Christoph Hellwig Cc: linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, linux-nvme@lists.infradead.org, linux-rdma@vger.kernel.org, linux-nvdimm@lists.01.org, linux-block@vger.kernel.org, Stephen Bates , Keith Busch , Sagi Grimberg , Bjorn Helgaas , Jason Gunthorpe , Max Gurtovoy , Dan Williams , =?UTF-8?B?SsOpcsO0bWUgR2xpc3Nl?= , Benjamin Herrenschmidt , Alex Williamson , =?UTF-8?Q?Christian_K=c3=b6nig?= References: <20180830185352.3369-1-logang@deltatee.com> <20180830185352.3369-8-logang@deltatee.com> <20180901082812.GB670@lst.de> <5f79c012-c6e1-56bb-62fd-0689181fb2c9@deltatee.com> <59b28977-8f2a-6228-2050-03fae6bdbedd@kernel.dk> <1b4283da-44df-4a02-3167-e295243cef78@deltatee.com> <09258b9b-3aed-9890-b31a-bd70a133966c@kernel.dk> <20180905195647.GA1626@lst.de> <20180905201152.GA1893@lst.de> <2a3394bd-5f13-4818-43f4-dfc61f501e05@kernel.dk> <3af4d1d4-da07-c0a6-8464-9ddc1378f2f4@kernel.dk> From: Jens Axboe Message-ID: <34d9b2f7-9e6d-4a0a-77e5-ec0e99610f5c@kernel.dk> Date: Wed, 5 Sep 2018 14:36:56 -0600 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 List-ID: On 9/5/18 2:32 PM, Logan Gunthorpe wrote: > > > On 05/09/18 02:19 PM, Jens Axboe wrote: >> On 9/5/18 2:18 PM, Logan Gunthorpe wrote: >>> >>> >>> On 05/09/18 02:14 PM, Jens Axboe wrote: >>>> But if the caller must absolutely know where the bio will end up, then >>>> it seems super redundant. So I'd vote for killing this check, it buys >>>> us absolutely nothing and isn't even exhaustive in its current form. >>> >>> >>> Ok, I'll remove it for v6. >> >> Since the drivers needs to know it's doing it right, it might not >> hurt to add a sanity check helper for that. Just have the driver >> call it, and don't add it in the normal IO submission path. > > I'm not sure I really see the value in that. It's the same principle in > asking the driver to do the WARN: if the developer knew enough to use > the special helper, they probably knew well enough to do the rest correctly. I don't agree with that at all. It's a "is my request valid" helper, it's not some obscure and rarely used functionality. You're making up this API right now, if you really want it done for every IO, make it part of the p2p submission process. You could even hide it behind a debug thing, if you like. > I guess one other thing to point out is that, on x86, if a driver > submits P2P pages to a PCI device that doesn't have kernel support, > everything will likely just work. Even though the driver isn't doing any > of the work correctly and the requests are not being mapped with > pci_p2pdma_map() functions. Such code on other arches would likely > break. So developers may be lulled into thinking they're doing the > correct thing when in fact they are not and the WARN in the common code > would prevent that. If you're adamant about having it in common code, put it in your common submission code. Most folks aren't going to care about P2P, let the ones that do have the checks. -- Jens Axboe From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jens Axboe Subject: Re: [PATCH v5 07/13] block: Add PCI P2P flag for request queue and check support for requests Date: Wed, 5 Sep 2018 14:36:56 -0600 Message-ID: <34d9b2f7-9e6d-4a0a-77e5-ec0e99610f5c@kernel.dk> References: <20180830185352.3369-1-logang@deltatee.com> <20180830185352.3369-8-logang@deltatee.com> <20180901082812.GB670@lst.de> <5f79c012-c6e1-56bb-62fd-0689181fb2c9@deltatee.com> <59b28977-8f2a-6228-2050-03fae6bdbedd@kernel.dk> <1b4283da-44df-4a02-3167-e295243cef78@deltatee.com> <09258b9b-3aed-9890-b31a-bd70a133966c@kernel.dk> <20180905195647.GA1626@lst.de> <20180905201152.GA1893@lst.de> <2a3394bd-5f13-4818-43f4-dfc61f501e05@kernel.dk> <3af4d1d4-da07-c0a6-8464-9ddc1378f2f4@kernel.dk> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: Content-Language: en-US List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: linux-nvdimm-bounces-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org Sender: "Linux-nvdimm" To: Logan Gunthorpe , Christoph Hellwig Cc: Alex Williamson , linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org, linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-pci-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org, linux-block-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, =?UTF-8?B?SsOpcsO0bWUgR2xpc3Nl?= , Jason Gunthorpe , Benjamin Herrenschmidt , Bjorn Helgaas , Max Gurtovoy , =?UTF-8?Q?Christian_K=c3=b6nig?= List-Id: linux-rdma@vger.kernel.org On 9/5/18 2:32 PM, Logan Gunthorpe wrote: > > > On 05/09/18 02:19 PM, Jens Axboe wrote: >> On 9/5/18 2:18 PM, Logan Gunthorpe wrote: >>> >>> >>> On 05/09/18 02:14 PM, Jens Axboe wrote: >>>> But if the caller must absolutely know where the bio will end up, then >>>> it seems super redundant. So I'd vote for killing this check, it buys >>>> us absolutely nothing and isn't even exhaustive in its current form. >>> >>> >>> Ok, I'll remove it for v6. >> >> Since the drivers needs to know it's doing it right, it might not >> hurt to add a sanity check helper for that. Just have the driver >> call it, and don't add it in the normal IO submission path. > > I'm not sure I really see the value in that. It's the same principle in > asking the driver to do the WARN: if the developer knew enough to use > the special helper, they probably knew well enough to do the rest correctly. I don't agree with that at all. It's a "is my request valid" helper, it's not some obscure and rarely used functionality. You're making up this API right now, if you really want it done for every IO, make it part of the p2p submission process. You could even hide it behind a debug thing, if you like. > I guess one other thing to point out is that, on x86, if a driver > submits P2P pages to a PCI device that doesn't have kernel support, > everything will likely just work. Even though the driver isn't doing any > of the work correctly and the requests are not being mapped with > pci_p2pdma_map() functions. Such code on other arches would likely > break. So developers may be lulled into thinking they're doing the > correct thing when in fact they are not and the WARN in the common code > would prevent that. If you're adamant about having it in common code, put it in your common submission code. Most folks aren't going to care about P2P, let the ones that do have the checks. -- Jens Axboe From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Return-Path: Subject: Re: [PATCH v5 07/13] block: Add PCI P2P flag for request queue and check support for requests To: Logan Gunthorpe , Christoph Hellwig References: <20180830185352.3369-1-logang@deltatee.com> <20180830185352.3369-8-logang@deltatee.com> <20180901082812.GB670@lst.de> <5f79c012-c6e1-56bb-62fd-0689181fb2c9@deltatee.com> <59b28977-8f2a-6228-2050-03fae6bdbedd@kernel.dk> <1b4283da-44df-4a02-3167-e295243cef78@deltatee.com> <09258b9b-3aed-9890-b31a-bd70a133966c@kernel.dk> <20180905195647.GA1626@lst.de> <20180905201152.GA1893@lst.de> <2a3394bd-5f13-4818-43f4-dfc61f501e05@kernel.dk> <3af4d1d4-da07-c0a6-8464-9ddc1378f2f4@kernel.dk> From: Jens Axboe Message-ID: <34d9b2f7-9e6d-4a0a-77e5-ec0e99610f5c@kernel.dk> Date: Wed, 5 Sep 2018 14:36:56 -0600 MIME-Version: 1.0 In-Reply-To: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Keith Busch , Alex Williamson , Sagi Grimberg , linux-nvdimm@lists.01.org, linux-rdma@vger.kernel.org, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, Stephen Bates , linux-block@vger.kernel.org, =?UTF-8?B?SsOpcsO0bWUgR2xpc3Nl?= , Jason Gunthorpe , Benjamin Herrenschmidt , Bjorn Helgaas , Max Gurtovoy , Dan Williams , =?UTF-8?Q?Christian_K=c3=b6nig?= Content-Type: text/plain; charset="us-ascii" Sender: "Linux-nvme" Errors-To: linux-nvme-bounces+bjorn=helgaas.com@lists.infradead.org List-ID: On 9/5/18 2:32 PM, Logan Gunthorpe wrote: > > > On 05/09/18 02:19 PM, Jens Axboe wrote: >> On 9/5/18 2:18 PM, Logan Gunthorpe wrote: >>> >>> >>> On 05/09/18 02:14 PM, Jens Axboe wrote: >>>> But if the caller must absolutely know where the bio will end up, then >>>> it seems super redundant. So I'd vote for killing this check, it buys >>>> us absolutely nothing and isn't even exhaustive in its current form. >>> >>> >>> Ok, I'll remove it for v6. >> >> Since the drivers needs to know it's doing it right, it might not >> hurt to add a sanity check helper for that. Just have the driver >> call it, and don't add it in the normal IO submission path. > > I'm not sure I really see the value in that. It's the same principle in > asking the driver to do the WARN: if the developer knew enough to use > the special helper, they probably knew well enough to do the rest correctly. I don't agree with that at all. It's a "is my request valid" helper, it's not some obscure and rarely used functionality. You're making up this API right now, if you really want it done for every IO, make it part of the p2p submission process. You could even hide it behind a debug thing, if you like. > I guess one other thing to point out is that, on x86, if a driver > submits P2P pages to a PCI device that doesn't have kernel support, > everything will likely just work. Even though the driver isn't doing any > of the work correctly and the requests are not being mapped with > pci_p2pdma_map() functions. Such code on other arches would likely > break. So developers may be lulled into thinking they're doing the > correct thing when in fact they are not and the WARN in the common code > would prevent that. If you're adamant about having it in common code, put it in your common submission code. Most folks aren't going to care about P2P, let the ones that do have the checks. -- Jens Axboe _______________________________________________ Linux-nvme mailing list Linux-nvme@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-nvme From mboxrd@z Thu Jan 1 00:00:00 1970 From: axboe@kernel.dk (Jens Axboe) Date: Wed, 5 Sep 2018 14:36:56 -0600 Subject: [PATCH v5 07/13] block: Add PCI P2P flag for request queue and check support for requests In-Reply-To: References: <20180830185352.3369-1-logang@deltatee.com> <20180830185352.3369-8-logang@deltatee.com> <20180901082812.GB670@lst.de> <5f79c012-c6e1-56bb-62fd-0689181fb2c9@deltatee.com> <59b28977-8f2a-6228-2050-03fae6bdbedd@kernel.dk> <1b4283da-44df-4a02-3167-e295243cef78@deltatee.com> <09258b9b-3aed-9890-b31a-bd70a133966c@kernel.dk> <20180905195647.GA1626@lst.de> <20180905201152.GA1893@lst.de> <2a3394bd-5f13-4818-43f4-dfc61f501e05@kernel.dk> <3af4d1d4-da07-c0a6-8464-9ddc1378f2f4@kernel.dk> Message-ID: <34d9b2f7-9e6d-4a0a-77e5-ec0e99610f5c@kernel.dk> On 9/5/18 2:32 PM, Logan Gunthorpe wrote: > > > On 05/09/18 02:19 PM, Jens Axboe wrote: >> On 9/5/18 2:18 PM, Logan Gunthorpe wrote: >>> >>> >>> On 05/09/18 02:14 PM, Jens Axboe wrote: >>>> But if the caller must absolutely know where the bio will end up, then >>>> it seems super redundant. So I'd vote for killing this check, it buys >>>> us absolutely nothing and isn't even exhaustive in its current form. >>> >>> >>> Ok, I'll remove it for v6. >> >> Since the drivers needs to know it's doing it right, it might not >> hurt to add a sanity check helper for that. Just have the driver >> call it, and don't add it in the normal IO submission path. > > I'm not sure I really see the value in that. It's the same principle in > asking the driver to do the WARN: if the developer knew enough to use > the special helper, they probably knew well enough to do the rest correctly. I don't agree with that at all. It's a "is my request valid" helper, it's not some obscure and rarely used functionality. You're making up this API right now, if you really want it done for every IO, make it part of the p2p submission process. You could even hide it behind a debug thing, if you like. > I guess one other thing to point out is that, on x86, if a driver > submits P2P pages to a PCI device that doesn't have kernel support, > everything will likely just work. Even though the driver isn't doing any > of the work correctly and the requests are not being mapped with > pci_p2pdma_map() functions. Such code on other arches would likely > break. So developers may be lulled into thinking they're doing the > correct thing when in fact they are not and the WARN in the common code > would prevent that. If you're adamant about having it in common code, put it in your common submission code. Most folks aren't going to care about P2P, let the ones that do have the checks. -- Jens Axboe