Re: [PATCH v3 01/11] PCI/P2PDMA: Support peer-to-peer memory

From: "Stephen  Bates" <sbates@raithlin.com>
To: Sinan Kaya <okaya@codeaurora.org>,
	Logan Gunthorpe <logang@deltatee.com>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>,
	"linux-nvme@lists.infradead.org" <linux-nvme@lists.infradead.org>,
	"linux-rdma@vger.kernel.org" <linux-rdma@vger.kernel.org>,
	"linux-nvdimm@lists.01.org" <linux-nvdimm@lists.01.org>,
	"linux-block@vger.kernel.org" <linux-block@vger.kernel.org>
Cc: "Jens Axboe" <axboe@kernel.dk>,
	"Benjamin Herrenschmidt" <benh@kernel.crashing.org>,
	"Alex Williamson" <alex.williamson@redhat.com>,
	"Keith Busch" <keith.busch@intel.com>,
	"Jérôme Glisse" <jglisse@redhat.com>,
	"Jason Gunthorpe" <jgg@mellanox.com>,
	"Bjorn Helgaas" <bhelgaas@google.com>,
	"Max Gurtovoy" <maxg@mellanox.com>,
	"Christoph Hellwig" <hch@lst.de>
Subject: Re: [PATCH v3 01/11] PCI/P2PDMA: Support peer-to-peer memory
Date: Tue, 13 Mar 2018 22:45:41 +0000	[thread overview]
Message-ID: <47F026DE-AFD0-4F63-8D39-375D3684C7B1@raithlin.com> (raw)
In-Reply-To: <016dc910-f96a-8a60-4bda-fa24eea98ea5@codeaurora.org>

Hi Sinan

>    If hardware doesn't support it, blacklisting should have been the right
>    path and I still think that you should remove all switch business from the code.
>    I did not hear enough justification for having a switch requirement
>    for P2P.

We disagree. As does the community based on feedback on previous version of this patch set.

> You are also saying that root ports have issues not because of functionality but
> because of performance. 

I have seen both.  Some systems fail in entirety to route TLPs across route ports. Others do route but with poor performance. Having a requirement for a switch is therefore a reasonable and sensible minimum criteria for enabling p2p.

> If you know what is bad, create a list and keep adding it. You can't assume
> universally that all root ports are broken/ have bad performance.

I actually wanted to do a blacklist originally but that idea was not accepted by the community. 

>    What if I come up with a very cheap/crappy switch (something like used in data
>    mining)?
> What guarantees that P2P will work with this device? You are making an
> assumption here that all switches have good performance.

A switch must support P2P or it is in violation of the PCIe specification.

>    I have been doing my best to provide feedback. It feels like you are throwing
>    them over the wall to be honest.

I think one issue Sinan is that a lot of your ideas have been discussed in previous versions of the series. So you are asking us to make changes which, in some cases, we know are not acceptable to others in the community. The input is welcome but it flies in the face of other input we have received. 

However if other developers feel strongly about the blacklist/whitelist and switch criteria please speak up! ;-)

Stephen

_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm