From: Bjorn Helgaas <helgaas@kernel.org>
To: Stephen Bates <sbates@raithlin.com>
Cc: "Jens Axboe" <axboe@kernel.dk>,
"linux-block@vger.kernel.org" <linux-block@vger.kernel.org>,
"Alex Williamson" <alex.williamson@redhat.com>,
"linux-nvdimm@lists.01.org" <linux-nvdimm@lists.01.org>,
"linux-rdma@vger.kernel.org" <linux-rdma@vger.kernel.org>,
"linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"linux-nvme@lists.infradead.org" <linux-nvme@lists.infradead.org>,
"Sinan Kaya" <okaya@codeaurora.org>,
"Jérôme Glisse" <jglisse@redhat.com>,
"Jason Gunthorpe" <jgg@mellanox.com>,
"Benjamin Herrenschmidt" <benh@kernel.crashing.org>,
"Bjorn Helgaas" <bhelgaas@google.com>,
"Max Gurtovoy" <maxg@mellanox.com>,
"Keith Busch" <keith.busch@intel.com>,
"Christoph Hellwig" <hch@lst.de>
Subject: Re: [PATCH v3 01/11] PCI/P2PDMA: Support peer-to-peer memory
Date: Tue, 13 Mar 2018 18:08:50 -0500 [thread overview]
Message-ID: <20180313230850.GA45763@bhelgaas-glaptop.roam.corp.google.com> (raw)
In-Reply-To: <BB94077B-7EB3-49C4-862D-BF77D0A692CC@raithlin.com>
On Tue, Mar 13, 2018 at 10:31:55PM +0000, Stephen Bates wrote:
> >> It sounds like you have very tight hardware expectations for this to work
> >> at this moment. You also don't want to generalize this code for others and
> >> address the shortcomings.
> > No, that's the way the community has pushed this work
>
> Hi Sinan
>
> Thanks for all the input. As Logan has pointed out the switch
> requirement is something that has evolved over time based on input
> from the community. You are more than welcome to have an opinion on
> this (and you have made that opinion clear ;-)). Over time the
> patchset may evolve from its current requirements but right now we
> are aligned with the feedback from the community.
This part of the community hasn't been convinced of the need to have
two bridges, e.g., both an Upstream Port and a Downstream Port, or two
conventional PCI bridges, above the peers.
Every PCI-to-PCI bridge is required to support routing transactions
between devices on its secondary side. Therefore, I think it is
sufficient to verify that the potential peers share a single common
upstream bridge. This could be a conventional PCI bridge, a Switch
Downstream Port, or a Root Port.
I've seen the response that peers directly below a Root Port could not
DMA to each other through the Root Port because of the "route to self"
issue, and I'm not disputing that. But enforcing a requirement for
two upstream bridges introduces a weird restriction on conventional
PCI topologies, makes the code hard to read, and I don't think it's
necessary.
If it *is* necessary because Root Ports and devices below them behave
differently than in conventional PCI, I think you should include a
reference to the relevant section of the spec and check directly for a
Root Port. I would prefer that over trying to exclude Root Ports by
looking for two upstream bridges.
Bjorn
_______________________________________________
Linux-nvdimm mailing list
Linux-nvdimm@lists.01.org
https://lists.01.org/mailman/listinfo/linux-nvdimm
next prev parent reply other threads:[~2018-03-13 23:02 UTC|newest]
Thread overview: 68+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-03-12 19:35 [PATCH v3 00/11] Copy Offload in NVMe Fabrics with P2P PCI Memory Logan Gunthorpe
2018-03-12 19:35 ` [PATCH v3 01/11] PCI/P2PDMA: Support peer-to-peer memory Logan Gunthorpe
2018-03-13 3:28 ` Sinan Kaya
2018-03-13 16:43 ` Logan Gunthorpe
2018-03-13 17:49 ` Sinan Kaya
2018-03-13 18:44 ` Logan Gunthorpe
2018-03-13 19:10 ` Sinan Kaya
2018-03-13 19:19 ` Logan Gunthorpe
2018-03-13 19:53 ` Sinan Kaya
2018-03-13 20:46 ` Logan Gunthorpe
2018-03-13 21:22 ` Sinan Kaya
2018-03-13 22:00 ` Logan Gunthorpe
2018-03-13 22:29 ` Sinan Kaya
2018-03-13 22:45 ` Stephen Bates
2018-03-13 22:48 ` Logan Gunthorpe
2018-03-13 23:19 ` Sinan Kaya
2018-03-13 23:45 ` Logan Gunthorpe
2018-03-14 12:16 ` David Laight
2018-03-14 16:23 ` Logan Gunthorpe
2018-03-13 22:31 ` Stephen Bates
2018-03-13 23:08 ` Bjorn Helgaas [this message]
2018-03-13 23:21 ` Logan Gunthorpe
2018-03-14 2:56 ` Bjorn Helgaas
2018-03-14 14:05 ` Stephen Bates
2018-03-14 16:17 ` Logan Gunthorpe
2018-03-14 18:51 ` Bjorn Helgaas
2018-03-14 19:03 ` Logan Gunthorpe
2018-03-14 19:28 ` Dan Williams
2018-03-14 19:30 ` Logan Gunthorpe
2018-03-14 19:34 ` Stephen Bates
2018-03-15 4:00 ` Martin K. Petersen
2018-03-15 4:30 ` Dan Williams
2018-03-22 22:57 ` Stephen Bates
2018-03-23 21:50 ` Bjorn Helgaas
2018-03-23 21:59 ` Logan Gunthorpe
2018-03-24 3:49 ` Bjorn Helgaas
2018-03-24 15:28 ` Stephen Bates
2018-03-26 15:43 ` Logan Gunthorpe
2018-03-26 11:11 ` Jonathan Cameron
2018-03-26 14:01 ` Bjorn Helgaas
2018-03-26 15:46 ` Logan Gunthorpe
2018-03-27 8:47 ` Jonathan Cameron
2018-03-27 15:37 ` Logan Gunthorpe
2018-04-13 21:56 ` Stephen Bates
2018-03-26 16:41 ` Jason Gunthorpe
2018-03-26 17:30 ` Logan Gunthorpe
2018-03-26 19:35 ` Jason Gunthorpe
2018-03-26 20:42 ` Logan Gunthorpe
2018-03-13 18:40 ` Logan Gunthorpe
2018-03-12 19:35 ` [PATCH v3 02/11] PCI/P2PDMA: Add sysfs group to display p2pmem stats Logan Gunthorpe
2018-03-12 19:35 ` [PATCH v3 03/11] PCI/P2PDMA: Add PCI p2pmem dma mappings to adjust the bus offset Logan Gunthorpe
2018-03-12 19:35 ` [PATCH v3 04/11] PCI/P2PDMA: Clear ACS P2P flags for all devices behind switches Logan Gunthorpe
2018-03-12 19:35 ` [PATCH v3 05/11] PCI/P2PDMA: Add P2P DMA driver writer's documentation Logan Gunthorpe
2018-03-12 19:41 ` Jonathan Corbet
2018-03-12 21:18 ` Logan Gunthorpe
2018-03-12 19:35 ` [PATCH v3 06/11] block: Introduce PCI P2P flags for request and request queue Logan Gunthorpe
2018-03-21 9:27 ` Christoph Hellwig
2018-03-12 19:35 ` [PATCH v3 07/11] IB/core: Ensure we map P2P memory correctly in rdma_rw_ctx_[init|destroy]() Logan Gunthorpe
2018-03-21 9:27 ` Christoph Hellwig
2018-03-12 19:35 ` [PATCH v3 08/11] nvme-pci: Use PCI p2pmem subsystem to manage the CMB Logan Gunthorpe
2018-03-13 1:55 ` Sinan Kaya
2018-03-13 1:58 ` Sinan Kaya
2018-03-12 19:35 ` [PATCH v3 09/11] nvme-pci: Add support for P2P memory in requests Logan Gunthorpe
2018-03-21 9:23 ` Christoph Hellwig
2018-03-12 19:35 ` [PATCH v3 10/11] nvme-pci: Add a quirk for a pseudo CMB Logan Gunthorpe
2018-03-12 19:35 ` [PATCH v3 11/11] nvmet: Optionally use PCI P2P memory Logan Gunthorpe
2018-03-21 9:27 ` Christoph Hellwig
2018-03-21 16:52 ` Logan Gunthorpe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180313230850.GA45763@bhelgaas-glaptop.roam.corp.google.com \
--to=helgaas@kernel.org \
--cc=alex.williamson@redhat.com \
--cc=axboe@kernel.dk \
--cc=benh@kernel.crashing.org \
--cc=bhelgaas@google.com \
--cc=hch@lst.de \
--cc=jgg@mellanox.com \
--cc=jglisse@redhat.com \
--cc=keith.busch@intel.com \
--cc=linux-block@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-nvdimm@lists.01.org \
--cc=linux-nvme@lists.infradead.org \
--cc=linux-pci@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=maxg@mellanox.com \
--cc=okaya@codeaurora.org \
--cc=sbates@raithlin.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).