From: Alex Williamson <alex.williamson@redhat.com>
To: Srinath Mannam <srinath.mannam@broadcom.com>
Cc: Sinan Kaya <okaya@codeaurora.org>, Christoph Hellwig <hch@lst.de>,
Bjorn Helgaas <bhelgaas@google.com>,
Abhishek Shah <abhishek.shah@broadcom.com>,
Vikram Prakash <vikram.prakash@broadcom.com>,
linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org,
linux-nvme@lists.infradead.org, kvm@vger.kernel.org,
linux-pci-owner@vger.kernel.org
Subject: Re: Requirement to get BAR pci_bus_address in user space
Date: Thu, 14 Jun 2018 08:50:15 -0600 [thread overview]
Message-ID: <20180614085015.3f39b367@w520.home> (raw)
In-Reply-To: <CABe79T5ZRBToOgCY0Myo47mkeOQ5FWrxgF-Bbdx-Pa3fmYPDGw@mail.gmail.com>
On Thu, 14 Jun 2018 16:18:15 +0530
Srinath Mannam <srinath.mannam@broadcom.com> wrote:
> Hi Sinan Kaya,
>
> Here are the details,
>
> The issue is, For CMB cards SQs are allocated inside device BAR memory
> which is different from normal cards.
> In Normal cards SQ memory allocated at host side.
> In both the cases physical address of CQ memory is programmed in NVMe
> controller register.
> This method works for normal cards because CQ memory is at host side.
> But in CMB cards pci bus address equivalent to CQ memory needs to program.
>
> More details are in the patch: nvme-pci: Use PCI bus address for
> data/queues in CMB.
>
> With the above patch issue is fixed in the NVMe kernel driver, But
> similar fix is required in SPDK library also.
> So, We need a mechanism to get pci_bus_address in user space libraries
> to address this issue.
I don't understand the CQ vs CMB, but I think I gather that there's some
sort of buffer that's allocated from within the devices MMIO BAR and
some programming of the device needs to reference that buffer.
Wouldn't you therefore use the vfio type1 IOMMU MAP_DMA ioctl to map
the BAR into the IOVA address space and you can then use the IOVA +
offset into the BAR for the device to reference the buffer? It seems
this is the same way we'd setup a peer-to-peer mapping, but we're using
it for the device to reference itself effectively. Thanks,
Alex
next prev parent reply other threads:[~2018-06-14 14:50 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-06-14 10:07 Requirement to get BAR pci_bus_address in user space Srinath Mannam
2018-06-14 10:20 ` Christoph Hellwig
2018-06-14 10:29 ` Srinath Mannam
2018-06-14 10:33 ` okaya
2018-06-14 10:48 ` Srinath Mannam
2018-06-14 14:42 ` Keith Busch
2018-06-14 14:50 ` Alex Williamson [this message]
2018-06-14 14:59 ` Christoph Hellwig
2018-06-14 20:04 ` Walker, Benjamin
2018-06-15 6:11 ` Srinath Mannam
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180614085015.3f39b367@w520.home \
--to=alex.williamson@redhat.com \
--cc=abhishek.shah@broadcom.com \
--cc=bhelgaas@google.com \
--cc=hch@lst.de \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-nvme@lists.infradead.org \
--cc=linux-pci-owner@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=okaya@codeaurora.org \
--cc=srinath.mannam@broadcom.com \
--cc=vikram.prakash@broadcom.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).