From: Gal Pressman <galpress@amazon.com>
To: Jason Gunthorpe <jgg@ziepe.ca>
Cc: "Sumit Semwal" <sumit.semwal@linaro.org>,
"Christian König" <christian.koenig@amd.com>,
"Doug Ledford" <dledford@redhat.com>,
linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org,
linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org,
"Oded Gabbay" <ogabbay@habana.ai>,
"Tomer Tayar" <ttayar@habana.ai>,
"Yossi Leybovich" <sleybo@amazon.com>,
"Alexander Matushevsky" <matua@amazon.com>,
"Leon Romanovsky" <leonro@nvidia.com>,
"Jianxin Xiong" <jianxin.xiong@intel.com>,
"Firas Jahjah" <firasj@amazon.com>
Subject: Re: [RFC PATCH 2/2] RDMA/efa: Add support for dmabuf memory regions
Date: Sun, 10 Oct 2021 09:55:49 +0300 [thread overview]
Message-ID: <77082c57-29f8-1eba-b260-7cb658ec34d1@amazon.com> (raw)
In-Reply-To: <20211007114018.GD2688930@ziepe.ca>
On 07/10/2021 14:40, Jason Gunthorpe wrote:
> On Thu, Oct 07, 2021 at 01:43:00PM +0300, Gal Pressman wrote:
>
>> @@ -1491,26 +1493,29 @@ static int efa_create_pbl(struct efa_dev *dev,
>> return 0;
>> }
>>
>> -struct ib_mr *efa_reg_mr(struct ib_pd *ibpd, u64 start, u64 length,
>> - u64 virt_addr, int access_flags,
>> - struct ib_udata *udata)
>> +static void efa_dmabuf_invalidate_cb(struct dma_buf_attachment *attach)
>> +{
>> + WARN_ON_ONCE(1,
>> + "Invalidate callback should not be called when memory is pinned\n");
>> +}
>> +
>> +static struct dma_buf_attach_ops efa_dmabuf_attach_ops = {
>> + .allow_peer2peer = true,
>> + .move_notify = efa_dmabuf_invalidate_cb,
>> +};
>
> Shouldn't move_notify really just be left as NULL? I mean fixing
> whatever is preventing that?
That's what I had in the previous RFC and I think Christian didn't really like it.
>> +struct ib_mr *efa_reg_user_mr_dmabuf(struct ib_pd *ibpd, u64 start,
>> + u64 length, u64 virt_addr,
>> + int fd, int access_flags,
>> + struct ib_udata *udata)
>> +{
>> + struct efa_dev *dev = to_edev(ibpd->device);
>> + struct ib_umem_dmabuf *umem_dmabuf;
>> + struct efa_mr *mr;
>> + int err;
>> +
>> + mr = efa_alloc_mr(ibpd, access_flags, udata);
>> + if (IS_ERR(mr)) {
>> + err = PTR_ERR(mr);
>> + goto err_out;
>> + }
>> +
>> + umem_dmabuf = ib_umem_dmabuf_get(ibpd->device, start, length, fd,
>> + access_flags, &efa_dmabuf_attach_ops);
>> + if (IS_ERR(umem_dmabuf)) {
>> + ibdev_dbg(&dev->ibdev, "Failed to get dmabuf[%d]\n", err);
>> + err = PTR_ERR(umem_dmabuf);
>> + goto err_free;
>> + }
>> +
>> + dma_resv_lock(umem_dmabuf->attach->dmabuf->resv, NULL);
>> + err = dma_buf_pin(umem_dmabuf->attach);
>> + if (err) {
>> + ibdev_dbg(&dev->ibdev, "Failed to pin dmabuf memory\n");
>> + goto err_release;
>> + }
>> +
>> + err = ib_umem_dmabuf_map_pages(umem_dmabuf);
>> + if (err) {
>> + ibdev_dbg(&dev->ibdev, "Failed to map dmabuf pages\n");
>> + goto err_unpin;
>> + }
>> + dma_resv_unlock(umem_dmabuf->attach->dmabuf->resv);
>
> If it is really this simple the core code should have this logic,
> 'ib_umem_dmabuf_get_pinned()' or something
Should get_pinned do just get + dma_buf_pin, or should it do
ib_umem_dmabuf_map_pages as well?
next prev parent reply other threads:[~2021-10-10 6:56 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-10-07 10:42 [RFC PATCH 0/2] EFA dmabuf memory regions Gal Pressman
2021-10-07 10:42 ` [RFC PATCH 1/2] dma-buf: Fix pin callback comment Gal Pressman
2021-10-07 10:44 ` Christian König
2021-10-10 6:50 ` Gal Pressman
2021-10-07 10:43 ` [RFC PATCH 2/2] RDMA/efa: Add support for dmabuf memory regions Gal Pressman
2021-10-07 11:40 ` Jason Gunthorpe
2021-10-10 6:55 ` Gal Pressman [this message]
2021-10-11 23:28 ` Jason Gunthorpe
2021-10-12 11:41 ` Gal Pressman
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=77082c57-29f8-1eba-b260-7cb658ec34d1@amazon.com \
--to=galpress@amazon.com \
--cc=christian.koenig@amd.com \
--cc=dledford@redhat.com \
--cc=dri-devel@lists.freedesktop.org \
--cc=firasj@amazon.com \
--cc=jgg@ziepe.ca \
--cc=jianxin.xiong@intel.com \
--cc=leonro@nvidia.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-media@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=matua@amazon.com \
--cc=ogabbay@habana.ai \
--cc=sleybo@amazon.com \
--cc=sumit.semwal@linaro.org \
--cc=ttayar@habana.ai \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).