From: Jason Gunthorpe <jgg@ziepe.ca>
To: Daniel Vetter <daniel@ffwll.ch>
Cc: "Leon Romanovsky" <leon@kernel.org>,
"linux-rdma@vger.kernel.org" <linux-rdma@vger.kernel.org>,
"dri-devel@lists.freedesktop.org"
<dri-devel@lists.freedesktop.org>,
"Doug Ledford" <dledford@redhat.com>,
"Vetter, Daniel" <daniel.vetter@intel.com>,
"Christian König" <christian.koenig@amd.com>,
"Xiong, Jianxin" <jianxin.xiong@intel.com>
Subject: Re: [RFC PATCH v3 1/4] RDMA/umem: Support importing dma-buf as user memory region
Date: Tue, 6 Oct 2020 15:38:34 -0300 [thread overview]
Message-ID: <20201006183834.GK5177@ziepe.ca> (raw)
In-Reply-To: <CAKMK7uEN7=QGOJMMf5UwR5Azsk+3-apFjn_hFmoSUTDOh1f85g@mail.gmail.com>
On Tue, Oct 06, 2020 at 08:17:05PM +0200, Daniel Vetter wrote:
> So on the gpu we pipeline this all. So step 4 doesn't happen on the
> cpu, but instead we queue up a bunch of command buffers so that the
> gpu writes these pagetables (and the flushes tlbs and then does the
> actual stuff userspace wants it to do).
mlx5 HW does basically this as well.
We just apply scheduling for this work on the device, not in the CPU.
> just queue it all up and let the gpu scheduler sort out the mess. End
> result is that you get a sgt that points at stuff which very well
> might have nothing even remotely resembling your buffer in there at
> the moment. But all the copy operations are queued up, so rsn the data
> will also be there.
The explanation make sense, thanks
> But rdma doesn't work like that, so it looks all a bit funny.
Well, I guess it could, but how would it make anything better? I can
overlap building the SGL and the device PTEs with something else doing
'move', but is that a workload that needs such agressive optimization?
> This is also why the precise semantics of move_notify for gpu<->gpu
> sharing took forever to discuss and are still a bit wip, because you
> have the inverse problem: The dma api mapping might still be there
Seems like this all makes a graph of operations, can't start the next
one until all deps are finished. Actually sounds a lot like futures.
Would be clearer if this attach API provided some indication that the
SGL is actually a future valid SGL..
Jason
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel
next prev parent reply other threads:[~2020-10-07 7:23 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-10-04 19:12 [RFC PATCH v3 0/4] RDMA: Add dma-buf support Jianxin Xiong
2020-10-04 19:12 ` [RFC PATCH v3 1/4] RDMA/umem: Support importing dma-buf as user memory region Jianxin Xiong
2020-10-05 10:54 ` Christian König
2020-10-05 16:19 ` Xiong, Jianxin
2020-10-05 13:13 ` Jason Gunthorpe
2020-10-05 16:18 ` Xiong, Jianxin
2020-10-05 16:33 ` Jason Gunthorpe
2020-10-05 19:41 ` Xiong, Jianxin
2020-10-06 9:22 ` Daniel Vetter
2020-10-06 15:26 ` Xiong, Jianxin
2020-10-06 15:49 ` Jason Gunthorpe
2020-10-06 16:34 ` Daniel Vetter
2020-10-06 17:24 ` Daniel Vetter
2020-10-06 18:02 ` Jason Gunthorpe
2020-10-06 18:17 ` Daniel Vetter
2020-10-06 18:38 ` Jason Gunthorpe [this message]
2020-10-06 19:12 ` Daniel Vetter
2020-10-07 7:13 ` Christian König
2020-10-06 16:40 ` Daniel Vetter
2020-10-04 19:12 ` [RFC PATCH v3 2/4] RDMA: Expand driver memory registration methods to support dma-buf Jianxin Xiong
2020-10-04 19:12 ` [RFC PATCH v3 3/4] RDMA/mlx5: Support dma-buf based userspace memory region Jianxin Xiong
2020-10-04 19:12 ` [RFC PATCH v3 4/4] RDMA/uverbs: Add uverbs command for dma-buf based MR registration Jianxin Xiong
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20201006183834.GK5177@ziepe.ca \
--to=jgg@ziepe.ca \
--cc=christian.koenig@amd.com \
--cc=daniel.vetter@intel.com \
--cc=daniel@ffwll.ch \
--cc=dledford@redhat.com \
--cc=dri-devel@lists.freedesktop.org \
--cc=jianxin.xiong@intel.com \
--cc=leon@kernel.org \
--cc=linux-rdma@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).