From: Jason Gunthorpe <jgg@ziepe.ca>
To: Bob Pearson <rpearsonhpe@gmail.com>
Cc: Bart Van Assche <bvanassche@acm.org>,
"linux-rdma@vger.kernel.org" <linux-rdma@vger.kernel.org>
Subject: Re: blktest/rxe almost working
Date: Tue, 7 Sep 2021 13:39:39 -0300 [thread overview]
Message-ID: <20210907163939.GW1200268@ziepe.ca> (raw)
In-Reply-To: <9e6783d0-554c-17de-c72f-fae766099480@gmail.com>
On Tue, Sep 07, 2021 at 11:35:17AM -0500, Bob Pearson wrote:
> Interesting. But if that is the case the bigger problem is the ib_map_mr_sg() call which updates the
> mapping. rxe definitely does look at the mr->rkey value but we could fix that. It also looks at the
> mapping which is updated by ib_map_mr_sg(). My impression is that HW also uses this mapping or does
> HW also copy all the FMRs into SRAM?
Yes, real HW has a copy of the DMA list. The sg in the mr struct is
for CPU use only.
It is not OK to use the CPU SG list inside the MR for DMA by HW, it
has to be synchronized with the WR.
> There seems to be an assumption that users will be looking at CQE.
Yes, the kernel has to be driven by CQE, not only for data transfer
but the DMA unmap of the SGL cannot be until after the invalidation
CQE is observed.
Ie the CPU should have two DMA lists active during the invalidation
cycle.
Jason
next prev parent reply other threads:[~2021-09-07 16:39 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-09-02 21:41 blktest/rxe almost working Bob Pearson
2021-09-02 23:38 ` Jason Gunthorpe
2021-09-03 22:18 ` Bob Pearson
2021-09-03 23:13 ` Bart Van Assche
2021-09-04 22:30 ` Jason Gunthorpe
2021-09-05 18:02 ` Bob Pearson
2021-09-07 12:01 ` Jason Gunthorpe
2021-09-07 16:35 ` Bob Pearson
2021-09-07 16:39 ` Jason Gunthorpe [this message]
2021-09-07 16:47 ` Bob Pearson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210907163939.GW1200268@ziepe.ca \
--to=jgg@ziepe.ca \
--cc=bvanassche@acm.org \
--cc=linux-rdma@vger.kernel.org \
--cc=rpearsonhpe@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).