All of lore.kernel.org
 help / color / mirror / Atom feed
From: Bob Pearson <rpearsonhpe@gmail.com>
To: Jason Gunthorpe <jgg@nvidia.com>
Cc: leon@kernel.org, zyjzyj2000@gmail.com, linux-rdma@vger.kernel.org
Subject: Re: [PATCH for-next v2 06/18] RDMA/rxe: Add rxe_add_frag() to rxe_mr.c
Date: Thu, 1 Dec 2022 09:04:39 -0600	[thread overview]
Message-ID: <d80f31c4-2e51-c726-2954-a7039befe329@gmail.com> (raw)
In-Reply-To: <Y4f4NkV+4ZbubQjp@nvidia.com>

On 11/30/22 18:41, Jason Gunthorpe wrote:
> On Wed, Nov 30, 2022 at 06:36:56PM -0600, Bob Pearson wrote:
>> I'm not looking at my patch you responded to but the one you posted to replace maps
>> by xarrays.
> 
> I see, I botched that part
> 
>> The existing rxe driver assumes that if ibmr->type == IB_MR_TYPE_DMA
>> that the iova is just a kernel (virtual) address that is already
>> mapped.
> 
> No, it is not correct
> 
>> Maybe this is not correct but it has always worked this way. These
>> are heavily used by storage stacks (e.g. Lustre) which always use
>> DMA mr's. Since we don't actually do any DMAs we don't need to setup
>> the iommu for these and just do memcpy's without dealing with pages.
> 
> You still should be doing the kmap
> 
> Jason

Something was disconnected in my memory. So I went back and looked at lustre.
Turns out it never uses IB_MR_TYPE_DMA and for that matter I can't find any
use cases in the rdma tree or online. So, the implementation in rxe has almost
certainly never been used.

So I need to choose to 'fix' the current implementation or just delete type dma support.
I get the idea that I need to convert the iova to a page and kmap it but i'm not
clear how to do that. This 64 bit numnber (iova) needs to convert to a struct page *.
Without a use case to look at I don't know how to interpret it. Apparently it's not a
virtual address.

Bob


  parent reply	other threads:[~2022-12-01 15:05 UTC|newest]

Thread overview: 36+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-10-31 20:27 [PATCH for-next v2 01/18] RDMA/rxe: Isolate code to fill request roce headers Bob Pearson
2022-10-31 20:27 ` [PATCH for-next v2 02/18] RDMA/rxe: Isolate request payload code in a subroutine Bob Pearson
2022-10-31 20:27 ` [PATCH for-next v2 03/18] RDMA/rxe: Remove paylen parameter from rxe_init_packet Bob Pearson
2022-10-31 20:27 ` [PATCH for-next v2 04/18] RDMA/rxe: Isolate code to build request packet Bob Pearson
2022-10-31 20:27 ` [PATCH for-next v2 05/18] RDMA/rxe: Add sg fragment ops Bob Pearson
2022-11-24 19:05   ` Jason Gunthorpe
2022-10-31 20:27 ` [PATCH for-next v2 06/18] RDMA/rxe: Add rxe_add_frag() to rxe_mr.c Bob Pearson
2022-11-24 19:10   ` Jason Gunthorpe
2022-11-30 20:53     ` Bob Pearson
2022-11-30 23:36       ` Jason Gunthorpe
2022-12-01  0:16         ` Bob Pearson
2022-12-01  0:20           ` Jason Gunthorpe
2022-12-01  0:36             ` Bob Pearson
2022-12-01  0:41               ` Jason Gunthorpe
2022-12-01  5:05                 ` Bob Pearson
2022-12-01 12:51                   ` Jason Gunthorpe
2022-12-01 15:04                 ` Bob Pearson [this message]
2022-12-01 15:16                   ` Bob Pearson
2022-12-01 15:38                     ` Bob Pearson
2022-12-01 15:39                       ` Jason Gunthorpe
2022-12-01 17:11                         ` Bob Pearson
2022-12-01 18:00                           ` Jason Gunthorpe
2022-10-31 20:27 ` [PATCH for-next v2 07/18] RDMA/rxe: Add routine to compute the number of frags Bob Pearson
2022-11-24 19:15   ` Jason Gunthorpe
2022-10-31 20:27 ` [PATCH for-next v2 08/18] RDMA/rxe: Extend rxe_mr_copy to support skb frags Bob Pearson
2022-10-31 20:27 ` [PATCH for-next v2 09/18] RDMA/rxe: Add routine to compute number of frags for dma Bob Pearson
2022-11-24 19:16   ` Jason Gunthorpe
2022-10-31 20:27 ` [PATCH for-next v2 10/18] RDMA/rxe: Extend copy_data to support skb frags Bob Pearson
2022-10-31 20:28 ` [PATCH for-next v2 11/18] RDMA/rxe: Replace rxe by qp as a parameter Bob Pearson
2022-10-31 20:28 ` [PATCH for-next v2 12/18] RDMA/rxe: Extend rxe_init_packet() to support frags Bob Pearson
2022-10-31 20:28 ` [PATCH for-next v2 13/18] RDMA/rxe: Extend rxe_icrc.c " Bob Pearson
2022-10-31 20:28 ` [PATCH for-next v2 14/18] RDMA/rxe: Extend rxe_init_req_packet() for frags Bob Pearson
2022-10-31 20:28 ` [PATCH for-next v2 15/18] RDMA/rxe: Extend response packets " Bob Pearson
2022-10-31 20:28 ` [PATCH for-next v2 16/18] RDMA/rxe: Extend send/write_data_in() " Bob Pearson
2022-10-31 20:28 ` [PATCH for-next v2 17/18] RDMA/rxe: Extend do_read() in rxe_comp,c " Bob Pearson
2022-10-31 20:28 ` [PATCH for-next v2 18/18] RDMA/rxe: Enable sg code in rxe Bob Pearson

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=d80f31c4-2e51-c726-2954-a7039befe329@gmail.com \
    --to=rpearsonhpe@gmail.com \
    --cc=jgg@nvidia.com \
    --cc=leon@kernel.org \
    --cc=linux-rdma@vger.kernel.org \
    --cc=zyjzyj2000@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.