All of lore.kernel.org
 help / color / mirror / Atom feed
From: Tom Talpey <tom@talpey.com>
To: David Howells <dhowells@redhat.com>,
	Long Li <longli@microsoft.com>,
	Namjae Jeon <linkinjeon@kernel.org>
Cc: Hyunchul Lee <hyc.lee@gmail.com>,
	Steve French <smfrench@gmail.com>,
	CIFS <linux-cifs@vger.kernel.org>
Subject: Re: RDMA (smbdirect) testing
Date: Sat, 21 May 2022 07:54:22 -0400	[thread overview]
Message-ID: <dc1dff3e-d19e-4300-41b8-ccb7459eacde@talpey.com> (raw)
In-Reply-To: <84589.1653070372@warthog.procyon.org.uk>


On 5/20/2022 2:12 PM, David Howells wrote:
> Tom Talpey <tom@talpey.com> wrote:
> 
>> SoftROCE is a bit of a hot mess in upstream right now. It's
>> getting a lot of attention, but it's still pretty shaky.
>> If you're testing, I'd STRONGLY recommend SoftiWARP.
> 
> I'm having problems getting that working.  I'm setting the client up with:
> 
> rdma link add siw0 type siw netdev enp6s0
> mount //192.168.6.1/scratch /xfstest.scratch -o rdma,user=shares,pass=...
> 
> and then see:
> 
> CIFS: Attempting to mount \\192.168.6.1\scratch
> CIFS: VFS: _smbd_get_connection:1513 warning: device max_send_sge = 6 too small
> CIFS: VFS: _smbd_get_connection:1516 Queue Pair creation may fail
> CIFS: VFS: _smbd_get_connection:1519 warning: device max_recv_sge = 6 too small
> CIFS: VFS: _smbd_get_connection:1522 Queue Pair creation may fail
> CIFS: VFS: _smbd_get_connection:1559 rdma_create_qp failed -22
> CIFS: VFS: _smbd_get_connection:1513 warning: device max_send_sge = 6 too small
> CIFS: VFS: _smbd_get_connection:1516 Queue Pair creation may fail
> CIFS: VFS: _smbd_get_connection:1519 warning: device max_recv_sge = 6 too small
> CIFS: VFS: _smbd_get_connection:1522 Queue Pair creation may fail
> CIFS: VFS: _smbd_get_connection:1559 rdma_create_qp failed -22
> CIFS: VFS: cifs_mount failed w/return code = -2
> 
> in dmesg.
> 
> Problem is, I don't know what to do about it:-/

It looks like the client is hardcoding 16 sge's, and has no option to
configure a smaller value, or reduce its requested number. That's bad,
because providers all have their own limits - and SIW_MAX_SGE is 6. I
thought I'd seen this working (metze?), but either the code changed or
someone built a custom version.

Namjae/Long, have you used siw successfully? Why does the code require
16 sge's, regardless of other size limits? Normally, if the lower layer
supports fewer, the upper layer will simply reduce its operation sizes.

Tom.

  reply	other threads:[~2022-05-21 11:54 UTC|newest]

Thread overview: 42+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-05-19 20:41 RDMA (smbdirect) testing Steve French
2022-05-19 23:06 ` Namjae Jeon
2022-05-20  6:01   ` Hyunchul Lee
2022-05-20 18:03     ` Tom Talpey
2022-05-20 18:12     ` David Howells
2022-05-21 11:54       ` Tom Talpey [this message]
2022-05-22 23:06         ` Namjae Jeon
2022-05-23 13:45           ` Tom Talpey
2022-05-23 15:05             ` Namjae Jeon
2022-05-23 16:05               ` Tom Talpey
2022-05-23 19:17                 ` Long Li
2022-05-24  1:01                   ` Namjae Jeon
2022-05-24 21:08                     ` Long Li
2022-06-02 23:32                       ` Namjae Jeon
2022-06-03  0:07                         ` Long Li
2022-06-07 17:26                           ` Tom Talpey
2022-06-07 22:25                             ` Namjae Jeon
2022-05-24  0:59                 ` Namjae Jeon
2022-05-24  9:16               ` David Howells
2022-05-24 17:49                 ` Steve French
2022-05-24 18:12                   ` Tom Talpey
2022-05-25  9:29             ` David Howells
2022-05-25  9:41             ` David Howells
2022-05-25 10:00               ` Stefan Metzmacher
2022-05-25 10:20               ` David Howells
2022-05-26 14:56                 ` Stefan Metzmacher
2022-05-26 15:52                   ` Tom Talpey
2022-05-27  8:27                     ` Stefan Metzmacher
2022-05-27 11:46                     ` David Howells
2022-05-27 13:45                       ` Stefan Metzmacher
2022-05-27 22:22                       ` David Howells
2022-08-02 15:10             ` David Howells
2022-08-03  0:55               ` Namjae Jeon
2022-08-03  2:36                 ` Namjae Jeon
2022-08-03  6:16                 ` David Howells
     [not found]         ` <747882.1653311226@warthog.procyon.org.uk>
2022-05-23 13:37           ` Tom Talpey
2022-05-23 14:03         ` Stefan Metzmacher
2022-05-25  9:35         ` David Howells
2022-05-20  6:20 ` David Howells
2022-05-20  8:37   ` Namjae Jeon
2022-05-24 20:12 ` David Howells
2022-05-27 10:33 ` UAF in smbd_reconnect() when using softIWarp David Howells

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=dc1dff3e-d19e-4300-41b8-ccb7459eacde@talpey.com \
    --to=tom@talpey.com \
    --cc=dhowells@redhat.com \
    --cc=hyc.lee@gmail.com \
    --cc=linkinjeon@kernel.org \
    --cc=linux-cifs@vger.kernel.org \
    --cc=longli@microsoft.com \
    --cc=smfrench@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.