Re: [RFC PATCH 0/6] Reduce SMBDirect RDMA SGE counts and sizes

From: Tom Talpey <tom@talpey.com>
To: Steve French <smfrench@gmail.com>
Cc: linux-cifs@vger.kernel.org, linkinjeon@kernel.org,
	senozhatsky@chromium.org, bmt@zurich.ibm.com
Subject: Re: [RFC PATCH 0/6] Reduce SMBDirect RDMA SGE counts and sizes
Date: Thu, 22 Sep 2022 15:35:37 -0400	[thread overview]
Message-ID: <c68dd87d-0c42-2673-6467-e1063c4fa9a3@talpey.com> (raw)
In-Reply-To: <CAH2r5muhROptfZ+fUjhCfwzoeJqNku4Nkpjv63N5WTdhpEWWoA@mail.gmail.com>

[ Resending to fix the missing "vger."! ]

On 9/20/2022 9:58 PM, Steve French wrote:
> are these still current?

Hi Steve, yes these are current. As you know we were going to add
siw-based RDMA testing to the buildbot during last week's IOLab,
but we didn't get the chance. And this week, I've been forced to
self-isolate and I therefore have had no chance to test further.

Here's the deal:

1) The changes are tested by me as described in the message. They
passed all my confidence checks, including artificially reducing
SGEs all the way down to 2.

2) But, I haven't seen any reviews, or anyone testing.

3) I won't be very productive until next week, assuming I feel better.
I'll be back on it as soon as I can.

4) Because neither client nor server ever worked over softiWARP before,
the patches shouldn't go to stable.

5) Because they only address RDMA operation, they won't break ordinary
users.

Your call.

Tom.

> On Tue, Sep 13, 2022 at 5:04 PM Tom Talpey <tom@talpey.com> wrote:
>>
>> Allocate fewer SGEs and standard packet sizes in both kernel SMBDirect
>> implementations.
>>
>> The current maximum values (16 SGEs and 8192 bytes) cause failures on the
>> SoftiWARP provider, and are suboptimal on others. Reduce these to 6 and
>> 1364. Additionally, recode smbd_send() to work with as few as 2 SGEs,
>> and for debug sanity, reformat client-side logging to more clearly show
>> addresses, lengths and flags in the appropriate base.
>>
>> Tested over SoftiWARP and SoftRoCE with shell, Connectathon basic and general,
>> and to be tested with XFStests here at SMB3 IOLab this week.
>>
>> Tom Talpey (6):
>>    Decrease the number of SMB3 smbdirect client SGEs
>>    Decrease the number of SMB3 smbdirect server SGEs
>>    Reduce client smbdirect max receive segment size
>>    Reduce server smbdirect max send/receive segment sizes
>>    Handle variable number of SGEs in client smbdirect send.
>>    Fix formatting of client smbdirect RDMA logging
>>
>>   fs/cifs/smbdirect.c       | 227 ++++++++++++++++----------------------
>>   fs/cifs/smbdirect.h       |  14 ++-
>>   fs/ksmbd/transport_rdma.c |   6 +-
>>   3 files changed, 109 insertions(+), 138 deletions(-)
>>
>> --
>> 2.34.1
>>
> 
> 