smbd_max_receive_size == 8192?

smbd_max_receive_size == 8192?

[Tom Talpey]

Long, I notice that smbdirect.c sets the max receive size to 8192.
It's tunable, but I'm curious why the default is so large. The
SMBDirect protocol normally limits its packets to 1364 bytes.

With an SMBDirect credit limit of 255, the present default allocates
over 500 pages/connection, in O(2) granularity, when 85 O(1) pages
would suffice. Kernel virtual mapping would also be greatly reduced
as buffers can be arranged to share, and never extend beyond, a page.

Any insight into the 8192? Thanks.

Tom.

Re: smbd_max_receive_size == 8192?

[Tom Talpey]

...ping?

I have since noticed that the smbdirect default max send size is 1364,
so I'm even more confused on the 8192. And, ksmbd sets both sizes
to 8192.

On 8/24/2022 11:44 AM, Tom Talpey wrote:
> Long, I notice that smbdirect.c sets the max receive size to 8192.
> It's tunable, but I'm curious why the default is so large. The
> SMBDirect protocol normally limits its packets to 1364 bytes.
> 
> With an SMBDirect credit limit of 255, the present default allocates
> over 500 pages/connection, in O(2) granularity, when 85 O(1) pages
> would suffice. Kernel virtual mapping would also be greatly reduced
> as buffers can be arranged to share, and never extend beyond, a page.
> 
> Any insight into the 8192? Thanks.
> 
> Tom.
> 

RE: smbd_max_receive_size == 8192?

[Long Li]

> Subject: Re: smbd_max_receive_size == 8192?
> 
> ...ping?
> 
> I have since noticed that the smbdirect default max send size is 1364, so I'm
> even more confused on the 8192. And, ksmbd sets both sizes to 8192.

Yes, it seems it can be reduced to a smaller value. I don't remember the reason why it was set to 8192 initially.

This value is further override by the preferred_send_size in SMBD negotiation response from the server. In practice, it could be much lower than 8192.

> 
> On 8/24/2022 11:44 AM, Tom Talpey wrote:
> > Long, I notice that smbdirect.c sets the max receive size to 8192.
> > It's tunable, but I'm curious why the default is so large. The
> > SMBDirect protocol normally limits its packets to 1364 bytes.
> >
> > With an SMBDirect credit limit of 255, the present default allocates
> > over 500 pages/connection, in O(2) granularity, when 85 O(1) pages
> > would suffice. Kernel virtual mapping would also be greatly reduced as
> > buffers can be arranged to share, and never extend beyond, a page.
> >
> > Any insight into the 8192? Thanks.
> >
> > Tom.
> >

Re: smbd_max_receive_size == 8192?

[Tom Talpey]

Thanks. As part of my "too_many_sges" branch I'm experimenting with smaller buffers, with very good results. Using standard 1364-byte segments and 6 SGEs on both sides, it passes connectathon tests and normal workloads over both softiwarp and softroce. I have another patch to use fragmentation to further reduce SGEs that should improve performance on hardware adapters.

It bothers me that there are two implementations of smbdirect in the code, fs/cifs/smbdirect.c and fs/ksmbd/transport_rdma.c. Because smbdirect is purely a framing protocol and therefore peer-to-peer, this seems to be unnecessary duplication. I plan to experiment with refactoring them and split out the common code later.

Tom.

Sep 2, 2022 10:17:32 PM Long Li <longli@microsoft.com>:

>> Subject: Re: smbd_max_receive_size == 8192?
>> 
>> ...ping?
>> 
>> I have since noticed that the smbdirect default max send size is 1364, so I'm
>> even more confused on the 8192. And, ksmbd sets both sizes to 8192.
> 
> Yes, it seems it can be reduced to a smaller value. I don't remember the reason why it was set to 8192 initially.
> 
> This value is further override by the preferred_send_size in SMBD negotiation response from the server. In practice, it could be much lower than 8192.
> 
>> 
>> On 8/24/2022 11:44 AM, Tom Talpey wrote:
>>> Long, I notice that smbdirect.c sets the max receive size to 8192.
>>> It's tunable, but I'm curious why the default is so large. The
>>> SMBDirect protocol normally limits its packets to 1364 bytes.
>>> 
>>> With an SMBDirect credit limit of 255, the present default allocates
>>> over 500 pages/connection, in O(2) granularity, when 85 O(1) pages
>>> would suffice. Kernel virtual mapping would also be greatly reduced as
>>> buffers can be arranged to share, and never extend beyond, a page.
>>> 
>>> Any insight into the 8192? Thanks.
>>> 
>>> Tom.
>>>