Re: dynamic-sg patch has broken rdma_rxe

From: Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com>
To: Jason Gunthorpe <jgg@nvidia.com>
Cc: Maor Gottlieb <maorg@nvidia.com>, Christoph Hellwig <hch@lst.de>,
	Gal Pressman <galpress@amazon.com>,
	Bob Pearson <rpearsonhpe@gmail.com>,
	Leon Romanovsky <leonro@nvidia.com>,
	linux-rdma@vger.kernel.org
Subject: Re: dynamic-sg patch has broken rdma_rxe
Date: Tue, 20 Oct 2020 12:37:05 +0100	[thread overview]
Message-ID: <03541c89-92d0-2dc8-5e40-03f3fe527fef@linux.intel.com> (raw)
In-Reply-To: <20201019124822.GD6219@nvidia.com>

On 19/10/2020 13:48, Jason Gunthorpe wrote:
> On Mon, Oct 19, 2020 at 01:29:42PM +0100, Tvrtko Ursulin wrote:
>> On 19/10/2020 13:12, Jason Gunthorpe wrote:
>>> On Mon, Oct 19, 2020 at 10:50:14AM +0100, Tvrtko Ursulin wrote:
>>>>> overshoot the max_segment if it is not a multiple of PAGE_SIZE. Simply fix
>>>>> the alignment before starting and don't expose this implementation detail
>>>>> to the callers.
>>>>
>>>> What does not make complete sense to me is the statement that input
>>>> alignment requirement makes it impossible to connect to DMA layer, but then
>>>> the patch goes to align behind the covers anyway.
>>>>
>>>> At minimum the kerneldoc should explain that max_segment will still be
>>>> rounded down. But wouldn't it be better for the API to be more explicit and
>>>> just require aligned anyway?
>>>
>>> Why?
>>>
>>> The API is to not produce sge's with a length longer than max_segment,
>>> it isn't to produce sge's of exactly max_segment.
>>>
>>> Everything else is an internal detail
>>
>> (Half-)pretending it is disconnected from PAGE_SIZE, when it really isn't,
>> isn't the most obvious API design in my view.
> 
> It is not information the callers need to care about

Unless they want a 1024 byte segment - then they do need to care. The 
patch just changes the rules from "any non-zero aligned" to "any greater 
or equal than PAGE_SIZE". I am simply asking why in more specific terms.

>> In other words, if you let users pass in 4097 and it just works by rounding
>> down to 4096, but you don't let them pass 4095, I just find this odd.
> 
> That is also an implementation detail, there is nothing preventing
> smaller than page size other than complexity of implementing the
> algorithm. If something ever needed it, it could be done.

But does non aligned segment size make sense? Is there any current or 
future platform which will need to to warrant changing the API in this way?

>> My question was why not have callers pass in page aligned max segment like
>> today which makes it completely transparent.
> 
> Why put this confusing code in every caller? Especially for something
> a driver is supposed to call. Will just make bugs

For max_segment to be aligned is a requirement today so callers are 
ready. I still don't understand why change that, what will it help with? 
Is it just about the desire to be able to pass in UINT_MAX?

Regards,

Tvrtko