linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Dennis Dalessandro <dennis.dalessandro@cornelisnetworks.com>
To: Leon Romanovsky <leon@kernel.org>
Cc: "Marciniszyn, Mike" <mike.marciniszyn@cornelisnetworks.com>,
	Haakon Bugge <haakon.bugge@oracle.com>,
	Doug Ledford <dledford@redhat.com>,
	Jason Gunthorpe <jgg@nvidia.com>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	OFED mailing list <linux-rdma@vger.kernel.org>
Subject: Re: [PATCH rdma-next] RDMA/rdmavt: Decouple QP and SGE lists allocations
Date: Wed, 12 May 2021 08:45:45 -0400	[thread overview]
Message-ID: <7284da8c-9993-76c4-b495-32c814607a4b@cornelisnetworks.com> (raw)
In-Reply-To: <YJvGWPimIFbptgdC@unreal>

On 5/12/21 8:13 AM, Leon Romanovsky wrote:
> On Wed, May 12, 2021 at 12:08:59AM -0400, Dennis Dalessandro wrote:
>>
>> On 5/11/21 3:27 PM, Leon Romanovsky wrote:
>>> On Tue, May 11, 2021 at 07:15:09PM +0000, Marciniszyn, Mike wrote:
>>>>>>
>>>>>> Why not kzalloc_node() here?
>>>>
>>>> I agree here.
>>>>
>>>> Other allocations that have been promoted to the core have lost the node attribute in the allocation.
>>>
>>> Did you notice any performance degradation?
>>>
>>
>> So what's the motivation to change it from the way it was originally
>> designed? It seems to me if the original implementation went to the trouble
>> to allocate the memory on the local node, refactoring the code should
>> respect that.
> 
> I have no problem to make rdma_zalloc_*() node aware, but would like to get
> real performance justification. My assumption is that rdmavt use kzalloc_node
> for the control plane based on some internal performance testing and we finally
> can see the difference between kzalloc and kzalloc_node, am I right?
> 
> Is the claim of performance degradation backed by data?

Yes, in the past. I don't have access anymore now that I'm not with 
Intel. It probably would not have been publishable anyway.

> The main reason (maybe I'm wrong here) is to avoid _node() allocators
> because they increase chances of memory allocation failure due to not
> doing fallback in case node memory is depleted.

Agreed. It's a trade-off that was deemed acceptable.

> Again, I'm suggesting to do plain kzalloc() for control part of QP.

Now I don't recall data for that specifically, but to be on the safe 
side I would not want to risk a performance regression.

-Denny

  reply	other threads:[~2021-05-12 12:45 UTC|newest]

Thread overview: 38+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-05-11 10:36 [PATCH rdma-next] RDMA/rdmavt: Decouple QP and SGE lists allocations Leon Romanovsky
2021-05-11 10:59 ` Haakon Bugge
2021-05-11 12:34   ` Leon Romanovsky
2021-05-11 19:15     ` Marciniszyn, Mike
2021-05-11 19:27       ` Leon Romanovsky
2021-05-11 19:39         ` Marciniszyn, Mike
2021-05-12  4:08         ` Dennis Dalessandro
2021-05-12 12:13           ` Leon Romanovsky
2021-05-12 12:45             ` Dennis Dalessandro [this message]
2021-05-11 12:26 ` Dennis Dalessandro
2021-05-11 12:34   ` Leon Romanovsky
2021-05-12 12:25     ` Marciniszyn, Mike
2021-05-12 12:50       ` Leon Romanovsky
2021-05-13 19:03         ` Dennis Dalessandro
2021-05-13 19:15           ` Jason Gunthorpe
2021-05-13 19:31             ` Dennis Dalessandro
2021-05-14 13:02               ` Jason Gunthorpe
2021-05-14 14:07                 ` Dennis Dalessandro
2021-05-14 14:35                   ` Jason Gunthorpe
2021-05-14 15:00                     ` Marciniszyn, Mike
2021-05-14 15:02                       ` Jason Gunthorpe
2021-05-19  7:50                         ` Leon Romanovsky
2021-05-19 11:56                           ` Dennis Dalessandro
2021-05-19 18:29                             ` Jason Gunthorpe
2021-05-19 19:49                               ` Dennis Dalessandro
2021-05-19 20:26                                 ` Jason Gunthorpe
2021-05-20 22:02                                   ` Dennis Dalessandro
2021-05-21  6:29                                     ` Leon Romanovsky
2021-05-25 13:13                                     ` Jason Gunthorpe
2021-05-25 14:10                                       ` Dennis Dalessandro
2021-05-25 14:20                                         ` Jason Gunthorpe
2021-05-25 14:29                                           ` Dennis Dalessandro
2021-06-28 21:59                                           ` Marciniszyn, Mike
2021-06-28 23:19                                             ` Jason Gunthorpe
2021-07-04  6:34                                               ` Leon Romanovsky
2021-06-02  4:33                                         ` Leon Romanovsky
2021-05-16 10:56           ` Leon Romanovsky
2021-05-12 12:23 ` Marciniszyn, Mike

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=7284da8c-9993-76c4-b495-32c814607a4b@cornelisnetworks.com \
    --to=dennis.dalessandro@cornelisnetworks.com \
    --cc=dledford@redhat.com \
    --cc=haakon.bugge@oracle.com \
    --cc=jgg@nvidia.com \
    --cc=leon@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-rdma@vger.kernel.org \
    --cc=mike.marciniszyn@cornelisnetworks.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).