All of lore.kernel.org
 help / color / mirror / Atom feed
From: Dennis Dalessandro <dennis.dalessandro@cornelisnetworks.com>
To: Leon Romanovsky <leon@kernel.org>
Cc: "Marciniszyn, Mike" <mike.marciniszyn@cornelisnetworks.com>,
	Haakon Bugge <haakon.bugge@oracle.com>,
	Doug Ledford <dledford@redhat.com>,
	Jason Gunthorpe <jgg@nvidia.com>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	OFED mailing list <linux-rdma@vger.kernel.org>
Subject: Re: [PATCH rdma-next] RDMA/rdmavt: Decouple QP and SGE lists allocations
Date: Wed, 12 May 2021 08:45:45 -0400	[thread overview]
Message-ID: <7284da8c-9993-76c4-b495-32c814607a4b@cornelisnetworks.com> (raw)
In-Reply-To: <YJvGWPimIFbptgdC@unreal>

On 5/12/21 8:13 AM, Leon Romanovsky wrote:
> On Wed, May 12, 2021 at 12:08:59AM -0400, Dennis Dalessandro wrote:
>>
>> On 5/11/21 3:27 PM, Leon Romanovsky wrote:
>>> On Tue, May 11, 2021 at 07:15:09PM +0000, Marciniszyn, Mike wrote:
>>>>>>
>>>>>> Why not kzalloc_node() here?
>>>>
>>>> I agree here.
>>>>
>>>> Other allocations that have been promoted to the core have lost the node attribute in the allocation.
>>>
>>> Did you notice any performance degradation?
>>>
>>
>> So what's the motivation to change it from the way it was originally
>> designed? It seems to me if the original implementation went to the trouble
>> to allocate the memory on the local node, refactoring the code should
>> respect that.
> 
> I have no problem to make rdma_zalloc_*() node aware, but would like to get
> real performance justification. My assumption is that rdmavt use kzalloc_node
> for the control plane based on some internal performance testing and we finally
> can see the difference between kzalloc and kzalloc_node, am I right?
> 
> Is the claim of performance degradation backed by data?

Yes, in the past. I don't have access anymore now that I'm not with 
Intel. It probably would not have been publishable anyway.

> The main reason (maybe I'm wrong here) is to avoid _node() allocators
> because they increase chances of memory allocation failure due to not
> doing fallback in case node memory is depleted.

Agreed. It's a trade-off that was deemed acceptable.

> Again, I'm suggesting to do plain kzalloc() for control part of QP.

Now I don't recall data for that specifically, but to be on the safe 
side I would not want to risk a performance regression.

-Denny

  reply	other threads:[~2021-05-12 12:45 UTC|newest]

Thread overview: 38+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-05-11 10:36 [PATCH rdma-next] RDMA/rdmavt: Decouple QP and SGE lists allocations Leon Romanovsky
2021-05-11 10:59 ` Haakon Bugge
2021-05-11 12:34   ` Leon Romanovsky
2021-05-11 19:15     ` Marciniszyn, Mike
2021-05-11 19:27       ` Leon Romanovsky
2021-05-11 19:39         ` Marciniszyn, Mike
2021-05-12  4:08         ` Dennis Dalessandro
2021-05-12 12:13           ` Leon Romanovsky
2021-05-12 12:45             ` Dennis Dalessandro [this message]
2021-05-11 12:26 ` Dennis Dalessandro
2021-05-11 12:34   ` Leon Romanovsky
2021-05-12 12:25     ` Marciniszyn, Mike
2021-05-12 12:50       ` Leon Romanovsky
2021-05-13 19:03         ` Dennis Dalessandro
2021-05-13 19:15           ` Jason Gunthorpe
2021-05-13 19:31             ` Dennis Dalessandro
2021-05-14 13:02               ` Jason Gunthorpe
2021-05-14 14:07                 ` Dennis Dalessandro
2021-05-14 14:35                   ` Jason Gunthorpe
2021-05-14 15:00                     ` Marciniszyn, Mike
2021-05-14 15:02                       ` Jason Gunthorpe
2021-05-19  7:50                         ` Leon Romanovsky
2021-05-19 11:56                           ` Dennis Dalessandro
2021-05-19 18:29                             ` Jason Gunthorpe
2021-05-19 19:49                               ` Dennis Dalessandro
2021-05-19 20:26                                 ` Jason Gunthorpe
2021-05-20 22:02                                   ` Dennis Dalessandro
2021-05-21  6:29                                     ` Leon Romanovsky
2021-05-25 13:13                                     ` Jason Gunthorpe
2021-05-25 14:10                                       ` Dennis Dalessandro
2021-05-25 14:20                                         ` Jason Gunthorpe
2021-05-25 14:29                                           ` Dennis Dalessandro
2021-06-28 21:59                                           ` Marciniszyn, Mike
2021-06-28 23:19                                             ` Jason Gunthorpe
2021-07-04  6:34                                               ` Leon Romanovsky
2021-06-02  4:33                                         ` Leon Romanovsky
2021-05-16 10:56           ` Leon Romanovsky
2021-05-12 12:23 ` Marciniszyn, Mike

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=7284da8c-9993-76c4-b495-32c814607a4b@cornelisnetworks.com \
    --to=dennis.dalessandro@cornelisnetworks.com \
    --cc=dledford@redhat.com \
    --cc=haakon.bugge@oracle.com \
    --cc=jgg@nvidia.com \
    --cc=leon@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-rdma@vger.kernel.org \
    --cc=mike.marciniszyn@cornelisnetworks.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.