From: Leon Romanovsky <leon@kernel.org>
To: Haakon Bugge <haakon.bugge@oracle.com>
Cc: Doug Ledford <dledford@redhat.com>,
Jason Gunthorpe <jgg@nvidia.com>,
Dennis Dalessandro <dennis.dalessandro@cornelisnetworks.com>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
OFED mailing list <linux-rdma@vger.kernel.org>,
Mike Marciniszyn <mike.marciniszyn@cornelisnetworks.com>
Subject: Re: [PATCH rdma-next] RDMA/rdmavt: Decouple QP and SGE lists allocations
Date: Tue, 11 May 2021 15:34:26 +0300 [thread overview]
Message-ID: <YJp50nw6JD3ptVDp@unreal> (raw)
In-Reply-To: <F62CF3D3-E605-4CBA-B171-5BB98594C658@oracle.com>
On Tue, May 11, 2021 at 10:59:52AM +0000, Haakon Bugge wrote:
>
>
> > On 11 May 2021, at 12:36, Leon Romanovsky <leon@kernel.org> wrote:
> >
> > From: Leon Romanovsky <leonro@nvidia.com>
> >
> > The rdmavt QP has fields that are both needed for the control and data
> > path. Such mixed declaration caused to the very specific allocation flow
> > with kzalloc_node and SGE list embedded into the struct rvt_qp.
> >
> > This patch separates QP creation to two: regular memory allocation for
> > the control path and specific code for the SGE list, while the access to
> > the later is performed through derefenced pointer.
> >
> > Such pointer and its context are expected to be in the cache, so
> > performance difference is expected to be negligible, if any exists.
> >
> > Signed-off-by: Leon Romanovsky <leonro@nvidia.com>
> > ---
> > Hi,
> >
> > This change is needed to convert QP to core allocation scheme. In that
> > scheme QP is allocated outside of the driver and size of such allocation
> > is constant and can be calculated at the compile time.
> >
> > Thanks
> > ---
> > drivers/infiniband/sw/rdmavt/qp.c | 13 ++++++++-----
> > include/rdma/rdmavt_qp.h | 2 +-
> > 2 files changed, 9 insertions(+), 6 deletions(-)
> >
> > diff --git a/drivers/infiniband/sw/rdmavt/qp.c b/drivers/infiniband/sw/rdmavt/qp.c
> > index 9d13db68283c..4522071fc220 100644
> > --- a/drivers/infiniband/sw/rdmavt/qp.c
> > +++ b/drivers/infiniband/sw/rdmavt/qp.c
> > @@ -1077,7 +1077,7 @@ struct ib_qp *rvt_create_qp(struct ib_pd *ibpd,
> > int err;
> > struct rvt_swqe *swq = NULL;
> > size_t sz;
> > - size_t sg_list_sz;
> > + size_t sg_list_sz = 0;
> > struct ib_qp *ret = ERR_PTR(-ENOMEM);
> > struct rvt_dev_info *rdi = ib_to_rvt(ibpd->device);
> > void *priv = NULL;
> > @@ -1125,8 +1125,6 @@ struct ib_qp *rvt_create_qp(struct ib_pd *ibpd,
> > if (!swq)
> > return ERR_PTR(-ENOMEM);
> >
> > - sz = sizeof(*qp);
> > - sg_list_sz = 0;
> > if (init_attr->srq) {
> > struct rvt_srq *srq = ibsrq_to_rvtsrq(init_attr->srq);
> >
> > @@ -1136,10 +1134,13 @@ struct ib_qp *rvt_create_qp(struct ib_pd *ibpd,
> > } else if (init_attr->cap.max_recv_sge > 1)
> > sg_list_sz = sizeof(*qp->r_sg_list) *
> > (init_attr->cap.max_recv_sge - 1);
> > - qp = kzalloc_node(sz + sg_list_sz, GFP_KERNEL,
> > - rdi->dparms.node);
> > + qp = kzalloc(sizeof(*qp), GFP_KERNEL);
>
> Why not kzalloc_node() here?
The idea is to delete this kzalloc later in next patch, because all
drivers are doing same thing "qp = kzalloc(sizeof(*qp), GFP_KERNEL);".
Thanks
next prev parent reply other threads:[~2021-05-11 12:34 UTC|newest]
Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-05-11 10:36 [PATCH rdma-next] RDMA/rdmavt: Decouple QP and SGE lists allocations Leon Romanovsky
2021-05-11 10:59 ` Haakon Bugge
2021-05-11 12:34 ` Leon Romanovsky [this message]
2021-05-11 19:15 ` Marciniszyn, Mike
2021-05-11 19:27 ` Leon Romanovsky
2021-05-11 19:39 ` Marciniszyn, Mike
2021-05-12 4:08 ` Dennis Dalessandro
2021-05-12 12:13 ` Leon Romanovsky
2021-05-12 12:45 ` Dennis Dalessandro
2021-05-11 12:26 ` Dennis Dalessandro
2021-05-11 12:34 ` Leon Romanovsky
2021-05-12 12:25 ` Marciniszyn, Mike
2021-05-12 12:50 ` Leon Romanovsky
2021-05-13 19:03 ` Dennis Dalessandro
2021-05-13 19:15 ` Jason Gunthorpe
2021-05-13 19:31 ` Dennis Dalessandro
2021-05-14 13:02 ` Jason Gunthorpe
2021-05-14 14:07 ` Dennis Dalessandro
2021-05-14 14:35 ` Jason Gunthorpe
2021-05-14 15:00 ` Marciniszyn, Mike
2021-05-14 15:02 ` Jason Gunthorpe
2021-05-19 7:50 ` Leon Romanovsky
2021-05-19 11:56 ` Dennis Dalessandro
2021-05-19 18:29 ` Jason Gunthorpe
2021-05-19 19:49 ` Dennis Dalessandro
2021-05-19 20:26 ` Jason Gunthorpe
2021-05-20 22:02 ` Dennis Dalessandro
2021-05-21 6:29 ` Leon Romanovsky
2021-05-25 13:13 ` Jason Gunthorpe
2021-05-25 14:10 ` Dennis Dalessandro
2021-05-25 14:20 ` Jason Gunthorpe
2021-05-25 14:29 ` Dennis Dalessandro
2021-06-28 21:59 ` Marciniszyn, Mike
2021-06-28 23:19 ` Jason Gunthorpe
2021-07-04 6:34 ` Leon Romanovsky
2021-06-02 4:33 ` Leon Romanovsky
2021-05-16 10:56 ` Leon Romanovsky
2021-05-12 12:23 ` Marciniszyn, Mike
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=YJp50nw6JD3ptVDp@unreal \
--to=leon@kernel.org \
--cc=dennis.dalessandro@cornelisnetworks.com \
--cc=dledford@redhat.com \
--cc=haakon.bugge@oracle.com \
--cc=jgg@nvidia.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=mike.marciniszyn@cornelisnetworks.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).