* [rdma-rc PATCH 1/2] cxgb4: export sge_host_page_size to ulds
2019-02-14 12:10 [rdma-rc PATCH 0/2] iw_cxgb4: Adjust the cq/qp mask Raju Rangoju
@ 2019-02-14 12:10 ` Raju Rangoju
2019-02-14 15:51 ` Leon Romanovsky
2019-02-14 12:10 ` [rdma-rc PATCH 2/2] iw_cxgb4: cq/qp mask depends on bar2 pages in a host page Raju Rangoju
2019-02-15 22:47 ` [rdma-rc PATCH 0/2] iw_cxgb4: Adjust the cq/qp mask Jason Gunthorpe
2 siblings, 1 reply; 8+ messages in thread
From: Raju Rangoju @ 2019-02-14 12:10 UTC (permalink / raw)
To: jgg, davem, linux-rdma; +Cc: netdev, swise, rajur
Export the sge_host_page_size field to ULDs via
cxgb4_lld_info, so that iw_cxgb4 can make use of
this in calculating the correct qp/cq mask.
Fixes: 2391b0030e ("cxgb4: Remove SGE_HOST_PAGE_SIZE dependency on page
size")
Signed-off-by: Raju Rangoju <rajur@chelsio.com>
---
drivers/net/ethernet/chelsio/cxgb4/cxgb4_uld.c | 1 +
drivers/net/ethernet/chelsio/cxgb4/cxgb4_uld.h | 1 +
2 files changed, 2 insertions(+)
diff --git a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_uld.c b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_uld.c
index c041f44324db..b3654598a2d5 100644
--- a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_uld.c
+++ b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_uld.c
@@ -660,6 +660,7 @@ static void uld_init(struct adapter *adap, struct cxgb4_lld_info *lld)
lld->cclk_ps = 1000000000 / adap->params.vpd.cclk;
lld->udb_density = 1 << adap->params.sge.eq_qpp;
lld->ucq_density = 1 << adap->params.sge.iq_qpp;
+ lld->sge_host_page_size = 1 << (adap->params.sge.hps + 10);
lld->filt_mode = adap->params.tp.vlan_pri_map;
/* MODQ_REQ_MAP sets queues 0-3 to chan 0-3 */
for (i = 0; i < NCHAN; i++)
diff --git a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_uld.h b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_uld.h
index 5fa9a2d5fc4b..21da34a4ca24 100644
--- a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_uld.h
+++ b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_uld.h
@@ -336,6 +336,7 @@ struct cxgb4_lld_info {
unsigned int cclk_ps; /* Core clock period in psec */
unsigned short udb_density; /* # of user DB/page */
unsigned short ucq_density; /* # of user CQs/page */
+ unsigned int sge_host_page_size; /* SGE host page size */
unsigned short filt_mode; /* filter optional components */
unsigned short tx_modq[NCHAN]; /* maps each tx channel to a */
/* scheduler queue */
--
2.13.0
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [rdma-rc PATCH 1/2] cxgb4: export sge_host_page_size to ulds
2019-02-14 12:10 ` [rdma-rc PATCH 1/2] cxgb4: export sge_host_page_size to ulds Raju Rangoju
@ 2019-02-14 15:51 ` Leon Romanovsky
0 siblings, 0 replies; 8+ messages in thread
From: Leon Romanovsky @ 2019-02-14 15:51 UTC (permalink / raw)
To: Raju Rangoju; +Cc: jgg, davem, linux-rdma, netdev, swise
[-- Attachment #1: Type: text/plain, Size: 334 bytes --]
On Thu, Feb 14, 2019 at 05:40:53PM +0530, Raju Rangoju wrote:
> Export the sge_host_page_size field to ULDs via
> cxgb4_lld_info, so that iw_cxgb4 can make use of
> this in calculating the correct qp/cq mask.
>
> Fixes: 2391b0030e ("cxgb4: Remove SGE_HOST_PAGE_SIZE dependency on page
> size")
Please don't break Fixes line.
Thanks
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 801 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread
* [rdma-rc PATCH 2/2] iw_cxgb4: cq/qp mask depends on bar2 pages in a host page
2019-02-14 12:10 [rdma-rc PATCH 0/2] iw_cxgb4: Adjust the cq/qp mask Raju Rangoju
2019-02-14 12:10 ` [rdma-rc PATCH 1/2] cxgb4: export sge_host_page_size to ulds Raju Rangoju
@ 2019-02-14 12:10 ` Raju Rangoju
2019-02-14 15:41 ` Jason Gunthorpe
2019-02-15 22:47 ` [rdma-rc PATCH 0/2] iw_cxgb4: Adjust the cq/qp mask Jason Gunthorpe
2 siblings, 1 reply; 8+ messages in thread
From: Raju Rangoju @ 2019-02-14 12:10 UTC (permalink / raw)
To: jgg, davem, linux-rdma; +Cc: netdev, swise, rajur
Adjust the cq/qp mask based on no.of bar2 pages in a host page.
For user-mode rdma, the granularity of the BAR2 memory mapped
to a user rdma process during queue allocation must be based
on the host page size. The lld attributes udb_density and
ucq_density are used to figure out how many sge contexts are
in a bar2 page. So the rdev->qpmask and rdev->cqmask in
iw_cxgb4 need to now be adjusted based on how many sge bar2
pages are in a host page.
Fixes: 2391b0030e ("cxgb4: Remove SGE_HOST_PAGE_SIZE dependency on page size")
Signed-off-by: Raju Rangoju <rajur@chelsio.com>
---
drivers/infiniband/hw/cxgb4/device.c | 15 +++++++++++++--
1 file changed, 13 insertions(+), 2 deletions(-)
diff --git a/drivers/infiniband/hw/cxgb4/device.c b/drivers/infiniband/hw/cxgb4/device.c
index c13c0ba30f63..d499cd61c0e8 100644
--- a/drivers/infiniband/hw/cxgb4/device.c
+++ b/drivers/infiniband/hw/cxgb4/device.c
@@ -783,6 +783,7 @@ void c4iw_init_dev_ucontext(struct c4iw_rdev *rdev,
static int c4iw_rdev_open(struct c4iw_rdev *rdev)
{
int err;
+ unsigned int factor;
c4iw_init_dev_ucontext(rdev, &rdev->uctx);
@@ -806,8 +807,18 @@ static int c4iw_rdev_open(struct c4iw_rdev *rdev)
return -EINVAL;
}
- rdev->qpmask = rdev->lldi.udb_density - 1;
- rdev->cqmask = rdev->lldi.ucq_density - 1;
+ /* This implementation requires a sge_host_page_size <= PAGE_SIZE. */
+ if (rdev->lldi.sge_host_page_size > PAGE_SIZE) {
+ pr_err("%s: unsupported sge host page size %u\n",
+ pci_name(rdev->lldi.pdev),
+ rdev->lldi.sge_host_page_size);
+ return -EINVAL;
+ }
+
+ factor = PAGE_SIZE / rdev->lldi.sge_host_page_size;
+ rdev->qpmask = (rdev->lldi.udb_density * factor) - 1;
+ rdev->cqmask = (rdev->lldi.ucq_density * factor) - 1;
+
pr_debug("dev %s stag start 0x%0x size 0x%0x num stags %d pbl start 0x%0x size 0x%0x rq start 0x%0x size 0x%0x qp qid start %u size %u cq qid start %u size %u srq size %u\n",
pci_name(rdev->lldi.pdev), rdev->lldi.vr->stag.start,
rdev->lldi.vr->stag.size, c4iw_num_stags(rdev),
--
2.13.0
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [rdma-rc PATCH 2/2] iw_cxgb4: cq/qp mask depends on bar2 pages in a host page
2019-02-14 12:10 ` [rdma-rc PATCH 2/2] iw_cxgb4: cq/qp mask depends on bar2 pages in a host page Raju Rangoju
@ 2019-02-14 15:41 ` Jason Gunthorpe
2019-02-14 17:28 ` Raju Rangoju
0 siblings, 1 reply; 8+ messages in thread
From: Jason Gunthorpe @ 2019-02-14 15:41 UTC (permalink / raw)
To: Raju Rangoju; +Cc: davem, linux-rdma, netdev, swise
On Thu, Feb 14, 2019 at 05:40:54PM +0530, Raju Rangoju wrote:
> Adjust the cq/qp mask based on no.of bar2 pages in a host page.
>
> For user-mode rdma, the granularity of the BAR2 memory mapped
> to a user rdma process during queue allocation must be based
> on the host page size. The lld attributes udb_density and
> ucq_density are used to figure out how many sge contexts are
> in a bar2 page. So the rdev->qpmask and rdev->cqmask in
> iw_cxgb4 need to now be adjusted based on how many sge bar2
> pages are in a host page.
Why is this rc? Do certain arches fail to work or something?
Jason
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [rdma-rc PATCH 2/2] iw_cxgb4: cq/qp mask depends on bar2 pages in a host page
2019-02-14 15:41 ` Jason Gunthorpe
@ 2019-02-14 17:28 ` Raju Rangoju
2019-02-14 17:54 ` Steve Wise
0 siblings, 1 reply; 8+ messages in thread
From: Raju Rangoju @ 2019-02-14 17:28 UTC (permalink / raw)
To: Jason Gunthorpe; +Cc: davem, linux-rdma, netdev, swise
On Thursday, February 02/14/19, 2019 at 15:41:34 +0000, Jason Gunthorpe wrote:
> On Thu, Feb 14, 2019 at 05:40:54PM +0530, Raju Rangoju wrote:
> > Adjust the cq/qp mask based on no.of bar2 pages in a host page.
> >
> > For user-mode rdma, the granularity of the BAR2 memory mapped
> > to a user rdma process during queue allocation must be based
> > on the host page size. The lld attributes udb_density and
> > ucq_density are used to figure out how many sge contexts are
> > in a bar2 page. So the rdev->qpmask and rdev->cqmask in
> > iw_cxgb4 need to now be adjusted based on how many sge bar2
> > pages are in a host page.
>
> Why is this rc? Do certain arches fail to work or something?
>
Yes, this series fixes a regression that was introduced by commit
2391b0030e (v5.0-rc1~129^2~272)
> Jason
^ permalink raw reply [flat|nested] 8+ messages in thread
* RE: [rdma-rc PATCH 2/2] iw_cxgb4: cq/qp mask depends on bar2 pages in a host page
2019-02-14 17:28 ` Raju Rangoju
@ 2019-02-14 17:54 ` Steve Wise
0 siblings, 0 replies; 8+ messages in thread
From: Steve Wise @ 2019-02-14 17:54 UTC (permalink / raw)
To: 'Raju Rangoju', 'Jason Gunthorpe'
Cc: davem, linux-rdma, netdev
> -----Original Message-----
> From: linux-rdma-owner@vger.kernel.org <linux-rdma-
> owner@vger.kernel.org> On Behalf Of Raju Rangoju
> Sent: Thursday, February 14, 2019 11:28 AM
> To: Jason Gunthorpe <jgg@mellanox.com>
> Cc: davem@davemloft.net; linux-rdma@vger.kernel.org;
> netdev@vger.kernel.org; swise@opengridcomputing.com
> Subject: Re: [rdma-rc PATCH 2/2] iw_cxgb4: cq/qp mask depends on bar2
> pages in a host page
>
> On Thursday, February 02/14/19, 2019 at 15:41:34 +0000, Jason Gunthorpe
> wrote:
> > On Thu, Feb 14, 2019 at 05:40:54PM +0530, Raju Rangoju wrote:
> > > Adjust the cq/qp mask based on no.of bar2 pages in a host page.
> > >
> > > For user-mode rdma, the granularity of the BAR2 memory mapped
> > > to a user rdma process during queue allocation must be based
> > > on the host page size. The lld attributes udb_density and
> > > ucq_density are used to figure out how many sge contexts are
> > > in a bar2 page. So the rdev->qpmask and rdev->cqmask in
> > > iw_cxgb4 need to now be adjusted based on how many sge bar2
> > > pages are in a host page.
> >
> > Why is this rc? Do certain arches fail to work or something?
> >
>
> Yes, this series fixes a regression that was introduced by commit
> 2391b0030e (v5.0-rc1~129^2~272)
>
> > Jason
Rdma over cxgb4 on arches with a non-4K page size are busted w/o this fix.
That was the motivation for -rc.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [rdma-rc PATCH 0/2] iw_cxgb4: Adjust the cq/qp mask
2019-02-14 12:10 [rdma-rc PATCH 0/2] iw_cxgb4: Adjust the cq/qp mask Raju Rangoju
2019-02-14 12:10 ` [rdma-rc PATCH 1/2] cxgb4: export sge_host_page_size to ulds Raju Rangoju
2019-02-14 12:10 ` [rdma-rc PATCH 2/2] iw_cxgb4: cq/qp mask depends on bar2 pages in a host page Raju Rangoju
@ 2019-02-15 22:47 ` Jason Gunthorpe
2 siblings, 0 replies; 8+ messages in thread
From: Jason Gunthorpe @ 2019-02-15 22:47 UTC (permalink / raw)
To: Raju Rangoju; +Cc: davem, linux-rdma, netdev, swise
On Thu, Feb 14, 2019 at 05:40:52PM +0530, Raju Rangoju wrote:
> Export the LLD sge_host_page_size field to ULDs via
> cxgb4_lld_info, so that iw_cxgb4 can adjust the cq/qp
> mask based on no.of bar2 pages in a host page.
>
> This series has both net(cxgb4) and rdma(iw_cxgb4) changes,
> and I would request this merge via rdma repo.
>
> I have made sure this series applies cleanly on both net-next
> and rdma-for-rc and doesn't cause any merge conflicts.
>
> Raju Rangoju (2):
> cxgb4: export sge_host_page_size to ulds
> iw_cxgb4: cq/qp mask depends on bar2 pages in a host page
As requested, applied to for-rc with some revisions to the commit
messages.
Thanks,
Jason
^ permalink raw reply [flat|nested] 8+ messages in thread