* [PATCH mlx5-next 1/2] net/mlx5: Enable QP number request when creating IPoIB underlay QP
2020-06-23 11:01 [PATCH rdma-next 0/2] Create IPoIB QP with specific QP number Leon Romanovsky
@ 2020-06-23 11:01 ` Leon Romanovsky
2020-06-23 11:01 ` [PATCH rdma-next 2/2] RDMA/ipoib: Handle user-supplied address when creating child Leon Romanovsky
` (2 subsequent siblings)
3 siblings, 0 replies; 8+ messages in thread
From: Leon Romanovsky @ 2020-06-23 11:01 UTC (permalink / raw)
To: Doug Ledford, Jason Gunthorpe
Cc: Michael Guralnik, Jakub Kicinski, linux-rdma, netdev, Saeed Mahameed
From: Michael Guralnik <michaelgur@mellanox.com>
If in the process of creating the underlay QP for an IPoIB interface
the user has set the address and specifically the 1st-3rd bytes
representing the QP number, use the requested QP number when creating
the underlay QP.
For a user to be able to request a QP number on QP creation, the MKEY_BY_NAME
NVCONFIG should be set. As mkey_by_name and qp_by_name are coupled in FW.
This requires driver to query the mkey_by_name max cap during initialization
and set the current cap if it was enabled in FW.
Signed-off-by: Michael Guralnik <michaelgur@mellanox.com>
Reviewed-by: Saeed Mahameed <saeedm@mellanox.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
---
drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c | 7 +++++++
drivers/net/ethernet/mellanox/mlx5/core/main.c | 3 +++
include/linux/mlx5/mlx5_ifc.h | 9 +++++++--
3 files changed, 17 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c b/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c
index 690b822c6152..d1266d8fed97 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c
@@ -226,13 +226,20 @@ void mlx5i_uninit_underlay_qp(struct mlx5e_priv *priv)
int mlx5i_create_underlay_qp(struct mlx5e_priv *priv)
{
+ unsigned char *dev_addr = priv->netdev->dev_addr;
u32 out[MLX5_ST_SZ_DW(create_qp_out)] = {};
u32 in[MLX5_ST_SZ_DW(create_qp_in)] = {};
struct mlx5i_priv *ipriv = priv->ppriv;
void *addr_path;
+ int qpn = 0;
int ret = 0;
void *qpc;
+ if (MLX5_CAP_GEN(priv->mdev, mkey_by_name)) {
+ qpn = (dev_addr[1] << 16) + (dev_addr[2] << 8) + dev_addr[3];
+ MLX5_SET(create_qp_in, in, input_qpn, qpn);
+ }
+
qpc = MLX5_ADDR_OF(create_qp_in, in, qpc);
MLX5_SET(qpc, qpc, st, MLX5_QP_ST_UD);
MLX5_SET(qpc, qpc, pm_state, MLX5_QP_PM_MIGRATED);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c
index 8b658908f044..623785fe74b2 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c
@@ -557,6 +557,9 @@ static int handle_hca_cap(struct mlx5_core_dev *dev, void *set_ctx)
if (MLX5_CAP_GEN_MAX(dev, release_all_pages))
MLX5_SET(cmd_hca_cap, set_hca_cap, release_all_pages, 1);
+ if (MLX5_CAP_GEN_MAX(dev, mkey_by_name))
+ MLX5_SET(cmd_hca_cap, set_hca_cap, mkey_by_name, 1);
+
return set_caps(dev, set_ctx, MLX5_SET_HCA_CAP_OP_MOD_GENERAL_DEVICE);
}
diff --git a/include/linux/mlx5/mlx5_ifc.h b/include/linux/mlx5/mlx5_ifc.h
index ca1887dd0423..7a00110f2f01 100644
--- a/include/linux/mlx5/mlx5_ifc.h
+++ b/include/linux/mlx5/mlx5_ifc.h
@@ -1392,7 +1392,10 @@ struct mlx5_ifc_cmd_hca_cap_bits {
u8 bf[0x1];
u8 driver_version[0x1];
u8 pad_tx_eth_packet[0x1];
- u8 reserved_at_263[0x8];
+ u8 reserved_at_263[0x3];
+ u8 mkey_by_name[0x1];
+ u8 reserved_at_267[0x4];
+
u8 log_bf_reg_size[0x5];
u8 reserved_at_270[0x8];
@@ -7714,8 +7717,10 @@ struct mlx5_ifc_create_qp_in_bits {
u8 reserved_at_20[0x10];
u8 op_mod[0x10];
- u8 reserved_at_40[0x40];
+ u8 reserved_at_40[0x8];
+ u8 input_qpn[0x18];
+ u8 reserved_at_60[0x20];
u8 opt_param_mask[0x20];
u8 ece[0x20];
--
2.26.2
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH rdma-next 2/2] RDMA/ipoib: Handle user-supplied address when creating child
2020-06-23 11:01 [PATCH rdma-next 0/2] Create IPoIB QP with specific QP number Leon Romanovsky
2020-06-23 11:01 ` [PATCH mlx5-next 1/2] net/mlx5: Enable QP number request when creating IPoIB underlay QP Leon Romanovsky
@ 2020-06-23 11:01 ` Leon Romanovsky
2020-07-02 17:55 ` [PATCH rdma-next 0/2] Create IPoIB QP with specific QP number Jason Gunthorpe
2020-07-06 17:53 ` Jason Gunthorpe
3 siblings, 0 replies; 8+ messages in thread
From: Leon Romanovsky @ 2020-06-23 11:01 UTC (permalink / raw)
To: Doug Ledford, Jason Gunthorpe; +Cc: Michael Guralnik, Feras Daoud, linux-rdma
From: Michael Guralnik <michaelgur@mellanox.com>
Use the address supplied by user when creating a child interface.
Previously, the address requested by the user was ignored and overridden
with parent's GID and the random QP number assigned to the child.
Signed-off-by: Michael Guralnik <michaelgur@mellanox.com>
Reviewed-by: Feras Daoud <ferasda@mellanox.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
---
drivers/infiniband/ulp/ipoib/ipoib_main.c | 11 +++++++++--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c
index 3cfb682b91b0..a9f1174f7320 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_main.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c
@@ -1892,8 +1892,15 @@ static void ipoib_child_init(struct net_device *ndev)
priv->max_ib_mtu = ppriv->max_ib_mtu;
set_bit(IPOIB_FLAG_SUBINTERFACE, &priv->flags);
- memcpy(priv->dev->dev_addr, ppriv->dev->dev_addr, INFINIBAND_ALEN);
- memcpy(&priv->local_gid, &ppriv->local_gid, sizeof(priv->local_gid));
+ if (memchr_inv(priv->dev->dev_addr, 0, INFINIBAND_ALEN))
+ memcpy(&priv->local_gid, priv->dev->dev_addr + 4,
+ sizeof(priv->local_gid));
+ else {
+ memcpy(priv->dev->dev_addr, ppriv->dev->dev_addr,
+ INFINIBAND_ALEN);
+ memcpy(&priv->local_gid, &ppriv->local_gid,
+ sizeof(priv->local_gid));
+ }
}
static int ipoib_ndo_init(struct net_device *ndev)
--
2.26.2
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH rdma-next 0/2] Create IPoIB QP with specific QP number
2020-06-23 11:01 [PATCH rdma-next 0/2] Create IPoIB QP with specific QP number Leon Romanovsky
2020-06-23 11:01 ` [PATCH mlx5-next 1/2] net/mlx5: Enable QP number request when creating IPoIB underlay QP Leon Romanovsky
2020-06-23 11:01 ` [PATCH rdma-next 2/2] RDMA/ipoib: Handle user-supplied address when creating child Leon Romanovsky
@ 2020-07-02 17:55 ` Jason Gunthorpe
2020-07-03 6:28 ` Leon Romanovsky
2020-07-03 15:40 ` Leon Romanovsky
2020-07-06 17:53 ` Jason Gunthorpe
3 siblings, 2 replies; 8+ messages in thread
From: Jason Gunthorpe @ 2020-07-02 17:55 UTC (permalink / raw)
To: Leon Romanovsky
Cc: Doug Ledford, Leon Romanovsky, Feras Daoud, Jakub Kicinski,
linux-rdma, Michael Guralnik, netdev, Saeed Mahameed
On Tue, Jun 23, 2020 at 02:01:03PM +0300, Leon Romanovsky wrote:
> From: Leon Romanovsky <leonro@mellanox.com>
>
> >From Michael,
>
> This series handles IPoIB child interface creation with setting
> interface's HW address.
>
> In current implementation, lladdr requested by user is ignored and
> overwritten. Child interface gets the same GID as the parent interface
> and a QP number which is assigned by the underlying drivers.
>
> In this series we fix this behavior so that user's requested address is
> assigned to the newly created interface.
>
> As specific QP number request is not supported for all vendors, QP
> number requested by user will still be overwritten when this is not
> supported.
>
> Behavior of creation of child interfaces through the sysfs mechanism or
> without specifying a requested address, stays the same.
>
> Thanks
>
> Michael Guralnik (2):
> net/mlx5: Enable QP number request when creating IPoIB underlay QP
> RDMA/ipoib: Handle user-supplied address when creating child
Applied to for-next, thanks
Jason
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH rdma-next 0/2] Create IPoIB QP with specific QP number
2020-07-02 17:55 ` [PATCH rdma-next 0/2] Create IPoIB QP with specific QP number Jason Gunthorpe
@ 2020-07-03 6:28 ` Leon Romanovsky
2020-07-03 11:40 ` Jason Gunthorpe
2020-07-03 15:40 ` Leon Romanovsky
1 sibling, 1 reply; 8+ messages in thread
From: Leon Romanovsky @ 2020-07-03 6:28 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Doug Ledford, Feras Daoud, Jakub Kicinski, linux-rdma,
Michael Guralnik, netdev, Saeed Mahameed
On Thu, Jul 02, 2020 at 02:55:41PM -0300, Jason Gunthorpe wrote:
> On Tue, Jun 23, 2020 at 02:01:03PM +0300, Leon Romanovsky wrote:
> > From: Leon Romanovsky <leonro@mellanox.com>
> >
> > >From Michael,
> >
> > This series handles IPoIB child interface creation with setting
> > interface's HW address.
> >
> > In current implementation, lladdr requested by user is ignored and
> > overwritten. Child interface gets the same GID as the parent interface
> > and a QP number which is assigned by the underlying drivers.
> >
> > In this series we fix this behavior so that user's requested address is
> > assigned to the newly created interface.
> >
> > As specific QP number request is not supported for all vendors, QP
> > number requested by user will still be overwritten when this is not
> > supported.
> >
> > Behavior of creation of child interfaces through the sysfs mechanism or
> > without specifying a requested address, stays the same.
> >
> > Thanks
> >
> > Michael Guralnik (2):
> > net/mlx5: Enable QP number request when creating IPoIB underlay QP
> > RDMA/ipoib: Handle user-supplied address when creating child
>
> Applied to for-next, thanks
Thanks Jason,
Won't it better that first patch be applied to mlx5-next in order to
avoid possible merge conflicts?
Thanks
>
> Jason
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH rdma-next 0/2] Create IPoIB QP with specific QP number
2020-07-03 6:28 ` Leon Romanovsky
@ 2020-07-03 11:40 ` Jason Gunthorpe
0 siblings, 0 replies; 8+ messages in thread
From: Jason Gunthorpe @ 2020-07-03 11:40 UTC (permalink / raw)
To: Leon Romanovsky
Cc: Doug Ledford, Feras Daoud, Jakub Kicinski, linux-rdma,
Michael Guralnik, netdev, Saeed Mahameed
On Fri, Jul 03, 2020 at 09:28:09AM +0300, Leon Romanovsky wrote:
> On Thu, Jul 02, 2020 at 02:55:41PM -0300, Jason Gunthorpe wrote:
> > On Tue, Jun 23, 2020 at 02:01:03PM +0300, Leon Romanovsky wrote:
> > > From: Leon Romanovsky <leonro@mellanox.com>
> > >
> > > >From Michael,
> > >
> > > This series handles IPoIB child interface creation with setting
> > > interface's HW address.
> > >
> > > In current implementation, lladdr requested by user is ignored and
> > > overwritten. Child interface gets the same GID as the parent interface
> > > and a QP number which is assigned by the underlying drivers.
> > >
> > > In this series we fix this behavior so that user's requested address is
> > > assigned to the newly created interface.
> > >
> > > As specific QP number request is not supported for all vendors, QP
> > > number requested by user will still be overwritten when this is not
> > > supported.
> > >
> > > Behavior of creation of child interfaces through the sysfs mechanism or
> > > without specifying a requested address, stays the same.
> > >
> > > Thanks
> > >
> > > Michael Guralnik (2):
> > > net/mlx5: Enable QP number request when creating IPoIB underlay QP
> > > RDMA/ipoib: Handle user-supplied address when creating child
> >
> > Applied to for-next, thanks
>
> Thanks Jason,
>
> Won't it better that first patch be applied to mlx5-next in order to
> avoid possible merge conflicts?
Oops, sure, go ahead please
Jason
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH rdma-next 0/2] Create IPoIB QP with specific QP number
2020-07-02 17:55 ` [PATCH rdma-next 0/2] Create IPoIB QP with specific QP number Jason Gunthorpe
2020-07-03 6:28 ` Leon Romanovsky
@ 2020-07-03 15:40 ` Leon Romanovsky
1 sibling, 0 replies; 8+ messages in thread
From: Leon Romanovsky @ 2020-07-03 15:40 UTC (permalink / raw)
To: Jason Gunthorpe
Cc: Doug Ledford, Feras Daoud, Jakub Kicinski, linux-rdma,
Michael Guralnik, netdev, Saeed Mahameed
On Thu, Jul 02, 2020 at 02:55:41PM -0300, Jason Gunthorpe wrote:
> On Tue, Jun 23, 2020 at 02:01:03PM +0300, Leon Romanovsky wrote:
> > From: Leon Romanovsky <leonro@mellanox.com>
> >
> > >From Michael,
> >
> > This series handles IPoIB child interface creation with setting
> > interface's HW address.
> >
> > In current implementation, lladdr requested by user is ignored and
> > overwritten. Child interface gets the same GID as the parent interface
> > and a QP number which is assigned by the underlying drivers.
> >
> > In this series we fix this behavior so that user's requested address is
> > assigned to the newly created interface.
> >
> > As specific QP number request is not supported for all vendors, QP
> > number requested by user will still be overwritten when this is not
> > supported.
> >
> > Behavior of creation of child interfaces through the sysfs mechanism or
> > without specifying a requested address, stays the same.
> >
> > Thanks
> >
> > Michael Guralnik (2):
> > net/mlx5: Enable QP number request when creating IPoIB underlay QP
> > RDMA/ipoib: Handle user-supplied address when creating child
>
> Applied to for-next, thanks
I pushed first patch.
dca650991e4 net/mlx5: Enable QP number request when creating IPoIB underlay QP
>
> Jason
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH rdma-next 0/2] Create IPoIB QP with specific QP number
2020-06-23 11:01 [PATCH rdma-next 0/2] Create IPoIB QP with specific QP number Leon Romanovsky
` (2 preceding siblings ...)
2020-07-02 17:55 ` [PATCH rdma-next 0/2] Create IPoIB QP with specific QP number Jason Gunthorpe
@ 2020-07-06 17:53 ` Jason Gunthorpe
3 siblings, 0 replies; 8+ messages in thread
From: Jason Gunthorpe @ 2020-07-06 17:53 UTC (permalink / raw)
To: Leon Romanovsky
Cc: Doug Ledford, Leon Romanovsky, Feras Daoud, Jakub Kicinski,
linux-rdma, Michael Guralnik, netdev, Saeed Mahameed
On Tue, Jun 23, 2020 at 02:01:03PM +0300, Leon Romanovsky wrote:
> From: Leon Romanovsky <leonro@mellanox.com>
>
> >From Michael,
>
> This series handles IPoIB child interface creation with setting
> interface's HW address.
>
> In current implementation, lladdr requested by user is ignored and
> overwritten. Child interface gets the same GID as the parent interface
> and a QP number which is assigned by the underlying drivers.
>
> In this series we fix this behavior so that user's requested address is
> assigned to the newly created interface.
>
> As specific QP number request is not supported for all vendors, QP
> number requested by user will still be overwritten when this is not
> supported.
>
> Behavior of creation of child interfaces through the sysfs mechanism or
> without specifying a requested address, stays the same.
>
> Thanks
>
> Michael Guralnik (2):
> net/mlx5: Enable QP number request when creating IPoIB underlay QP
> RDMA/ipoib: Handle user-supplied address when creating child
>
> drivers/infiniband/ulp/ipoib/ipoib_main.c | 11 +++++++++--
> drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c | 7 +++++++
> drivers/net/ethernet/mellanox/mlx5/core/main.c | 3 +++
> include/linux/mlx5/mlx5_ifc.h | 9 +++++++--
> 4 files changed, 26 insertions(+), 4 deletions(-)
Applied to for-next, thanks
Jason
^ permalink raw reply [flat|nested] 8+ messages in thread