[PATCH rdma-next 0/2] Create IPoIB QP with specific QP number

[PATCH rdma-next 0/2] Create IPoIB QP with specific QP number

[Leon Romanovsky]

From: Leon Romanovsky <leonro@mellanox.com>

From Michael,

This series handles IPoIB child interface creation with setting
interface's HW address.

In current implementation, lladdr requested by user is ignored and
overwritten. Child interface gets the same GID as the parent interface
and a QP number which is assigned by the underlying drivers.

In this series we fix this behavior so that user's requested address is
assigned to the newly created interface.

As specific QP number request is not supported for all vendors, QP
number requested by user will still be overwritten when this is not
supported.

Behavior of creation of child interfaces through the sysfs mechanism or
without specifying a requested address, stays the same.

Thanks

Michael Guralnik (2):
  net/mlx5: Enable QP number request when creating IPoIB underlay QP
  RDMA/ipoib: Handle user-supplied address when creating child

 drivers/infiniband/ulp/ipoib/ipoib_main.c             | 11 +++++++++--
 drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c |  7 +++++++
 drivers/net/ethernet/mellanox/mlx5/core/main.c        |  3 +++
 include/linux/mlx5/mlx5_ifc.h                         |  9 +++++++--
 4 files changed, 26 insertions(+), 4 deletions(-)

--
2.26.2

[PATCH mlx5-next 1/2] net/mlx5: Enable QP number request when creating IPoIB underlay QP

[Leon Romanovsky]

From: Michael Guralnik <michaelgur@mellanox.com>

If in the process of creating the underlay QP for an IPoIB interface
the user has set the address and specifically the 1st-3rd bytes
representing the QP number, use the requested QP number when creating
the underlay QP.

For a user to be able to request a QP number on QP creation, the MKEY_BY_NAME
NVCONFIG should be set. As mkey_by_name and qp_by_name are coupled in FW.
This requires driver to query the mkey_by_name max cap during initialization
and set the current cap if it was enabled in FW.

Signed-off-by: Michael Guralnik <michaelgur@mellanox.com>
Reviewed-by: Saeed Mahameed <saeedm@mellanox.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
---
 drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c | 7 +++++++
 drivers/net/ethernet/mellanox/mlx5/core/main.c        | 3 +++
 include/linux/mlx5/mlx5_ifc.h                         | 9 +++++++--
 3 files changed, 17 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c b/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c
index 690b822c6152..d1266d8fed97 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c
@@ -226,13 +226,20 @@ void mlx5i_uninit_underlay_qp(struct mlx5e_priv *priv)
 
 int mlx5i_create_underlay_qp(struct mlx5e_priv *priv)
 {
+	unsigned char *dev_addr = priv->netdev->dev_addr;
 	u32 out[MLX5_ST_SZ_DW(create_qp_out)] = {};
 	u32 in[MLX5_ST_SZ_DW(create_qp_in)] = {};
 	struct mlx5i_priv *ipriv = priv->ppriv;
 	void *addr_path;
+	int qpn = 0;
 	int ret = 0;
 	void *qpc;
 
+	if (MLX5_CAP_GEN(priv->mdev, mkey_by_name)) {
+		qpn = (dev_addr[1] << 16) + (dev_addr[2] << 8) + dev_addr[3];
+		MLX5_SET(create_qp_in, in, input_qpn, qpn);
+	}
+
 	qpc = MLX5_ADDR_OF(create_qp_in, in, qpc);
 	MLX5_SET(qpc, qpc, st, MLX5_QP_ST_UD);
 	MLX5_SET(qpc, qpc, pm_state, MLX5_QP_PM_MIGRATED);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c
index 8b658908f044..623785fe74b2 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c
@@ -557,6 +557,9 @@ static int handle_hca_cap(struct mlx5_core_dev *dev, void *set_ctx)
 	if (MLX5_CAP_GEN_MAX(dev, release_all_pages))
 		MLX5_SET(cmd_hca_cap, set_hca_cap, release_all_pages, 1);
 
+	if (MLX5_CAP_GEN_MAX(dev, mkey_by_name))
+		MLX5_SET(cmd_hca_cap, set_hca_cap, mkey_by_name, 1);
+
 	return set_caps(dev, set_ctx, MLX5_SET_HCA_CAP_OP_MOD_GENERAL_DEVICE);
 }
 
diff --git a/include/linux/mlx5/mlx5_ifc.h b/include/linux/mlx5/mlx5_ifc.h
index ca1887dd0423..7a00110f2f01 100644
--- a/include/linux/mlx5/mlx5_ifc.h
+++ b/include/linux/mlx5/mlx5_ifc.h
@@ -1392,7 +1392,10 @@ struct mlx5_ifc_cmd_hca_cap_bits {
 	u8         bf[0x1];
 	u8         driver_version[0x1];
 	u8         pad_tx_eth_packet[0x1];
-	u8         reserved_at_263[0x8];
+	u8         reserved_at_263[0x3];
+	u8         mkey_by_name[0x1];
+	u8         reserved_at_267[0x4];
+
 	u8         log_bf_reg_size[0x5];
 
 	u8         reserved_at_270[0x8];
@@ -7714,8 +7717,10 @@ struct mlx5_ifc_create_qp_in_bits {
 	u8         reserved_at_20[0x10];
 	u8         op_mod[0x10];
 
-	u8         reserved_at_40[0x40];
+	u8         reserved_at_40[0x8];
+	u8         input_qpn[0x18];
 
+	u8         reserved_at_60[0x20];
 	u8         opt_param_mask[0x20];
 
 	u8         ece[0x20];
-- 
2.26.2

[PATCH rdma-next 2/2] RDMA/ipoib: Handle user-supplied address when creating child

[Leon Romanovsky]

From: Michael Guralnik <michaelgur@mellanox.com>

Use the address supplied by user when creating a child interface.

Previously, the address requested by the user was ignored and overridden
with parent's GID and the random QP number assigned to the child.

Signed-off-by: Michael Guralnik <michaelgur@mellanox.com>
Reviewed-by: Feras Daoud <ferasda@mellanox.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
---
 drivers/infiniband/ulp/ipoib/ipoib_main.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c
index 3cfb682b91b0..a9f1174f7320 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_main.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c
@@ -1892,8 +1892,15 @@ static void ipoib_child_init(struct net_device *ndev)
 
 	priv->max_ib_mtu = ppriv->max_ib_mtu;
 	set_bit(IPOIB_FLAG_SUBINTERFACE, &priv->flags);
-	memcpy(priv->dev->dev_addr, ppriv->dev->dev_addr, INFINIBAND_ALEN);
-	memcpy(&priv->local_gid, &ppriv->local_gid, sizeof(priv->local_gid));
+	if (memchr_inv(priv->dev->dev_addr, 0, INFINIBAND_ALEN))
+		memcpy(&priv->local_gid, priv->dev->dev_addr + 4,
+		       sizeof(priv->local_gid));
+	else {
+		memcpy(priv->dev->dev_addr, ppriv->dev->dev_addr,
+		       INFINIBAND_ALEN);
+		memcpy(&priv->local_gid, &ppriv->local_gid,
+		       sizeof(priv->local_gid));
+	}
 }
 
 static int ipoib_ndo_init(struct net_device *ndev)
-- 
2.26.2

Re: [PATCH rdma-next 0/2] Create IPoIB QP with specific QP number

[Jason Gunthorpe]

On Tue, Jun 23, 2020 at 02:01:03PM +0300, Leon Romanovsky wrote:
> From: Leon Romanovsky <leonro@mellanox.com>
> 
> >From Michael,
> 
> This series handles IPoIB child interface creation with setting
> interface's HW address.
> 
> In current implementation, lladdr requested by user is ignored and
> overwritten. Child interface gets the same GID as the parent interface
> and a QP number which is assigned by the underlying drivers.
> 
> In this series we fix this behavior so that user's requested address is
> assigned to the newly created interface.
> 
> As specific QP number request is not supported for all vendors, QP
> number requested by user will still be overwritten when this is not
> supported.
> 
> Behavior of creation of child interfaces through the sysfs mechanism or
> without specifying a requested address, stays the same.
> 
> Thanks
> 
> Michael Guralnik (2):
>   net/mlx5: Enable QP number request when creating IPoIB underlay QP
>   RDMA/ipoib: Handle user-supplied address when creating child

Applied to for-next, thanks

Jason

Re: [PATCH rdma-next 0/2] Create IPoIB QP with specific QP number

[Leon Romanovsky]

On Thu, Jul 02, 2020 at 02:55:41PM -0300, Jason Gunthorpe wrote:
> On Tue, Jun 23, 2020 at 02:01:03PM +0300, Leon Romanovsky wrote:
> > From: Leon Romanovsky <leonro@mellanox.com>
> >
> > >From Michael,
> >
> > This series handles IPoIB child interface creation with setting
> > interface's HW address.
> >
> > In current implementation, lladdr requested by user is ignored and
> > overwritten. Child interface gets the same GID as the parent interface
> > and a QP number which is assigned by the underlying drivers.
> >
> > In this series we fix this behavior so that user's requested address is
> > assigned to the newly created interface.
> >
> > As specific QP number request is not supported for all vendors, QP
> > number requested by user will still be overwritten when this is not
> > supported.
> >
> > Behavior of creation of child interfaces through the sysfs mechanism or
> > without specifying a requested address, stays the same.
> >
> > Thanks
> >
> > Michael Guralnik (2):
> >   net/mlx5: Enable QP number request when creating IPoIB underlay QP
> >   RDMA/ipoib: Handle user-supplied address when creating child
>
> Applied to for-next, thanks

Thanks Jason,

Won't it better that first patch be applied to mlx5-next in order to
avoid possible merge conflicts?

Thanks

>
> Jason

Re: [PATCH rdma-next 0/2] Create IPoIB QP with specific QP number

[Jason Gunthorpe]

On Fri, Jul 03, 2020 at 09:28:09AM +0300, Leon Romanovsky wrote:
> On Thu, Jul 02, 2020 at 02:55:41PM -0300, Jason Gunthorpe wrote:
> > On Tue, Jun 23, 2020 at 02:01:03PM +0300, Leon Romanovsky wrote:
> > > From: Leon Romanovsky <leonro@mellanox.com>
> > >
> > > >From Michael,
> > >
> > > This series handles IPoIB child interface creation with setting
> > > interface's HW address.
> > >
> > > In current implementation, lladdr requested by user is ignored and
> > > overwritten. Child interface gets the same GID as the parent interface
> > > and a QP number which is assigned by the underlying drivers.
> > >
> > > In this series we fix this behavior so that user's requested address is
> > > assigned to the newly created interface.
> > >
> > > As specific QP number request is not supported for all vendors, QP
> > > number requested by user will still be overwritten when this is not
> > > supported.
> > >
> > > Behavior of creation of child interfaces through the sysfs mechanism or
> > > without specifying a requested address, stays the same.
> > >
> > > Thanks
> > >
> > > Michael Guralnik (2):
> > >   net/mlx5: Enable QP number request when creating IPoIB underlay QP
> > >   RDMA/ipoib: Handle user-supplied address when creating child
> >
> > Applied to for-next, thanks
> 
> Thanks Jason,
> 
> Won't it better that first patch be applied to mlx5-next in order to
> avoid possible merge conflicts?

Oops, sure, go ahead please

Jason

Re: [PATCH rdma-next 0/2] Create IPoIB QP with specific QP number

[Leon Romanovsky]

On Thu, Jul 02, 2020 at 02:55:41PM -0300, Jason Gunthorpe wrote:
> On Tue, Jun 23, 2020 at 02:01:03PM +0300, Leon Romanovsky wrote:
> > From: Leon Romanovsky <leonro@mellanox.com>
> >
> > >From Michael,
> >
> > This series handles IPoIB child interface creation with setting
> > interface's HW address.
> >
> > In current implementation, lladdr requested by user is ignored and
> > overwritten. Child interface gets the same GID as the parent interface
> > and a QP number which is assigned by the underlying drivers.
> >
> > In this series we fix this behavior so that user's requested address is
> > assigned to the newly created interface.
> >
> > As specific QP number request is not supported for all vendors, QP
> > number requested by user will still be overwritten when this is not
> > supported.
> >
> > Behavior of creation of child interfaces through the sysfs mechanism or
> > without specifying a requested address, stays the same.
> >
> > Thanks
> >
> > Michael Guralnik (2):
> >   net/mlx5: Enable QP number request when creating IPoIB underlay QP
> >   RDMA/ipoib: Handle user-supplied address when creating child
>
> Applied to for-next, thanks

I pushed first patch.
dca650991e4 net/mlx5: Enable QP number request when creating IPoIB underlay QP

>
> Jason

Re: [PATCH rdma-next 0/2] Create IPoIB QP with specific QP number

[Jason Gunthorpe]

On Tue, Jun 23, 2020 at 02:01:03PM +0300, Leon Romanovsky wrote:
> From: Leon Romanovsky <leonro@mellanox.com>
> 
> >From Michael,
> 
> This series handles IPoIB child interface creation with setting
> interface's HW address.
> 
> In current implementation, lladdr requested by user is ignored and
> overwritten. Child interface gets the same GID as the parent interface
> and a QP number which is assigned by the underlying drivers.
> 
> In this series we fix this behavior so that user's requested address is
> assigned to the newly created interface.
> 
> As specific QP number request is not supported for all vendors, QP
> number requested by user will still be overwritten when this is not
> supported.
> 
> Behavior of creation of child interfaces through the sysfs mechanism or
> without specifying a requested address, stays the same.
> 
> Thanks
> 
> Michael Guralnik (2):
>   net/mlx5: Enable QP number request when creating IPoIB underlay QP
>   RDMA/ipoib: Handle user-supplied address when creating child
> 
>  drivers/infiniband/ulp/ipoib/ipoib_main.c             | 11 +++++++++--
>  drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c |  7 +++++++
>  drivers/net/ethernet/mellanox/mlx5/core/main.c        |  3 +++
>  include/linux/mlx5/mlx5_ifc.h                         |  9 +++++++--
>  4 files changed, 26 insertions(+), 4 deletions(-)

Applied to for-next, thanks

Jason