[PATCH V4 0/8] net/mlx4_en: DCB QoS support

[PATCH V4 0/8] net/mlx4_en: DCB QoS support

[Amir Vadai]

DCBX version 802.1qaz is supported.
User Priority (UP) is set in QP context instead of in WQE (QP Work Queue
Element), which means that all traffic from a queue will have the same UP.
UP is also set for untagged traffic to be able to classify such traffic too.

Mapping from sk_prio to User Priority is done by sch_mqprio mapping. Although
confusingly sch_mqprio maps sk_prio to something called TC, it is not related
to DCBX's TC, and is interpreted by mlx4_en driver as UP.

User can set maximal BW for an ETS TC. This could be done by a new optional
attribute added to DCB_ATTR_IEEE called DCBNL DCB_ATTR_IEEE_MAXRATE. It accept
an array of 8 64 bits values, 1 for every ETS TC. Units are in Kbps.

CC: John Fastabend <john.r.fastabend@intel.com>
CC: Eric Dumazet <eric.dumazet@gmail.com>

---
Changes from V3:
After having a discussion about the API to set/get rate limit of an ETS TC,
decided to extend DCBNL with an optional attribute for that:
- new patch 7/8 Add an optional max rate attribute to DCBNL
- new patch 8/8 Set max rate-limit for a TC in mlx4 driver using the
  infrastructure from patch 7/8
Changes from V2:
- Removed patch 4 who deal with setting ratelimit through sysfs - Still under discussion
- Patch 2/7 - set port QoS attributes
  - ratelimit is given in Kbps instead of Mbps units.
Changes from V1:
- Patch 1/7 - Force user priority by QP attribute
  - removed unneeded comment
- Patch 2/7 - set port QoS attributes
  - ratelimit is given in Mbps instead of 100Mbps units.
- Patch 3/7 - DCB QoS support
  - Split mlx4_en_config_port_scheduler from mlx4_en_dcbnl_ieee_setets - enable
    setting ratelimit only, without setting up to tc mapping
- Patch 4/7 - Set max rate-limit for a TC
  - Rewritten patch according to comment from Eric D.
  - setting ratelimit is through sysfs files:
    /sys/class/net/<intf>/ratelimit/tc0
    /sys/class/net/<intf>/ratelimit/tc1
    ...
    /sys/class/net/<intf>/ratelimit/tc7
- Patch 5/7 - sk_prio <=> UP for untagged traffic  
  - Using skb_tx_hash for queue selection of untagged traffic
- Added Patch 6/7 - export symbol ip_tos2prio
  - Fixed patch according to comment from Eric D.
- Patch 7/7 - TOS <=> UP mapping for IBoE
  - No need to clone code from route.c anymore - removed it
    
Changes from V0:
- Removed patches 6 and 7 who deal with the interaction between the kernel HW
  QoS constructs to the queue selection logic are still under discussion with
  John and some changes might be needed there.

Amir Vadai (8):
  net/mlx4_en: Force user priority by QP attribute
  net/mlx4_core: set port QoS attributes
  net/mlx4_en: DCB QoS support
  net/mlx4_en: sk_prio <=> UP for untagged traffic
  net/route: export symbol ip_tos2prio
  IB/rdma_cm: TOS <=> UP mapping for IBoE
  net/dcb: Add an optional max rate attribute
  net/mlx4_en: Set max rate-limit for a TC

 drivers/infiniband/core/cma.c                     |    6 +-
 drivers/net/ethernet/mellanox/mlx4/Kconfig        |   12 +
 drivers/net/ethernet/mellanox/mlx4/Makefile       |    1 +
 drivers/net/ethernet/mellanox/mlx4/en_dcb_nl.c    |  242 +++++++++++++++++++++
 drivers/net/ethernet/mellanox/mlx4/en_main.c      |    2 +-
 drivers/net/ethernet/mellanox/mlx4/en_netdev.c    |   44 ++++-
 drivers/net/ethernet/mellanox/mlx4/en_port.h      |    2 +
 drivers/net/ethernet/mellanox/mlx4/en_resources.c |    6 +-
 drivers/net/ethernet/mellanox/mlx4/en_rx.c        |    4 +-
 drivers/net/ethernet/mellanox/mlx4/en_tx.c        |   10 +-
 drivers/net/ethernet/mellanox/mlx4/mlx4.h         |   21 ++
 drivers/net/ethernet/mellanox/mlx4/mlx4_en.h      |   27 ++-
 drivers/net/ethernet/mellanox/mlx4/port.c         |   67 ++++++
 include/linux/dcbnl.h                             |    5 +
 include/linux/mlx4/cmd.h                          |    4 +
 include/linux/mlx4/device.h                       |    3 +
 include/linux/mlx4/qp.h                           |    3 +-
 include/net/dcbnl.h                               |    2 +
 net/dcb/dcbnl.c                                   |   18 ++
 net/ipv4/route.c                                  |    2 +-
 20 files changed, 463 insertions(+), 18 deletions(-)
 create mode 100644 drivers/net/ethernet/mellanox/mlx4/en_dcb_nl.c

-- 
1.7.8.2

[PATCH V4 1/8] net/mlx4_en: Force user priority by QP attribute

[Amir Vadai]

Instead of relying on HW to change schedule queue by UP, schedule
queue is fixed for a tx_ring, and UP in WQE is ignored in this aspect.  This
resolves two issues with untagged traffic:
1. untagged traffic has no UP in packet which is needed for QoS. The change
   above allows setting the schedule queue (and by that the UP) of such a stream.
2. BlueFlame uses the same field used by vlan tag. So forcing UP from QPC
   allows using BF for untagged but prioritized traffic.

In old firmware that force UP is not supported, untagged traffic will not subject to
QoS.

Because UP is set by QP, need to always have a tx ring per UP, even if pfcrx
module paramter is false.

Signed-off-by: Amir Vadai <amirv@mellanox.com>
---
 drivers/net/ethernet/mellanox/mlx4/en_main.c      |    2 +-
 drivers/net/ethernet/mellanox/mlx4/en_netdev.c    |    3 ++-
 drivers/net/ethernet/mellanox/mlx4/en_resources.c |    6 +++++-
 drivers/net/ethernet/mellanox/mlx4/en_rx.c        |    4 ++--
 drivers/net/ethernet/mellanox/mlx4/en_tx.c        |   12 ++++--------
 drivers/net/ethernet/mellanox/mlx4/mlx4_en.h      |    6 +++---
 include/linux/mlx4/qp.h                           |    3 ++-
 7 files changed, 19 insertions(+), 17 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx4/en_main.c b/drivers/net/ethernet/mellanox/mlx4/en_main.c
index 2097a7d..346fdb2 100644
--- a/drivers/net/ethernet/mellanox/mlx4/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx4/en_main.c
@@ -114,7 +114,7 @@ static int mlx4_en_get_profile(struct mlx4_en_dev *mdev)
 		params->prof[i].tx_ring_size = MLX4_EN_DEF_TX_RING_SIZE;
 		params->prof[i].rx_ring_size = MLX4_EN_DEF_RX_RING_SIZE;
 		params->prof[i].tx_ring_num = MLX4_EN_NUM_TX_RINGS +
-			(!!pfcrx) * MLX4_EN_NUM_PPP_RINGS;
+			MLX4_EN_NUM_PPP_RINGS;
 		params->prof[i].rss_rings = 0;
 	}
 
diff --git a/drivers/net/ethernet/mellanox/mlx4/en_netdev.c b/drivers/net/ethernet/mellanox/mlx4/en_netdev.c
index 31b455a..2322622 100644
--- a/drivers/net/ethernet/mellanox/mlx4/en_netdev.c
+++ b/drivers/net/ethernet/mellanox/mlx4/en_netdev.c
@@ -650,7 +650,8 @@ int mlx4_en_start_port(struct net_device *dev)
 
 		/* Configure ring */
 		tx_ring = &priv->tx_ring[i];
-		err = mlx4_en_activate_tx_ring(priv, tx_ring, cq->mcq.cqn);
+		err = mlx4_en_activate_tx_ring(priv, tx_ring, cq->mcq.cqn,
+				max(0, i - MLX4_EN_NUM_TX_RINGS));
 		if (err) {
 			en_err(priv, "Failed allocating Tx ring\n");
 			mlx4_en_deactivate_cq(priv, cq);
diff --git a/drivers/net/ethernet/mellanox/mlx4/en_resources.c b/drivers/net/ethernet/mellanox/mlx4/en_resources.c
index bcbc54c..10c24c7 100644
--- a/drivers/net/ethernet/mellanox/mlx4/en_resources.c
+++ b/drivers/net/ethernet/mellanox/mlx4/en_resources.c
@@ -39,7 +39,7 @@
 
 void mlx4_en_fill_qp_context(struct mlx4_en_priv *priv, int size, int stride,
 			     int is_tx, int rss, int qpn, int cqn,
-			     struct mlx4_qp_context *context)
+			     int user_prio, struct mlx4_qp_context *context)
 {
 	struct mlx4_en_dev *mdev = priv->mdev;
 
@@ -57,6 +57,10 @@ void mlx4_en_fill_qp_context(struct mlx4_en_priv *priv, int size, int stride,
 	context->local_qpn = cpu_to_be32(qpn);
 	context->pri_path.ackto = 1 & 0x07;
 	context->pri_path.sched_queue = 0x83 | (priv->port - 1) << 6;
+	if (user_prio >= 0) {
+		context->pri_path.sched_queue |= user_prio << 3;
+		context->pri_path.feup = 1 << 6;
+	}
 	context->pri_path.counter_index = 0xff;
 	context->cqn_send = cpu_to_be32(cqn);
 	context->cqn_recv = cpu_to_be32(cqn);
diff --git a/drivers/net/ethernet/mellanox/mlx4/en_rx.c b/drivers/net/ethernet/mellanox/mlx4/en_rx.c
index 9adbd53..d49a7ac 100644
--- a/drivers/net/ethernet/mellanox/mlx4/en_rx.c
+++ b/drivers/net/ethernet/mellanox/mlx4/en_rx.c
@@ -823,7 +823,7 @@ static int mlx4_en_config_rss_qp(struct mlx4_en_priv *priv, int qpn,
 
 	memset(context, 0, sizeof *context);
 	mlx4_en_fill_qp_context(priv, ring->actual_size, ring->stride, 0, 0,
-				qpn, ring->cqn, context);
+				qpn, ring->cqn, -1, context);
 	context->db_rec_addr = cpu_to_be64(ring->wqres.db.dma);
 
 	/* Cancel FCS removal if FW allows */
@@ -890,7 +890,7 @@ int mlx4_en_config_rss_steer(struct mlx4_en_priv *priv)
 	}
 	rss_map->indir_qp.event = mlx4_en_sqp_event;
 	mlx4_en_fill_qp_context(priv, 0, 0, 0, 1, priv->base_qpn,
-				priv->rx_ring[0].cqn, &context);
+				priv->rx_ring[0].cqn, -1, &context);
 
 	if (!priv->prof->rss_rings || priv->prof->rss_rings > priv->rx_ring_num)
 		rss_rings = priv->rx_ring_num;
diff --git a/drivers/net/ethernet/mellanox/mlx4/en_tx.c b/drivers/net/ethernet/mellanox/mlx4/en_tx.c
index 1796824..94a605a 100644
--- a/drivers/net/ethernet/mellanox/mlx4/en_tx.c
+++ b/drivers/net/ethernet/mellanox/mlx4/en_tx.c
@@ -156,7 +156,7 @@ void mlx4_en_destroy_tx_ring(struct mlx4_en_priv *priv,
 
 int mlx4_en_activate_tx_ring(struct mlx4_en_priv *priv,
 			     struct mlx4_en_tx_ring *ring,
-			     int cq)
+			     int cq, int user_prio)
 {
 	struct mlx4_en_dev *mdev = priv->mdev;
 	int err;
@@ -174,7 +174,7 @@ int mlx4_en_activate_tx_ring(struct mlx4_en_priv *priv,
 	ring->doorbell_qpn = ring->qp.qpn << 8;
 
 	mlx4_en_fill_qp_context(priv, ring->size, ring->stride, 1, 0, ring->qpn,
-				ring->cqn, &ring->context);
+				ring->cqn, user_prio, &ring->context);
 	if (ring->bf_enabled)
 		ring->context.usr_page = cpu_to_be32(ring->bf.uar->index);
 
@@ -570,18 +570,14 @@ static void build_inline_wqe(struct mlx4_en_tx_desc *tx_desc, struct sk_buff *sk
 
 u16 mlx4_en_select_queue(struct net_device *dev, struct sk_buff *skb)
 {
-	struct mlx4_en_priv *priv = netdev_priv(dev);
 	u16 vlan_tag = 0;
 
-	/* If we support per priority flow control and the packet contains
-	 * a vlan tag, send the packet to the TX ring assigned to that priority
-	 */
-	if (priv->prof->rx_ppp && vlan_tx_tag_present(skb)) {
+	if (vlan_tx_tag_present(skb)) {
 		vlan_tag = vlan_tx_tag_get(skb);
 		return MLX4_EN_NUM_TX_RINGS + (vlan_tag >> 13);
 	}
 
-	return skb_tx_hash(dev, skb);
+	return __skb_tx_hash(dev, skb, MLX4_EN_NUM_TX_RINGS);
 }
 
 static void mlx4_bf_copy(void __iomem *dst, unsigned long *src, unsigned bytecnt)
diff --git a/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h b/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h
index 9e2b911..5bd7c2a 100644
--- a/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h
+++ b/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h
@@ -521,7 +521,7 @@ int mlx4_en_create_tx_ring(struct mlx4_en_priv *priv, struct mlx4_en_tx_ring *ri
 void mlx4_en_destroy_tx_ring(struct mlx4_en_priv *priv, struct mlx4_en_tx_ring *ring);
 int mlx4_en_activate_tx_ring(struct mlx4_en_priv *priv,
 			     struct mlx4_en_tx_ring *ring,
-			     int cq);
+			     int cq, int user_prio);
 void mlx4_en_deactivate_tx_ring(struct mlx4_en_priv *priv,
 				struct mlx4_en_tx_ring *ring);
 
@@ -539,8 +539,8 @@ int mlx4_en_process_rx_cq(struct net_device *dev,
 			  int budget);
 int mlx4_en_poll_rx_cq(struct napi_struct *napi, int budget);
 void mlx4_en_fill_qp_context(struct mlx4_en_priv *priv, int size, int stride,
-			     int is_tx, int rss, int qpn, int cqn,
-			     struct mlx4_qp_context *context);
+		int is_tx, int rss, int qpn, int cqn, int user_prio,
+		struct mlx4_qp_context *context);
 void mlx4_en_sqp_event(struct mlx4_qp *qp, enum mlx4_event event);
 int mlx4_en_map_buffer(struct mlx4_buf *buf);
 void mlx4_en_unmap_buffer(struct mlx4_buf *buf);
diff --git a/include/linux/mlx4/qp.h b/include/linux/mlx4/qp.h
index 091f9e7..96005d7 100644
--- a/include/linux/mlx4/qp.h
+++ b/include/linux/mlx4/qp.h
@@ -139,7 +139,8 @@ struct mlx4_qp_path {
 	u8			rgid[16];
 	u8			sched_queue;
 	u8			vlan_index;
-	u8			reserved3[2];
+	u8			feup;
+	u8			reserved3;
 	u8			reserved4[2];
 	u8			dmac[6];
 };
-- 
1.7.8.2

[PATCH V4 2/8] net/mlx4_core: set port QoS attributes

[Amir Vadai]

Adding QoS firmware commands:
- mlx4_en_SET_PORT_PRIO2TC - set UP <=> TC
- mlx4_en_SET_PORT_SCHEDULER - set promised BW, max BW and PG number

Signed-off-by: Amir Vadai <amirv@mellanox.com>
---
 drivers/net/ethernet/mellanox/mlx4/en_port.h |    2 +
 drivers/net/ethernet/mellanox/mlx4/mlx4.h    |   21 ++++++++
 drivers/net/ethernet/mellanox/mlx4/port.c    |   67 ++++++++++++++++++++++++++
 include/linux/mlx4/cmd.h                     |    4 ++
 include/linux/mlx4/device.h                  |    3 +
 5 files changed, 97 insertions(+), 0 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx4/en_port.h b/drivers/net/ethernet/mellanox/mlx4/en_port.h
index 6934fd7..745090b 100644
--- a/drivers/net/ethernet/mellanox/mlx4/en_port.h
+++ b/drivers/net/ethernet/mellanox/mlx4/en_port.h
@@ -39,6 +39,8 @@
 #define SET_PORT_PROMISC_SHIFT	31
 #define SET_PORT_MC_PROMISC_SHIFT	30
 
+#define MLX4_EN_NUM_TC		8
+
 #define VLAN_FLTR_SIZE	128
 struct mlx4_set_vlan_fltr_mbox {
 	__be32 entry[VLAN_FLTR_SIZE];
diff --git a/drivers/net/ethernet/mellanox/mlx4/mlx4.h b/drivers/net/ethernet/mellanox/mlx4/mlx4.h
index 2a0ff2c..98cbb2f 100644
--- a/drivers/net/ethernet/mellanox/mlx4/mlx4.h
+++ b/drivers/net/ethernet/mellanox/mlx4/mlx4.h
@@ -53,6 +53,27 @@
 #define DRV_VERSION	"1.1"
 #define DRV_RELDATE	"Dec, 2011"
 
+#define MLX4_NUM_UP		8
+#define MLX4_NUM_TC		8
+#define MLX4_RATELIMIT_UNITS 3 /* 100 Mbps */
+#define MLX4_RATELIMIT_KB_TO_HW(_mb) ((_mb) / 100000)
+#define MLX4_RATELIMIT_DEFAULT 0xffff
+
+struct mlx4_set_port_prio2tc_context {
+	u8 prio2tc[4];
+};
+
+struct mlx4_port_scheduler_tc_cfg_be {
+	__be16 pg;
+	__be16 bw_precentage;
+	__be16 max_bw_units; /* 3-100Mbps, 4-1Gbps, other values - reserved */
+	__be16 max_bw_value;
+};
+
+struct mlx4_set_port_scheduler_context {
+	struct mlx4_port_scheduler_tc_cfg_be tc[MLX4_NUM_TC];
+};
+
 enum {
 	MLX4_HCR_BASE		= 0x80680,
 	MLX4_HCR_SIZE		= 0x0001c,
diff --git a/drivers/net/ethernet/mellanox/mlx4/port.c b/drivers/net/ethernet/mellanox/mlx4/port.c
index 77535ff..e619cd5 100644
--- a/drivers/net/ethernet/mellanox/mlx4/port.c
+++ b/drivers/net/ethernet/mellanox/mlx4/port.c
@@ -834,6 +834,73 @@ int mlx4_SET_PORT_qpn_calc(struct mlx4_dev *dev, u8 port, u32 base_qpn,
 }
 EXPORT_SYMBOL(mlx4_SET_PORT_qpn_calc);
 
+int mlx4_SET_PORT_PRIO2TC(struct mlx4_dev *dev, u8 port, u8 *prio2tc)
+{
+	struct mlx4_cmd_mailbox *mailbox;
+	struct mlx4_set_port_prio2tc_context *context;
+	int err;
+	u32 in_mod;
+	int i;
+
+	mailbox = mlx4_alloc_cmd_mailbox(dev);
+	if (IS_ERR(mailbox))
+		return PTR_ERR(mailbox);
+	context = mailbox->buf;
+	memset(context, 0, sizeof *context);
+
+	for (i = 0; i < MLX4_NUM_UP; i += 2)
+		context->prio2tc[i >> 1] = prio2tc[i] << 4 | prio2tc[i + 1];
+
+	in_mod = MLX4_SET_PORT_PRIO2TC << 8 | port;
+	err = mlx4_cmd(dev, mailbox->dma, in_mod, 1, MLX4_CMD_SET_PORT,
+		       MLX4_CMD_TIME_CLASS_B, MLX4_CMD_NATIVE);
+
+	mlx4_free_cmd_mailbox(dev, mailbox);
+	return err;
+}
+EXPORT_SYMBOL(mlx4_SET_PORT_PRIO2TC);
+
+int mlx4_SET_PORT_SCHEDULER(struct mlx4_dev *dev, u8 port, u8 *tc_tx_bw,
+		u8 *pg, u64 *ratelimit)
+{
+	struct mlx4_cmd_mailbox *mailbox;
+	struct mlx4_set_port_scheduler_context *context;
+	int err;
+	u32 in_mod;
+	int i;
+
+	mailbox = mlx4_alloc_cmd_mailbox(dev);
+	if (IS_ERR(mailbox))
+		return PTR_ERR(mailbox);
+	context = mailbox->buf;
+	memset(context, 0, sizeof *context);
+
+	for (i = 0; i < MLX4_NUM_TC; i++) {
+		struct mlx4_port_scheduler_tc_cfg_be *tc = &context->tc[i];
+		u16 r = 0;
+
+		if (ratelimit)
+			r = MLX4_RATELIMIT_KB_TO_HW(ratelimit[i]);
+
+		if (!r)
+			r = MLX4_RATELIMIT_DEFAULT;
+
+		tc->pg = htons(pg[i]);
+		tc->bw_precentage = htons(tc_tx_bw[i]);
+
+		tc->max_bw_units = htons(MLX4_RATELIMIT_UNITS);
+		tc->max_bw_value = htons(r);
+	}
+
+	in_mod = MLX4_SET_PORT_SCHEDULER << 8 | port;
+	err = mlx4_cmd(dev, mailbox->dma, in_mod, 1, MLX4_CMD_SET_PORT,
+		       MLX4_CMD_TIME_CLASS_B, MLX4_CMD_NATIVE);
+
+	mlx4_free_cmd_mailbox(dev, mailbox);
+	return err;
+}
+EXPORT_SYMBOL(mlx4_SET_PORT_SCHEDULER);
+
 int mlx4_SET_MCAST_FLTR_wrapper(struct mlx4_dev *dev, int slave,
 				struct mlx4_vhcr *vhcr,
 				struct mlx4_cmd_mailbox *inbox,
diff --git a/include/linux/mlx4/cmd.h b/include/linux/mlx4/cmd.h
index 9958ff2..1f3860a 100644
--- a/include/linux/mlx4/cmd.h
+++ b/include/linux/mlx4/cmd.h
@@ -150,6 +150,10 @@ enum {
 	/* statistics commands */
 	MLX4_CMD_QUERY_IF_STAT	 = 0X54,
 	MLX4_CMD_SET_IF_STAT	 = 0X55,
+
+	/* set port opcode modifiers */
+	MLX4_SET_PORT_PRIO2TC = 0x8,
+	MLX4_SET_PORT_SCHEDULER  = 0x9,
 };
 
 enum {
diff --git a/include/linux/mlx4/device.h b/include/linux/mlx4/device.h
index 834c96c..6627136 100644
--- a/include/linux/mlx4/device.h
+++ b/include/linux/mlx4/device.h
@@ -628,6 +628,9 @@ int mlx4_SET_PORT_general(struct mlx4_dev *dev, u8 port, int mtu,
 			  u8 pptx, u8 pfctx, u8 pprx, u8 pfcrx);
 int mlx4_SET_PORT_qpn_calc(struct mlx4_dev *dev, u8 port, u32 base_qpn,
 			   u8 promisc);
+int mlx4_SET_PORT_PRIO2TC(struct mlx4_dev *dev, u8 port, u8 *prio2tc);
+int mlx4_SET_PORT_SCHEDULER(struct mlx4_dev *dev, u8 port, u8 *tc_tx_bw,
+		u8 *pg, u64 *ratelimit);
 int mlx4_find_cached_vlan(struct mlx4_dev *dev, u8 port, u16 vid, int *idx);
 int mlx4_register_vlan(struct mlx4_dev *dev, u8 port, u16 vlan, int *index);
 void mlx4_unregister_vlan(struct mlx4_dev *dev, u8 port, int index);
-- 
1.7.8.2

[PATCH V4 3/8] net/mlx4_en: DCB QoS support

[Amir Vadai]

Set TSA, promised BW and PFC using IEEE 802.1qaz netlink commands.

Signed-off-by: Amir Vadai <amirv@mellanox.com>
---
 drivers/net/ethernet/mellanox/mlx4/Kconfig     |   12 ++
 drivers/net/ethernet/mellanox/mlx4/Makefile    |    1 +
 drivers/net/ethernet/mellanox/mlx4/en_dcb_nl.c |  211 ++++++++++++++++++++++++
 drivers/net/ethernet/mellanox/mlx4/en_netdev.c |   16 ++
 drivers/net/ethernet/mellanox/mlx4/mlx4_en.h   |   21 +++
 5 files changed, 261 insertions(+), 0 deletions(-)
 create mode 100644 drivers/net/ethernet/mellanox/mlx4/en_dcb_nl.c

diff --git a/drivers/net/ethernet/mellanox/mlx4/Kconfig b/drivers/net/ethernet/mellanox/mlx4/Kconfig
index 1bb9353..5f027f9 100644
--- a/drivers/net/ethernet/mellanox/mlx4/Kconfig
+++ b/drivers/net/ethernet/mellanox/mlx4/Kconfig
@@ -11,6 +11,18 @@ config MLX4_EN
 	  This driver supports Mellanox Technologies ConnectX Ethernet
 	  devices.
 
+config MLX4_EN_DCB
+	bool "Data Center Bridging (DCB) Support"
+	default y
+	depends on MLX4_EN && DCB
+	---help---
+	  Say Y here if you want to use Data Center Bridging (DCB) in the
+	  driver.
+	  If set to N, will not be able to configure QoS and ratelimit attributes.
+	  This flag is depended on the kernel's DCB support.
+
+	  If unsure, set to Y
+
 config MLX4_CORE
 	tristate
 	depends on PCI
diff --git a/drivers/net/ethernet/mellanox/mlx4/Makefile b/drivers/net/ethernet/mellanox/mlx4/Makefile
index 4a40ab9..293127d 100644
--- a/drivers/net/ethernet/mellanox/mlx4/Makefile
+++ b/drivers/net/ethernet/mellanox/mlx4/Makefile
@@ -7,3 +7,4 @@ obj-$(CONFIG_MLX4_EN)               += mlx4_en.o
 
 mlx4_en-y := 	en_main.o en_tx.o en_rx.o en_ethtool.o en_port.o en_cq.o \
 		en_resources.o en_netdev.o en_selftest.o
+mlx4_en-$(CONFIG_MLX4_EN_DCB) += en_dcb_nl.o
diff --git a/drivers/net/ethernet/mellanox/mlx4/en_dcb_nl.c b/drivers/net/ethernet/mellanox/mlx4/en_dcb_nl.c
new file mode 100644
index 0000000..5bf0106
--- /dev/null
+++ b/drivers/net/ethernet/mellanox/mlx4/en_dcb_nl.c
@@ -0,0 +1,211 @@
+/*
+ * Copyright (c) 2011 Mellanox Technologies. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses.  You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ *     Redistribution and use in source and binary forms, with or
+ *     without modification, are permitted provided that the following
+ *     conditions are met:
+ *
+ *      - Redistributions of source code must retain the above
+ *        copyright notice, this list of conditions and the following
+ *        disclaimer.
+ *
+ *      - Redistributions in binary form must reproduce the above
+ *        copyright notice, this list of conditions and the following
+ *        disclaimer in the documentation and/or other materials
+ *        provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ *
+ */
+
+#include <linux/dcbnl.h>
+
+#include "mlx4_en.h"
+
+static int mlx4_en_dcbnl_ieee_getets(struct net_device *dev,
+				   struct ieee_ets *ets)
+{
+	struct mlx4_en_priv *priv = netdev_priv(dev);
+	struct ieee_ets *my_ets = priv->mlx4_en_ieee_ets;
+
+	/* No IEEE PFC settings available */
+	if (!my_ets)
+		return -EINVAL;
+
+	ets->ets_cap = IEEE_8021QAZ_MAX_TCS;
+	ets->cbs = my_ets->cbs;
+	memcpy(ets->tc_tx_bw, my_ets->tc_tx_bw, sizeof(ets->tc_tx_bw));
+	memcpy(ets->tc_tsa, my_ets->tc_tsa, sizeof(ets->tc_tsa));
+	memcpy(ets->prio_tc, my_ets->prio_tc, sizeof(ets->prio_tc));
+
+	return 0;
+}
+
+static int mlx4_en_ets_validate(struct mlx4_en_priv *priv, struct ieee_ets *ets)
+{
+	int i;
+	int total_ets_bw = 0;
+	int has_ets_tc = 0;
+
+	for (i = 0; i < IEEE_8021QAZ_MAX_TCS; i++) {
+		if (ets->prio_tc[i] > MLX4_EN_NUM_UP) {
+			en_err(priv, "Bad priority in UP <=> TC mapping. "
+					"TC: %d, UP: %d\n", i, ets->prio_tc[i]);
+			return -EINVAL;
+		}
+
+		switch (ets->tc_tsa[i]) {
+		case IEEE_8021QAZ_TSA_STRICT:
+			break;
+		case IEEE_8021QAZ_TSA_ETS:
+			has_ets_tc = 1;
+			total_ets_bw += ets->tc_tx_bw[i];
+			break;
+		default:
+			en_err(priv, "TC[%d]: Not supported TSA: %d\n",
+					i, ets->tc_tsa[i]);
+			return -ENOTSUPP;
+		}
+	}
+
+	if (has_ets_tc && total_ets_bw != MLX4_EN_BW_MAX) {
+		en_err(priv, "Bad ETS BW sum: %d. Should be exactly 100%%\n",
+				total_ets_bw);
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static int mlx4_en_config_port_scheduler(struct mlx4_en_priv *priv,
+		struct ieee_ets *ets, u64 *ratelimit)
+{
+	struct mlx4_en_dev *mdev = priv->mdev;
+	int num_strict = 0;
+	int i;
+	__u8 tc_tx_bw[IEEE_8021QAZ_MAX_TCS] = { 0 };
+	__u8 pg[IEEE_8021QAZ_MAX_TCS] = { 0 };
+
+	ets = ets ?: priv->mlx4_en_ieee_ets;
+
+	/* higher TC means higher priority => lower pg */
+	for (i = IEEE_8021QAZ_MAX_TCS - 1; i >= 0; i--) {
+		switch (ets->tc_tsa[i]) {
+		case IEEE_8021QAZ_TSA_STRICT:
+			pg[i] = num_strict++;
+			tc_tx_bw[i] = MLX4_EN_BW_MAX;
+			break;
+		case IEEE_8021QAZ_TSA_ETS:
+			pg[i] = MLX4_EN_TC_ETS;
+			tc_tx_bw[i] = ets->tc_tx_bw[i] ?: MLX4_EN_BW_MIN;
+			break;
+		}
+	}
+
+	return mlx4_SET_PORT_SCHEDULER(mdev->dev, priv->port, tc_tx_bw, pg,
+			ratelimit);
+}
+
+static int
+mlx4_en_dcbnl_ieee_setets(struct net_device *dev, struct ieee_ets *ets)
+{
+	struct mlx4_en_priv *priv = netdev_priv(dev);
+	struct mlx4_en_dev *mdev = priv->mdev;
+	int err;
+
+	err = mlx4_en_ets_validate(priv, ets);
+	if (err)
+		return err;
+
+	err = mlx4_SET_PORT_PRIO2TC(mdev->dev, priv->port, ets->prio_tc);
+	if (err)
+		return err;
+
+	err = mlx4_en_config_port_scheduler(priv, ets, NULL);
+	if (err)
+		return err;
+
+	if (ets != priv->mlx4_en_ieee_ets)
+		memcpy(priv->mlx4_en_ieee_ets, ets,
+				sizeof(*priv->mlx4_en_ieee_ets));
+
+	return 0;
+}
+
+static int mlx4_en_dcbnl_ieee_getpfc(struct net_device *dev,
+		struct ieee_pfc *pfc)
+{
+	struct mlx4_en_priv *priv = netdev_priv(dev);
+
+	pfc->pfc_cap = IEEE_8021QAZ_MAX_TCS;
+	pfc->pfc_en = priv->prof->tx_ppp;
+
+	return 0;
+}
+
+static int mlx4_en_dcbnl_ieee_setpfc(struct net_device *dev,
+		struct ieee_pfc *pfc)
+{
+	struct mlx4_en_priv *priv = netdev_priv(dev);
+	struct mlx4_en_dev *mdev = priv->mdev;
+	int err;
+
+	en_dbg(DRV, priv, "cap: 0x%x en: 0x%x mbc: 0x%x delay: %d\n",
+			pfc->pfc_cap,
+			pfc->pfc_en,
+			pfc->mbc,
+			pfc->delay);
+
+	priv->prof->rx_pause = priv->prof->tx_pause = !!pfc->pfc_en;
+	priv->prof->rx_ppp = priv->prof->tx_ppp = pfc->pfc_en;
+
+	err = mlx4_SET_PORT_general(mdev->dev, priv->port,
+				    priv->rx_skb_size + ETH_FCS_LEN,
+				    priv->prof->tx_pause,
+				    priv->prof->tx_ppp,
+				    priv->prof->rx_pause,
+				    priv->prof->rx_ppp);
+	if (err)
+		en_err(priv, "Failed setting pause params\n");
+
+	return err;
+}
+
+static u8 mlx4_en_dcbnl_getdcbx(struct net_device *dev)
+{
+	return DCB_CAP_DCBX_VER_IEEE;
+}
+
+static u8 mlx4_en_dcbnl_setdcbx(struct net_device *dev, u8 mode)
+{
+	if ((mode & DCB_CAP_DCBX_LLD_MANAGED) ||
+	    (mode & DCB_CAP_DCBX_VER_CEE) ||
+	    !(mode & DCB_CAP_DCBX_VER_IEEE) ||
+	    !(mode & DCB_CAP_DCBX_HOST))
+		return 1;
+
+	return 0;
+}
+
+const struct dcbnl_rtnl_ops mlx4_en_dcbnl_ops = {
+	.ieee_getets	= mlx4_en_dcbnl_ieee_getets,
+	.ieee_setets	= mlx4_en_dcbnl_ieee_setets,
+	.ieee_getpfc	= mlx4_en_dcbnl_ieee_getpfc,
+	.ieee_setpfc	= mlx4_en_dcbnl_ieee_setpfc,
+
+	.getdcbx	= mlx4_en_dcbnl_getdcbx,
+	.setdcbx	= mlx4_en_dcbnl_setdcbx,
+};
diff --git a/drivers/net/ethernet/mellanox/mlx4/en_netdev.c b/drivers/net/ethernet/mellanox/mlx4/en_netdev.c
index 2322622..9b456ae 100644
--- a/drivers/net/ethernet/mellanox/mlx4/en_netdev.c
+++ b/drivers/net/ethernet/mellanox/mlx4/en_netdev.c
@@ -967,6 +967,11 @@ void mlx4_en_destroy_netdev(struct net_device *dev)
 	mutex_unlock(&mdev->state_lock);
 
 	mlx4_en_free_resources(priv);
+
+#ifdef CONFIG_MLX4_EN_DCB
+	vfree(priv->mlx4_en_ieee_ets);
+#endif
+
 	free_netdev(dev);
 }
 
@@ -1080,6 +1085,17 @@ int mlx4_en_init_netdev(struct mlx4_en_dev *mdev, int port,
 	INIT_WORK(&priv->watchdog_task, mlx4_en_restart);
 	INIT_WORK(&priv->linkstate_task, mlx4_en_linkstate);
 	INIT_DELAYED_WORK(&priv->stats_task, mlx4_en_do_get_stats);
+#ifdef CONFIG_MLX4_EN_DCB
+	if (!mlx4_is_slave(priv->mdev->dev)) {
+		dev->dcbnl_ops = &mlx4_en_dcbnl_ops;
+
+		priv->mlx4_en_ieee_ets = vzalloc(sizeof(struct ieee_ets));
+		if (!priv->mlx4_en_ieee_ets) {
+			err = -ENOMEM;
+			goto out;
+		}
+	}
+#endif
 
 	/* Query for default mac and max mtu */
 	priv->max_mtu = mdev->dev->caps.eth_mtu_cap[priv->port];
diff --git a/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h b/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h
index 5bd7c2a..fa09792 100644
--- a/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h
+++ b/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h
@@ -40,6 +40,9 @@
 #include <linux/mutex.h>
 #include <linux/netdevice.h>
 #include <linux/if_vlan.h>
+#ifdef CONFIG_MLX4_EN_DCB
+#include <linux/dcbnl.h>
+#endif
 
 #include <linux/mlx4/device.h>
 #include <linux/mlx4/qp.h>
@@ -110,6 +113,7 @@ enum {
 #define MLX4_EN_NUM_TX_RINGS		8
 #define MLX4_EN_NUM_PPP_RINGS		8
 #define MAX_TX_RINGS			(MLX4_EN_NUM_TX_RINGS + MLX4_EN_NUM_PPP_RINGS)
+#define MLX4_EN_NUM_UP			8
 #define MLX4_EN_DEF_TX_RING_SIZE	512
 #define MLX4_EN_DEF_RX_RING_SIZE  	1024
 
@@ -410,6 +414,15 @@ struct mlx4_en_frag_info {
 
 };
 
+#ifdef CONFIG_MLX4_EN_DCB
+/* Minimal TC BW - setting to 0 will block traffic */
+#define MLX4_EN_BW_MIN 1
+#define MLX4_EN_BW_MAX 100 /* Utilize 100% of the line */
+
+#define MLX4_EN_TC_ETS 7
+
+#endif
+
 struct mlx4_en_priv {
 	struct mlx4_en_dev *mdev;
 	struct mlx4_en_port_profile *prof;
@@ -483,6 +496,10 @@ struct mlx4_en_priv {
 	int vids[128];
 	bool wol;
 	struct device *ddev;
+
+#ifdef CONFIG_MLX4_EN_DCB
+	struct ieee_ets *mlx4_en_ieee_ets;
+#endif
 };
 
 enum mlx4_en_wol {
@@ -557,6 +574,10 @@ int mlx4_SET_VLAN_FLTR(struct mlx4_dev *dev, struct mlx4_en_priv *priv);
 int mlx4_en_DUMP_ETH_STATS(struct mlx4_en_dev *mdev, u8 port, u8 reset);
 int mlx4_en_QUERY_PORT(struct mlx4_en_dev *mdev, u8 port);
 
+#ifdef CONFIG_MLX4_EN_DCB
+extern const struct dcbnl_rtnl_ops mlx4_en_dcbnl_ops;
+#endif
+
 #define MLX4_EN_NUM_SELF_TEST	5
 void mlx4_en_ex_selftest(struct net_device *dev, u32 *flags, u64 *buf);
 u64 mlx4_en_mac_to_u64(u8 *addr);
-- 
1.7.8.2

[PATCH V4 4/8] net/mlx4_en: sk_prio <=> UP for untagged traffic

[Amir Vadai]

Since vlan egress map is only good for tagged traffic, need to have other
mapping to be used by untagged traffic.
For that, the driver uses sch_mqprio mapping. This mapping could be set by
using tc tool from iproute2 package.
Mapped UP will be used by the HW for QoS purposes, but won't go out on the
wire.

Signed-off-by: Amir Vadai <amirv@mellanox.com>
---
 drivers/net/ethernet/mellanox/mlx4/en_netdev.c |   18 ++++++++++++++++++
 drivers/net/ethernet/mellanox/mlx4/en_tx.c     |    2 +-
 2 files changed, 19 insertions(+), 1 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx4/en_netdev.c b/drivers/net/ethernet/mellanox/mlx4/en_netdev.c
index 9b456ae..44dbe1f 100644
--- a/drivers/net/ethernet/mellanox/mlx4/en_netdev.c
+++ b/drivers/net/ethernet/mellanox/mlx4/en_netdev.c
@@ -45,6 +45,14 @@
 #include "mlx4_en.h"
 #include "en_port.h"
 
+static int mlx4_en_setup_tc(struct net_device *dev, u8 up)
+{
+	if (up != MLX4_EN_NUM_UP)
+		return -EINVAL;
+
+	return 0;
+}
+
 static int mlx4_en_vlan_rx_add_vid(struct net_device *dev, unsigned short vid)
 {
 	struct mlx4_en_priv *priv = netdev_priv(dev);
@@ -1042,6 +1050,7 @@ static const struct net_device_ops mlx4_netdev_ops = {
 	.ndo_poll_controller	= mlx4_en_netpoll,
 #endif
 	.ndo_set_features	= mlx4_en_set_features,
+	.ndo_setup_tc		= mlx4_en_setup_tc,
 };
 
 int mlx4_en_init_netdev(struct mlx4_en_dev *mdev, int port,
@@ -1130,6 +1139,15 @@ int mlx4_en_init_netdev(struct mlx4_en_dev *mdev, int port,
 	netif_set_real_num_tx_queues(dev, priv->tx_ring_num);
 	netif_set_real_num_rx_queues(dev, priv->rx_ring_num);
 
+	netdev_set_num_tc(dev, MLX4_EN_NUM_UP);
+
+	/* First 9 rings are for UP 0 */
+	netdev_set_tc_queue(dev, 0, MLX4_EN_NUM_TX_RINGS + 1, 0);
+
+	/* Partition Tx queues evenly amongst UP's 1-7 */
+	for (i = 1; i < MLX4_EN_NUM_UP; i++)
+		netdev_set_tc_queue(dev, i, 1, MLX4_EN_NUM_TX_RINGS + i);
+
 	SET_ETHTOOL_OPS(dev, &mlx4_en_ethtool_ops);
 
 	/* Set defualt MAC */
diff --git a/drivers/net/ethernet/mellanox/mlx4/en_tx.c b/drivers/net/ethernet/mellanox/mlx4/en_tx.c
index 94a605a..d9bab53 100644
--- a/drivers/net/ethernet/mellanox/mlx4/en_tx.c
+++ b/drivers/net/ethernet/mellanox/mlx4/en_tx.c
@@ -577,7 +577,7 @@ u16 mlx4_en_select_queue(struct net_device *dev, struct sk_buff *skb)
 		return MLX4_EN_NUM_TX_RINGS + (vlan_tag >> 13);
 	}
 
-	return __skb_tx_hash(dev, skb, MLX4_EN_NUM_TX_RINGS);
+	return skb_tx_hash(dev, skb);
 }
 
 static void mlx4_bf_copy(void __iomem *dst, unsigned long *src, unsigned bytecnt)
-- 
1.7.8.2

[PATCH V4 5/8] net/route: export symbol ip_tos2prio

[Amir Vadai]

Need to export this to enable drivers use rt_tos2priority()

Signed-off-by: Amir Vadai <amirv@mellanox.com>
---
 net/ipv4/route.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/net/ipv4/route.c b/net/ipv4/route.c
index 12ccf88..69e0b86 100644
--- a/net/ipv4/route.c
+++ b/net/ipv4/route.c
@@ -230,7 +230,7 @@ const __u8 ip_tos2prio[16] = {
 	TC_PRIO_INTERACTIVE_BULK,
 	ECN_OR_COST(INTERACTIVE_BULK)
 };
-
+EXPORT_SYMBOL(ip_tos2prio);
 
 /*
  * Route cache.
-- 
1.7.8.2

[PATCH V4 6/8] IB/rdma_cm: TOS <=> UP mapping for IBoE

[Amir Vadai]

Both tagged traffic and untagged traffic use tc tool mapping.
Treat RDMA TOS same as IP TOS when mapping to SL

Signed-off-by: Amir Vadai <amirv@mellanox.com>
CC: Sean Hefty <sean.hefty@intel.com>
---
 drivers/infiniband/core/cma.c |    6 +++++-
 1 files changed, 5 insertions(+), 1 deletions(-)

diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index e3e470f..59fbd70 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -42,6 +42,7 @@
 #include <linux/inetdevice.h>
 #include <linux/slab.h>
 #include <linux/module.h>
+#include <net/route.h>
 
 #include <net/tcp.h>
 #include <net/ipv6.h>
@@ -1826,7 +1827,10 @@ static int cma_resolve_iboe_route(struct rdma_id_private *id_priv)
 	route->path_rec->reversible = 1;
 	route->path_rec->pkey = cpu_to_be16(0xffff);
 	route->path_rec->mtu_selector = IB_SA_EQ;
-	route->path_rec->sl = id_priv->tos >> 5;
+	route->path_rec->sl = netdev_get_prio_tc_map(
+			ndev->priv_flags & IFF_802_1Q_VLAN ?
+				vlan_dev_real_dev(ndev) : ndev,
+			rt_tos2priority(id_priv->tos));
 
 	route->path_rec->mtu = iboe_get_mtu(ndev->mtu);
 	route->path_rec->rate_selector = IB_SA_EQ;
-- 
1.7.8.2

[PATCH V4 7/8] net/dcb: Add an optional max rate attribute

[Amir Vadai]

Although not specified in 8021Qaz spec, it could be useful to enable drivers
setting a rate limit for an ETS TC.
This patch add this optional attribute to DCB netlink.
To use it, drivers should implement and register the callbacks ieee_setmaxrate
and ieee_getmaxrate.
Units are 64 bits long and in units of Kbps to enable it to be used both by
slow and very fast networks

Signed-off-by: Amir Vadai <amirv@mellanox.com>
---
 include/linux/dcbnl.h |    5 +++++
 include/net/dcbnl.h   |    2 ++
 net/dcb/dcbnl.c       |   18 ++++++++++++++++++
 3 files changed, 25 insertions(+), 0 deletions(-)

diff --git a/include/linux/dcbnl.h b/include/linux/dcbnl.h
index 65a2562..ec8e372 100644
--- a/include/linux/dcbnl.h
+++ b/include/linux/dcbnl.h
@@ -67,6 +67,10 @@ struct ieee_ets {
 	__u8	reco_prio_tc[IEEE_8021QAZ_MAX_TCS];
 };
 
+struct ieee_maxrate {
+	__u64	tc_maxrate[IEEE_8021QAZ_MAX_TCS];
+};
+
 /* This structure contains the IEEE 802.1Qaz PFC managed object
  *
  * @pfc_cap: Indicates the number of traffic classes on the local device
@@ -321,6 +325,7 @@ enum ieee_attrs {
 	DCB_ATTR_IEEE_PEER_ETS,
 	DCB_ATTR_IEEE_PEER_PFC,
 	DCB_ATTR_IEEE_PEER_APP,
+	DCB_ATTR_IEEE_MAXRATE,
 	__DCB_ATTR_IEEE_MAX
 };
 #define DCB_ATTR_IEEE_MAX (__DCB_ATTR_IEEE_MAX - 1)
diff --git a/include/net/dcbnl.h b/include/net/dcbnl.h
index f55c980..fc5d5dc 100644
--- a/include/net/dcbnl.h
+++ b/include/net/dcbnl.h
@@ -48,6 +48,8 @@ struct dcbnl_rtnl_ops {
 	/* IEEE 802.1Qaz std */
 	int (*ieee_getets) (struct net_device *, struct ieee_ets *);
 	int (*ieee_setets) (struct net_device *, struct ieee_ets *);
+	int (*ieee_getmaxrate) (struct net_device *, struct ieee_maxrate *);
+	int (*ieee_setmaxrate) (struct net_device *, struct ieee_maxrate *);
 	int (*ieee_getpfc) (struct net_device *, struct ieee_pfc *);
 	int (*ieee_setpfc) (struct net_device *, struct ieee_pfc *);
 	int (*ieee_getapp) (struct net_device *, struct dcb_app *);
diff --git a/net/dcb/dcbnl.c b/net/dcb/dcbnl.c
index d860530..77dbc1d 100644
--- a/net/dcb/dcbnl.c
+++ b/net/dcb/dcbnl.c
@@ -178,6 +178,8 @@ static const struct nla_policy dcbnl_ieee_policy[DCB_ATTR_IEEE_MAX + 1] = {
 	[DCB_ATTR_IEEE_ETS]	    = {.len = sizeof(struct ieee_ets)},
 	[DCB_ATTR_IEEE_PFC]	    = {.len = sizeof(struct ieee_pfc)},
 	[DCB_ATTR_IEEE_APP_TABLE]   = {.type = NLA_NESTED},
+	[DCB_ATTR_IEEE_MAXRATE]   = {.len = sizeof(struct ieee_maxrate)},
+
 };
 
 static const struct nla_policy dcbnl_ieee_app[DCB_ATTR_IEEE_APP_MAX + 1] = {
@@ -1243,6 +1245,14 @@ static int dcbnl_ieee_fill(struct sk_buff *skb, struct net_device *netdev)
 			NLA_PUT(skb, DCB_ATTR_IEEE_ETS, sizeof(ets), &ets);
 	}
 
+	if (ops->ieee_getmaxrate) {
+		struct ieee_maxrate maxrate;
+		err = ops->ieee_getmaxrate(netdev, &maxrate);
+		if (!err)
+			NLA_PUT(skb, DCB_ATTR_IEEE_MAXRATE, sizeof(maxrate),
+					&maxrate);
+	}
+
 	if (ops->ieee_getpfc) {
 		struct ieee_pfc pfc;
 		err = ops->ieee_getpfc(netdev, &pfc);
@@ -1589,6 +1599,14 @@ static int dcbnl_ieee_set(struct net_device *netdev, struct nlattr **tb,
 			goto err;
 	}
 
+	if (ieee[DCB_ATTR_IEEE_MAXRATE] && ops->ieee_setmaxrate) {
+		struct ieee_maxrate *maxrate =
+			nla_data(ieee[DCB_ATTR_IEEE_MAXRATE]);
+		err = ops->ieee_setmaxrate(netdev, maxrate);
+		if (err)
+			goto err;
+	}
+
 	if (ieee[DCB_ATTR_IEEE_PFC] && ops->ieee_setpfc) {
 		struct ieee_pfc *pfc = nla_data(ieee[DCB_ATTR_IEEE_PFC]);
 		err = ops->ieee_setpfc(netdev, pfc);
-- 
1.7.8.2

[PATCH V4 8/8] net/mlx4_en: Set max rate-limit for a TC

[Amir Vadai]

This patch is using the DCB netlink to set rate limit per ETS TC

Signed-off-by: Amir Vadai <amirv@mellanox.com>
---
 drivers/net/ethernet/mellanox/mlx4/en_dcb_nl.c |   31 ++++++++++++++++++++++++
 drivers/net/ethernet/mellanox/mlx4/en_netdev.c |    7 +++++
 drivers/net/ethernet/mellanox/mlx4/mlx4_en.h   |    2 +-
 3 files changed, 39 insertions(+), 1 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx4/en_dcb_nl.c b/drivers/net/ethernet/mellanox/mlx4/en_dcb_nl.c
index 5bf0106..0f92b2e 100644
--- a/drivers/net/ethernet/mellanox/mlx4/en_dcb_nl.c
+++ b/drivers/net/ethernet/mellanox/mlx4/en_dcb_nl.c
@@ -100,6 +100,7 @@ static int mlx4_en_config_port_scheduler(struct mlx4_en_priv *priv,
 	__u8 pg[IEEE_8021QAZ_MAX_TCS] = { 0 };
 
 	ets = ets ?: priv->mlx4_en_ieee_ets;
+	ratelimit = ratelimit ?: priv->maxrate->tc_maxrate;
 
 	/* higher TC means higher priority => lower pg */
 	for (i = IEEE_8021QAZ_MAX_TCS - 1; i >= 0; i--) {
@@ -200,9 +201,39 @@ static u8 mlx4_en_dcbnl_setdcbx(struct net_device *dev, u8 mode)
 	return 0;
 }
 
+static int mlx4_en_dcbnl_ieee_getmaxrate(struct net_device *dev,
+				   struct ieee_maxrate *maxrate)
+{
+	struct mlx4_en_priv *priv = netdev_priv(dev);
+
+	if (!priv->maxrate)
+		return -EINVAL;
+
+	memcpy(maxrate, priv->maxrate, sizeof(maxrate));
+
+	return 0;
+}
+
+static int mlx4_en_dcbnl_ieee_setmaxrate(struct net_device *dev,
+		struct ieee_maxrate *maxrate)
+{
+	int err;
+	struct mlx4_en_priv *priv = netdev_priv(dev);
+
+	err = mlx4_en_config_port_scheduler(priv, NULL, maxrate->tc_maxrate);
+	if (err)
+		return err;
+
+	memcpy(priv->maxrate, maxrate, sizeof(priv->maxrate));
+
+	return 0;
+}
+
 const struct dcbnl_rtnl_ops mlx4_en_dcbnl_ops = {
 	.ieee_getets	= mlx4_en_dcbnl_ieee_getets,
 	.ieee_setets	= mlx4_en_dcbnl_ieee_setets,
+	.ieee_getmaxrate = mlx4_en_dcbnl_ieee_getmaxrate,
+	.ieee_setmaxrate = mlx4_en_dcbnl_ieee_setmaxrate,
 	.ieee_getpfc	= mlx4_en_dcbnl_ieee_getpfc,
 	.ieee_setpfc	= mlx4_en_dcbnl_ieee_setpfc,
 
diff --git a/drivers/net/ethernet/mellanox/mlx4/en_netdev.c b/drivers/net/ethernet/mellanox/mlx4/en_netdev.c
index 44dbe1f..be7ffbf 100644
--- a/drivers/net/ethernet/mellanox/mlx4/en_netdev.c
+++ b/drivers/net/ethernet/mellanox/mlx4/en_netdev.c
@@ -978,6 +978,7 @@ void mlx4_en_destroy_netdev(struct net_device *dev)
 
 #ifdef CONFIG_MLX4_EN_DCB
 	vfree(priv->mlx4_en_ieee_ets);
+	vfree(priv->maxrate);
 #endif
 
 	free_netdev(dev);
@@ -1103,6 +1104,12 @@ int mlx4_en_init_netdev(struct mlx4_en_dev *mdev, int port,
 			err = -ENOMEM;
 			goto out;
 		}
+
+		priv->maxrate = vzalloc(sizeof(struct ieee_maxrate));
+		if (!priv->maxrate) {
+			err = -ENOMEM;
+			goto out;
+		}
 	}
 #endif
 
diff --git a/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h b/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h
index fa09792..d1f5ac2 100644
--- a/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h
+++ b/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h
@@ -420,7 +420,6 @@ struct mlx4_en_frag_info {
 #define MLX4_EN_BW_MAX 100 /* Utilize 100% of the line */
 
 #define MLX4_EN_TC_ETS 7
-
 #endif
 
 struct mlx4_en_priv {
@@ -499,6 +498,7 @@ struct mlx4_en_priv {
 
 #ifdef CONFIG_MLX4_EN_DCB
 	struct ieee_ets *mlx4_en_ieee_ets;
+	struct ieee_maxrate *maxrate;
 #endif
 };
 
-- 
1.7.8.2

Re: [PATCH V4 8/8] net/mlx4_en: Set max rate-limit for a TC

[Eric Dumazet]

On Thu, 2012-03-29 at 17:03 +0200, Amir Vadai wrote:
> This patch is using the DCB netlink to set rate limit per ETS TC
> 
> Signed-off-by: Amir Vadai <amirv@mellanox.com>
> ---
>  drivers/net/ethernet/mellanox/mlx4/en_dcb_nl.c |   31 ++++++++++++++++++++++++
>  drivers/net/ethernet/mellanox/mlx4/en_netdev.c |    7 +++++
>  drivers/net/ethernet/mellanox/mlx4/mlx4_en.h   |    2 +-
>  3 files changed, 39 insertions(+), 1 deletions(-)
> 
> diff --git a/drivers/net/ethernet/mellanox/mlx4/en_dcb_nl.c b/drivers/net/ethernet/mellanox/mlx4/en_dcb_nl.c
> index 5bf0106..0f92b2e 100644
> --- a/drivers/net/ethernet/mellanox/mlx4/en_dcb_nl.c
> +++ b/drivers/net/ethernet/mellanox/mlx4/en_dcb_nl.c
> @@ -100,6 +100,7 @@ static int mlx4_en_config_port_scheduler(struct mlx4_en_priv *priv,
>  	__u8 pg[IEEE_8021QAZ_MAX_TCS] = { 0 };
>  
>  	ets = ets ?: priv->mlx4_en_ieee_ets;
> +	ratelimit = ratelimit ?: priv->maxrate->tc_maxrate;
>  
>  	/* higher TC means higher priority => lower pg */
>  	for (i = IEEE_8021QAZ_MAX_TCS - 1; i >= 0; i--) {
> @@ -200,9 +201,39 @@ static u8 mlx4_en_dcbnl_setdcbx(struct net_device *dev, u8 mode)
>  	return 0;
>  }
>  
> +static int mlx4_en_dcbnl_ieee_getmaxrate(struct net_device *dev,
> +				   struct ieee_maxrate *maxrate)
> +{
> +	struct mlx4_en_priv *priv = netdev_priv(dev);
> +
> +	if (!priv->maxrate)
> +		return -EINVAL;
> +
> +	memcpy(maxrate, priv->maxrate, sizeof(maxrate));

sizeof(*maxrate) is probably what you wanted ?

> +
> +	return 0;
> +}
> +
> +static int mlx4_en_dcbnl_ieee_setmaxrate(struct net_device *dev,
> +		struct ieee_maxrate *maxrate)
> +{
> +	int err;
> +	struct mlx4_en_priv *priv = netdev_priv(dev);
> +
> +	err = mlx4_en_config_port_scheduler(priv, NULL, maxrate->tc_maxrate);
> +	if (err)
> +		return err;
> +
> +	memcpy(priv->maxrate, maxrate, sizeof(priv->maxrate));

Really ?

sizeof(priv->maxrate) seems to be 8/4 (sizeof a pointer)



> +
> +	return 0;
> +}
> +
>  const struct dcbnl_rtnl_ops mlx4_en_dcbnl_ops = {
>  	.ieee_getets	= mlx4_en_dcbnl_ieee_getets,
>  	.ieee_setets	= mlx4_en_dcbnl_ieee_setets,
> +	.ieee_getmaxrate = mlx4_en_dcbnl_ieee_getmaxrate,
> +	.ieee_setmaxrate = mlx4_en_dcbnl_ieee_setmaxrate,
>  	.ieee_getpfc	= mlx4_en_dcbnl_ieee_getpfc,
>  	.ieee_setpfc	= mlx4_en_dcbnl_ieee_setpfc,
>  
> diff --git a/drivers/net/ethernet/mellanox/mlx4/en_netdev.c b/drivers/net/ethernet/mellanox/mlx4/en_netdev.c
> index 44dbe1f..be7ffbf 100644
> --- a/drivers/net/ethernet/mellanox/mlx4/en_netdev.c
> +++ b/drivers/net/ethernet/mellanox/mlx4/en_netdev.c
> @@ -978,6 +978,7 @@ void mlx4_en_destroy_netdev(struct net_device *dev)
>  
>  #ifdef CONFIG_MLX4_EN_DCB
>  	vfree(priv->mlx4_en_ieee_ets);
> +	vfree(priv->maxrate);
>  #endif
>  
>  	free_netdev(dev);
> @@ -1103,6 +1104,12 @@ int mlx4_en_init_netdev(struct mlx4_en_dev *mdev, int port,
>  			err = -ENOMEM;
>  			goto out;
>  		}
> +
> +		priv->maxrate = vzalloc(sizeof(struct ieee_maxrate));

vzalloc() ? What is the size of struct ieee_maxrate ?

> +		if (!priv->maxrate) {
> +			err = -ENOMEM;
> +			goto out;
> +		}
>  	}
>  #endif
>  
> diff --git a/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h b/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h
> index fa09792..d1f5ac2 100644
> --- a/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h
> +++ b/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h
> @@ -420,7 +420,6 @@ struct mlx4_en_frag_info {
>  #define MLX4_EN_BW_MAX 100 /* Utilize 100% of the line */
>  
>  #define MLX4_EN_TC_ETS 7
> -
>  #endif
>  
>  struct mlx4_en_priv {
> @@ -499,6 +498,7 @@ struct mlx4_en_priv {
>  
>  #ifdef CONFIG_MLX4_EN_DCB
>  	struct ieee_ets *mlx4_en_ieee_ets;
> +	struct ieee_maxrate *maxrate;
>  #endif
>  };
>  

Re: [PATCH V4 8/8] net/mlx4_en: Set max rate-limit for a TC

[Amir Vadai]

On 03/29/2012 05:21 PM, Eric Dumazet wrote:
> On Thu, 2012-03-29 at 17:03 +0200, Amir Vadai wrote:
>> This patch is using the DCB netlink to set rate limit per ETS TC
>>
>> Signed-off-by: Amir Vadai<amirv@mellanox.com>
>> ---
>>   drivers/net/ethernet/mellanox/mlx4/en_dcb_nl.c |   31 ++++++++++++++++++++++++
>>   drivers/net/ethernet/mellanox/mlx4/en_netdev.c |    7 +++++
>>   drivers/net/ethernet/mellanox/mlx4/mlx4_en.h   |    2 +-
>>   3 files changed, 39 insertions(+), 1 deletions(-)
>>
>> diff --git a/drivers/net/ethernet/mellanox/mlx4/en_dcb_nl.c b/drivers/net/ethernet/mellanox/mlx4/en_dcb_nl.c
>> index 5bf0106..0f92b2e 100644
>> --- a/drivers/net/ethernet/mellanox/mlx4/en_dcb_nl.c
>> +++ b/drivers/net/ethernet/mellanox/mlx4/en_dcb_nl.c
>> @@ -100,6 +100,7 @@ static int mlx4_en_config_port_scheduler(struct mlx4_en_priv *priv,
>>   	__u8 pg[IEEE_8021QAZ_MAX_TCS] = { 0 };
>>
>>   	ets = ets ?: priv->mlx4_en_ieee_ets;
>> +	ratelimit = ratelimit ?: priv->maxrate->tc_maxrate;
>>
>>   	/* higher TC means higher priority =>  lower pg */
>>   	for (i = IEEE_8021QAZ_MAX_TCS - 1; i>= 0; i--) {
>> @@ -200,9 +201,39 @@ static u8 mlx4_en_dcbnl_setdcbx(struct net_device *dev, u8 mode)
>>   	return 0;
>>   }
>>
>> +static int mlx4_en_dcbnl_ieee_getmaxrate(struct net_device *dev,
>> +				   struct ieee_maxrate *maxrate)
>> +{
>> +	struct mlx4_en_priv *priv = netdev_priv(dev);
>> +
>> +	if (!priv->maxrate)
>> +		return -EINVAL;
>> +
>> +	memcpy(maxrate, priv->maxrate, sizeof(maxrate));
>
> sizeof(*maxrate) is probably what you wanted ?
>
>> +
>> +	return 0;
>> +}
>> +
>> +static int mlx4_en_dcbnl_ieee_setmaxrate(struct net_device *dev,
>> +		struct ieee_maxrate *maxrate)
>> +{
>> +	int err;
>> +	struct mlx4_en_priv *priv = netdev_priv(dev);
>> +
>> +	err = mlx4_en_config_port_scheduler(priv, NULL, maxrate->tc_maxrate);
>> +	if (err)
>> +		return err;
>> +
>> +	memcpy(priv->maxrate, maxrate, sizeof(priv->maxrate));
>
> Really ?
>
> sizeof(priv->maxrate) seems to be 8/4 (sizeof a pointer)
>
>
>
>> +
>> +	return 0;
>> +}
>> +
>>   const struct dcbnl_rtnl_ops mlx4_en_dcbnl_ops = {
>>   	.ieee_getets	= mlx4_en_dcbnl_ieee_getets,
>>   	.ieee_setets	= mlx4_en_dcbnl_ieee_setets,
>> +	.ieee_getmaxrate = mlx4_en_dcbnl_ieee_getmaxrate,
>> +	.ieee_setmaxrate = mlx4_en_dcbnl_ieee_setmaxrate,
>>   	.ieee_getpfc	= mlx4_en_dcbnl_ieee_getpfc,
>>   	.ieee_setpfc	= mlx4_en_dcbnl_ieee_setpfc,
>>
>> diff --git a/drivers/net/ethernet/mellanox/mlx4/en_netdev.c b/drivers/net/ethernet/mellanox/mlx4/en_netdev.c
>> index 44dbe1f..be7ffbf 100644
>> --- a/drivers/net/ethernet/mellanox/mlx4/en_netdev.c
>> +++ b/drivers/net/ethernet/mellanox/mlx4/en_netdev.c
>> @@ -978,6 +978,7 @@ void mlx4_en_destroy_netdev(struct net_device *dev)
>>
>>   #ifdef CONFIG_MLX4_EN_DCB
>>   	vfree(priv->mlx4_en_ieee_ets);
>> +	vfree(priv->maxrate);
>>   #endif
>>
>>   	free_netdev(dev);
>> @@ -1103,6 +1104,12 @@ int mlx4_en_init_netdev(struct mlx4_en_dev *mdev, int port,
>>   			err = -ENOMEM;
>>   			goto out;
>>   		}
>> +
>> +		priv->maxrate = vzalloc(sizeof(struct ieee_maxrate));
>
> vzalloc() ? What is the size of struct ieee_maxrate ?
>
>> +		if (!priv->maxrate) {
>> +			err = -ENOMEM;
>> +			goto out;
>> +		}
>>   	}
>>   #endif
>>
>> diff --git a/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h b/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h
>> index fa09792..d1f5ac2 100644
>> --- a/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h
>> +++ b/drivers/net/ethernet/mellanox/mlx4/mlx4_en.h
>> @@ -420,7 +420,6 @@ struct mlx4_en_frag_info {
>>   #define MLX4_EN_BW_MAX 100 /* Utilize 100% of the line */
>>
>>   #define MLX4_EN_TC_ETS 7
>> -
>>   #endif
>>
>>   struct mlx4_en_priv {
>> @@ -499,6 +498,7 @@ struct mlx4_en_priv {
>>
>>   #ifdef CONFIG_MLX4_EN_DCB
>>   	struct ieee_ets *mlx4_en_ieee_ets;
>> +	struct ieee_maxrate *maxrate;
>>   #endif
>>   };
>>
>
>
I was in a rush to send it on time and missed it (of course I tested 
only tc 0 so didn't see it in my tests too).
Anyway, will fix it.

- Amir

Re: [PATCH V4 8/8] net/mlx4_en: Set max rate-limit for a TC

[Eric Dumazet]

Le jeudi 29 mars 2012 à 20:27 +0200, Amir Vadai a écrit :

> I was in a rush to send it on time and missed it (of course I tested 
> only tc 0 so didn't see it in my tests too).
> Anyway, will fix it.

Not sure what you mean by "on time".

Take your time, linux wont freeze today nor next week.

Re: [PATCH V4 8/8] net/mlx4_en: Set max rate-limit for a TC

[David Miller]

From: Amir Vadai <amirv@mellanox.com>
Date: Thu, 29 Mar 2012 20:27:20 +0200

> I was in a rush to send it on time and missed it (of course I tested
> only tc 0 so didn't see it in my tests too).

Being in a rush when you expect others to spend time reviewing your
code is an extremely selfish act.

Re: [PATCH V4 8/8] net/mlx4_en: Set max rate-limit for a TC

[Or Gerlitz]

On Fri, Mar 30, 2012 at 12:25 AM, David Miller <davem@davemloft.net> wrote:
>
> From: Amir Vadai <amirv@mellanox.com>
> Date: Thu, 29 Mar 2012 20:27:20 +0200
>
> > I was in a rush to send it on time and missed it (of course I tested
> > only tc 0 so didn't see it in my tests too).
>
> Being in a rush when you expect others to spend time reviewing your
> code is an extremely selfish act.

Hi Dave,

Yep, a mistake was done here, but as they say "only he who never does
never errs..."
so Amir will surely learn from that, and so am I - here wearing the
internal reviewer hat,
didn't review V4 deep enough. We spotted also some vzalloc calls in
the patches which
are under the HW QoS model debate and will fix there as well. V5 will
be also against
net-next but depending on your decision maybe it can still go to 3.4?

Or.

Re: [PATCH V4 8/8] net/mlx4_en: Set max rate-limit for a TC

[Or Gerlitz]

On Sun, Apr 1, 2012 at 10:04 AM, Or Gerlitz <or.gerlitz@gmail.com> wrote:
> V5 will be also against net-next but depending on your decision maybe it can
> still go to 3.4?

Hi Dave, I didn't see a reply from you on this question, but just made
a comment in another thread that net-next haven't opened yet, can we
still submit this for net (3.4) or we should wait for net-next (3.5)?

Or.

Re: [PATCH V4 8/8] net/mlx4_en: Set max rate-limit for a TC

[David Miller]

From: Or Gerlitz <or.gerlitz@gmail.com>
Date: Sun, 1 Apr 2012 22:30:24 +0300

> On Sun, Apr 1, 2012 at 10:04 AM, Or Gerlitz <or.gerlitz@gmail.com> wrote:
>> V5 will be also against net-next but depending on your decision maybe it can
>> still go to 3.4?
> 
> Hi Dave, I didn't see a reply from you on this question, but just made
> a comment in another thread that net-next haven't opened yet, can we
> still submit this for net (3.4) or we should wait for net-next (3.5)?

New features are never valid for the net tree, only net-next.  Therefore
your question doesn't make any sense, because it is plain as day that
a patch like this is only appropriate for net-next.

Re: [PATCH V4 8/8] net/mlx4_en: Set max rate-limit for a TC

[Or Gerlitz]

On Sun, Apr 1, 2012 at 10:39 PM, David Miller <davem@davemloft.net> wrote:
> New features are never valid for the net tree, only net-next.  Therefore
> your question doesn't make any sense, because it is plain as day that
> a patch like this is only appropriate for net-next.

okay, will do

Or.