[PATCH v3 0/6] Calxeda xgmac performance fixes

[PATCH v3 0/6] Calxeda xgmac performance fixes

[Rob Herring]

From: Rob Herring <rob.herring@calxeda.com>

This is a series of performance improvements to the xgmac driver. The
most significant changes are the alignment fixes to avoid alignment
traps on received frames and using raw i/o accessors.

v3:
- Only patch 3 changed. Use raw i/o accessors instead of relaxed for
better build coverage.

v2:
- Only patch 5 changed. Add a missing enabling of tx irq.

Rob

Rob Herring (6):
  net: calxedaxgmac: enable operate on 2nd frame mode
  net: calxedaxgmac: remove explicit rx dma buffer polling
  net: calxedaxgmac: use raw i/o accessors in rx and tx paths
  net: calxedaxgmac: drop some unnecessary register writes
  net: calxedaxgmac: rework transmit ring handling
  net: calxedaxgmac: ip align receive buffers

 drivers/net/ethernet/calxeda/xgmac.c |   59 +++++++++++++++-------------------
 1 file changed, 26 insertions(+), 33 deletions(-)

-- 
1.7.10.4

[PATCH v3 1/6] net: calxedaxgmac: enable operate on 2nd frame mode

[Rob Herring]

From: Rob Herring <rob.herring@calxeda.com>

Enable the tx dma to start reading the next frame while sending the current
frame.

Signed-off-by: Rob Herring <rob.herring@calxeda.com>
---
 drivers/net/ethernet/calxeda/xgmac.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/calxeda/xgmac.c b/drivers/net/ethernet/calxeda/xgmac.c
index 16814b3..7f5fd17 100644
--- a/drivers/net/ethernet/calxeda/xgmac.c
+++ b/drivers/net/ethernet/calxeda/xgmac.c
@@ -191,6 +191,7 @@
 #define DMA_CONTROL_ST		0x00002000	/* Start/Stop Transmission */
 #define DMA_CONTROL_SR		0x00000002	/* Start/Stop Receive */
 #define DMA_CONTROL_DFF		0x01000000	/* Disable flush of rx frames */
+#define DMA_CONTROL_OSF		0x00000004	/* Operate on 2nd tx frame */
 
 /* DMA Normal interrupt */
 #define DMA_INTR_ENA_NIE	0x00010000	/* Normal Summary */
@@ -965,8 +966,7 @@ static int xgmac_hw_init(struct net_device *dev)
 		ctrl |= XGMAC_CONTROL_IPC;
 	writel(ctrl, ioaddr + XGMAC_CONTROL);
 
-	value = DMA_CONTROL_DFF;
-	writel(value, ioaddr + XGMAC_DMA_CONTROL);
+	writel(DMA_CONTROL_DFF | DMA_CONTROL_OSF, ioaddr + XGMAC_DMA_CONTROL);
 
 	/* Set the HW DMA mode and the COE */
 	writel(XGMAC_OMR_TSF | XGMAC_OMR_RFD | XGMAC_OMR_RFA |
-- 
1.7.10.4

[PATCH v3 2/6] net: calxedaxgmac: remove explicit rx dma buffer polling

[Rob Herring]

From: Rob Herring <rob.herring@calxeda.com>

New received frames will trigger the rx DMA to poll the DMA descriptors,
so there is no need to tell the h/w to poll. We also want to enable
dropping frames from the fifo when there is no buffer.

Signed-off-by: Rob Herring <rob.herring@calxeda.com>
---
 drivers/net/ethernet/calxeda/xgmac.c |    4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/drivers/net/ethernet/calxeda/xgmac.c b/drivers/net/ethernet/calxeda/xgmac.c
index 7f5fd17..728fcef 100644
--- a/drivers/net/ethernet/calxeda/xgmac.c
+++ b/drivers/net/ethernet/calxeda/xgmac.c
@@ -966,7 +966,7 @@ static int xgmac_hw_init(struct net_device *dev)
 		ctrl |= XGMAC_CONTROL_IPC;
 	writel(ctrl, ioaddr + XGMAC_CONTROL);
 
-	writel(DMA_CONTROL_DFF | DMA_CONTROL_OSF, ioaddr + XGMAC_DMA_CONTROL);
+	writel(DMA_CONTROL_OSF, ioaddr + XGMAC_DMA_CONTROL);
 
 	/* Set the HW DMA mode and the COE */
 	writel(XGMAC_OMR_TSF | XGMAC_OMR_RFD | XGMAC_OMR_RFA |
@@ -1180,8 +1180,6 @@ static int xgmac_rx(struct xgmac_priv *priv, int limit)
 
 	xgmac_rx_refill(priv);
 
-	writel(1, priv->base + XGMAC_DMA_RX_POLL);
-
 	return count;
 }
 
-- 
1.7.10.4

[PATCH v3 3/6] net: calxedaxgmac: use raw i/o accessors in rx and tx paths

[Rob Herring]

From: Rob Herring <rob.herring@calxeda.com>

The standard readl/writel accessors involve a spinlock and cache sync
operation on ARM platforms with an outer cache. Only DMA triggering
accesses need this, so use the raw variants instead in the critical paths.

The relaxed variants would be more appropriate, but don't exist on all
arches.

Signed-off-by: Rob Herring <rob.herring@calxeda.com>
---
v3:
- Use raw i/o accessors instead of relaxed for better build coverage.

 drivers/net/ethernet/calxeda/xgmac.c |   12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/drivers/net/ethernet/calxeda/xgmac.c b/drivers/net/ethernet/calxeda/xgmac.c
index 728fcef..84cd40e 100644
--- a/drivers/net/ethernet/calxeda/xgmac.c
+++ b/drivers/net/ethernet/calxeda/xgmac.c
@@ -1203,7 +1203,7 @@ static int xgmac_poll(struct napi_struct *napi, int budget)
 
 	if (work_done < budget) {
 		napi_complete(napi);
-		writel(DMA_INTR_DEFAULT_MASK, priv->base + XGMAC_DMA_INTR_ENA);
+		__raw_writel(DMA_INTR_DEFAULT_MASK, priv->base + XGMAC_DMA_INTR_ENA);
 	}
 	return work_done;
 }
@@ -1348,7 +1348,7 @@ static irqreturn_t xgmac_pmt_interrupt(int irq, void *dev_id)
 	struct xgmac_priv *priv = netdev_priv(dev);
 	void __iomem *ioaddr = priv->base;
 
-	intr_status = readl(ioaddr + XGMAC_INT_STAT);
+	intr_status = __raw_readl(ioaddr + XGMAC_INT_STAT);
 	if (intr_status & XGMAC_INT_STAT_PMT) {
 		netdev_dbg(priv->dev, "received Magic frame\n");
 		/* clear the PMT bits 5 and 6 by reading the PMT */
@@ -1366,9 +1366,9 @@ static irqreturn_t xgmac_interrupt(int irq, void *dev_id)
 	struct xgmac_extra_stats *x = &priv->xstats;
 
 	/* read the status register (CSR5) */
-	intr_status = readl(priv->base + XGMAC_DMA_STATUS);
-	intr_status &= readl(priv->base + XGMAC_DMA_INTR_ENA);
-	writel(intr_status, priv->base + XGMAC_DMA_STATUS);
+	intr_status = __raw_readl(priv->base + XGMAC_DMA_STATUS);
+	intr_status &= __raw_readl(priv->base + XGMAC_DMA_INTR_ENA);
+	__raw_writel(intr_status, priv->base + XGMAC_DMA_STATUS);
 
 	/* It displays the DMA process states (CSR5 register) */
 	/* ABNORMAL interrupts */
@@ -1404,7 +1404,7 @@ static irqreturn_t xgmac_interrupt(int irq, void *dev_id)
 
 	/* TX/RX NORMAL interrupts */
 	if (intr_status & (DMA_STATUS_RI | DMA_STATUS_TU)) {
-		writel(DMA_INTR_ABNORMAL, priv->base + XGMAC_DMA_INTR_ENA);
+		__raw_writel(DMA_INTR_ABNORMAL, priv->base + XGMAC_DMA_INTR_ENA);
 		napi_schedule(&priv->napi);
 	}
 
-- 
1.7.10.4

[PATCH v3 4/6] net: calxedaxgmac: drop some unnecessary register writes

[Rob Herring]

From: Rob Herring <rob.herring@calxeda.com>

The interrupts have already been cleared, so we don't need to clear them
again. Also, we could miss interrupts if they are cleared, but we don't
process the packet.

Signed-off-by: Rob Herring <rob.herring@calxeda.com>
---
 drivers/net/ethernet/calxeda/xgmac.c |    6 ------
 1 file changed, 6 deletions(-)

diff --git a/drivers/net/ethernet/calxeda/xgmac.c b/drivers/net/ethernet/calxeda/xgmac.c
index 84cd40e..8263219 100644
--- a/drivers/net/ethernet/calxeda/xgmac.c
+++ b/drivers/net/ethernet/calxeda/xgmac.c
@@ -846,9 +846,6 @@ static void xgmac_free_dma_desc_rings(struct xgmac_priv *priv)
 static void xgmac_tx_complete(struct xgmac_priv *priv)
 {
 	int i;
-	void __iomem *ioaddr = priv->base;
-
-	writel(DMA_STATUS_TU | DMA_STATUS_NIS, ioaddr + XGMAC_DMA_STATUS);
 
 	while (dma_ring_cnt(priv->tx_head, priv->tx_tail, DMA_TX_RING_SZ)) {
 		unsigned int entry = priv->tx_tail;
@@ -1139,9 +1136,6 @@ static int xgmac_rx(struct xgmac_priv *priv, int limit)
 		struct sk_buff *skb;
 		int frame_len;
 
-		writel(DMA_STATUS_RI | DMA_STATUS_NIS,
-		       priv->base + XGMAC_DMA_STATUS);
-
 		entry = priv->rx_tail;
 		p = priv->dma_rx + entry;
 		if (desc_get_owner(p))
-- 
1.7.10.4

[PATCH v3 5/6] net: calxedaxgmac: rework transmit ring handling

[Rob Herring]

From: Rob Herring <rob.herring@calxeda.com>

Only generate tx interrupts on every ring size / 4 descriptors. Move the
netif_stop_queue call to the end of the xmit function rather than
checking at the beginning.

Signed-off-by: Rob Herring <rob.herring@calxeda.com>
---
 drivers/net/ethernet/calxeda/xgmac.c |   24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)

diff --git a/drivers/net/ethernet/calxeda/xgmac.c b/drivers/net/ethernet/calxeda/xgmac.c
index 8263219..362b35e 100644
--- a/drivers/net/ethernet/calxeda/xgmac.c
+++ b/drivers/net/ethernet/calxeda/xgmac.c
@@ -211,7 +211,7 @@
 #define DMA_INTR_ENA_TIE	0x00000001	/* Transmit Interrupt */
 
 #define DMA_INTR_NORMAL		(DMA_INTR_ENA_NIE | DMA_INTR_ENA_RIE | \
-				 DMA_INTR_ENA_TUE)
+				 DMA_INTR_ENA_TUE | DMA_INTR_ENA_TIE)
 
 #define DMA_INTR_ABNORMAL	(DMA_INTR_ENA_AIE | DMA_INTR_ENA_FBE | \
 				 DMA_INTR_ENA_RWE | DMA_INTR_ENA_RSE | \
@@ -374,6 +374,7 @@ struct xgmac_priv {
 	struct sk_buff **tx_skbuff;
 	unsigned int tx_head;
 	unsigned int tx_tail;
+	int tx_irq_cnt;
 
 	void __iomem *base;
 	unsigned int dma_buf_sz;
@@ -886,7 +887,7 @@ static void xgmac_tx_complete(struct xgmac_priv *priv)
 	}
 
 	if (dma_ring_space(priv->tx_head, priv->tx_tail, DMA_TX_RING_SZ) >
-	    TX_THRESH)
+	    MAX_SKB_FRAGS)
 		netif_wake_queue(priv->dev);
 }
 
@@ -1057,19 +1058,15 @@ static netdev_tx_t xgmac_xmit(struct sk_buff *skb, struct net_device *dev)
 	struct xgmac_priv *priv = netdev_priv(dev);
 	unsigned int entry;
 	int i;
+	u32 irq_flag;
 	int nfrags = skb_shinfo(skb)->nr_frags;
 	struct xgmac_dma_desc *desc, *first;
 	unsigned int desc_flags;
 	unsigned int len;
 	dma_addr_t paddr;
 
-	if (dma_ring_space(priv->tx_head, priv->tx_tail, DMA_TX_RING_SZ) <
-	    (nfrags + 1)) {
-		writel(DMA_INTR_DEFAULT_MASK | DMA_INTR_ENA_TIE,
-			priv->base + XGMAC_DMA_INTR_ENA);
-		netif_stop_queue(dev);
-		return NETDEV_TX_BUSY;
-	}
+	priv->tx_irq_cnt = (priv->tx_irq_cnt + 1) & (DMA_TX_RING_SZ/4 - 1);
+	irq_flag = priv->tx_irq_cnt ? 0 : TXDESC_INTERRUPT;
 
 	desc_flags = (skb->ip_summed == CHECKSUM_PARTIAL) ?
 		TXDESC_CSUM_ALL : 0;
@@ -1110,9 +1107,9 @@ static netdev_tx_t xgmac_xmit(struct sk_buff *skb, struct net_device *dev)
 	/* Interrupt on completition only for the latest segment */
 	if (desc != first)
 		desc_set_tx_owner(desc, desc_flags |
-			TXDESC_LAST_SEG | TXDESC_INTERRUPT);
+			TXDESC_LAST_SEG | irq_flag);
 	else
-		desc_flags |= TXDESC_LAST_SEG | TXDESC_INTERRUPT;
+		desc_flags |= TXDESC_LAST_SEG | irq_flag;
 
 	/* Set owner on first desc last to avoid race condition */
 	wmb();
@@ -1121,6 +1118,9 @@ static netdev_tx_t xgmac_xmit(struct sk_buff *skb, struct net_device *dev)
 	priv->tx_head = dma_ring_incr(entry, DMA_TX_RING_SZ);
 
 	writel(1, priv->base + XGMAC_DMA_TX_POLL);
+	if (dma_ring_space(priv->tx_head, priv->tx_tail, DMA_TX_RING_SZ) <
+	    MAX_SKB_FRAGS)
+		netif_stop_queue(dev);
 
 	return NETDEV_TX_OK;
 }
@@ -1397,7 +1397,7 @@ static irqreturn_t xgmac_interrupt(int irq, void *dev_id)
 	}
 
 	/* TX/RX NORMAL interrupts */
-	if (intr_status & (DMA_STATUS_RI | DMA_STATUS_TU)) {
+	if (intr_status & (DMA_STATUS_RI | DMA_STATUS_TU | DMA_STATUS_TI)) {
 		__raw_writel(DMA_INTR_ABNORMAL, priv->base + XGMAC_DMA_INTR_ENA);
 		napi_schedule(&priv->napi);
 	}
-- 
1.7.10.4

[PATCH v3 6/6] net: calxedaxgmac: ip align receive buffers

[Rob Herring]

From: Rob Herring <rob.herring@calxeda.com>

On gcc 4.7, we will get alignment traps in the ip stack if we don't align
the ip headers on receive. The h/w can support this, so use ip aligned
allocations.

Cut down the unnecessary padding on the allocation. The buffer can start on
any byte alignment, but the size including the begining offset must be 8
byte aligned. So the h/w buffer size must include the NET_IP_ALIGN offset.

Thanks to Eric Dumazet for the initial patch highlighting the padding issues.

Signed-off-by: Rob Herring <rob.herring@calxeda.com>
---
 drivers/net/ethernet/calxeda/xgmac.c |   11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/drivers/net/ethernet/calxeda/xgmac.c b/drivers/net/ethernet/calxeda/xgmac.c
index 362b35e..b407043 100644
--- a/drivers/net/ethernet/calxeda/xgmac.c
+++ b/drivers/net/ethernet/calxeda/xgmac.c
@@ -665,6 +665,7 @@ static void xgmac_rx_refill(struct xgmac_priv *priv)
 {
 	struct xgmac_dma_desc *p;
 	dma_addr_t paddr;
+	int bufsz = priv->dev->mtu + ETH_HLEN + ETH_FCS_LEN;
 
 	while (dma_ring_space(priv->rx_head, priv->rx_tail, DMA_RX_RING_SZ) > 1) {
 		int entry = priv->rx_head;
@@ -673,13 +674,13 @@ static void xgmac_rx_refill(struct xgmac_priv *priv)
 		p = priv->dma_rx + entry;
 
 		if (priv->rx_skbuff[entry] == NULL) {
-			skb = netdev_alloc_skb(priv->dev, priv->dma_buf_sz);
+			skb = netdev_alloc_skb_ip_align(priv->dev, bufsz);
 			if (unlikely(skb == NULL))
 				break;
 
 			priv->rx_skbuff[entry] = skb;
 			paddr = dma_map_single(priv->device, skb->data,
-					       priv->dma_buf_sz, DMA_FROM_DEVICE);
+					       bufsz, DMA_FROM_DEVICE);
 			desc_set_buf_addr(p, paddr, priv->dma_buf_sz);
 		}
 
@@ -703,10 +704,10 @@ static int xgmac_dma_desc_rings_init(struct net_device *dev)
 	unsigned int bfsize;
 
 	/* Set the Buffer size according to the MTU;
-	 * indeed, in case of jumbo we need to bump-up the buffer sizes.
+	 * The total buffer size including any IP offset must be a multiple
+	 * of 8 bytes.
 	 */
-	bfsize = ALIGN(dev->mtu + ETH_HLEN + ETH_FCS_LEN + NET_IP_ALIGN + 64,
-		       64);
+	bfsize = ALIGN(dev->mtu + ETH_HLEN + ETH_FCS_LEN + NET_IP_ALIGN, 8);
 
 	netdev_dbg(priv->dev, "mtu [%d] bfsize [%d]\n", dev->mtu, bfsize);
 
-- 
1.7.10.4

Re: [PATCH v3 6/6] net: calxedaxgmac: ip align receive buffers

[Eric Dumazet]

On Mon, 2012-11-05 at 10:22 -0600, Rob Herring wrote:
> From: Rob Herring <rob.herring@calxeda.com>
> 
> On gcc 4.7, we will get alignment traps in the ip stack if we don't align
> the ip headers on receive. The h/w can support this, so use ip aligned
> allocations.
> 
> Cut down the unnecessary padding on the allocation. The buffer can start on
> any byte alignment, but the size including the begining offset must be 8
> byte aligned. So the h/w buffer size must include the NET_IP_ALIGN offset.
> 
> Thanks to Eric Dumazet for the initial patch highlighting the padding issues.
> 
> Signed-off-by: Rob Herring <rob.herring@calxeda.com>
> ---

Acked-by: Eric Dumazet <edumazet@google.com>

Re: [PATCH v3 5/6] net: calxedaxgmac: rework transmit ring handling

[David Miller]

From: Rob Herring <robherring2@gmail.com>
Date: Mon,  5 Nov 2012 10:22:23 -0600

> Only generate tx interrupts on every ring size / 4 descriptors.

I thought we told you that you cannot do this.

With this change if we get a few packets, then stop generating any
traffic, there will be SKBs that just sit dead in your TX queue.

This cannot ever happen.  All TX SKBs must be freed up in a short,
finite, amount of time.  Under all conditions, and in every situation.

Otherwise memory accounted to sockets is not liberated, and such
sockets cannot be destroyed or closed.

SKBs also hold onto other kinds of resources, for which it is critical
to liberate in a finite amount of time.

I'm not applying this series, it still needs more work.

Re: [PATCH v3 5/6] net: calxedaxgmac: rework transmit ring handling

[Rob Herring]

David,

On 11/06/2012 05:57 PM, David Miller wrote:
> From: Rob Herring <robherring2@gmail.com>
> Date: Mon,  5 Nov 2012 10:22:23 -0600
> 
>> Only generate tx interrupts on every ring size / 4 descriptors.
> 
> I thought we told you that you cannot do this.
> 
> With this change if we get a few packets, then stop generating any
> traffic, there will be SKBs that just sit dead in your TX queue.

And as I previously mentioned, we do get a tx complete interrupt in
addition. The h/w will interrupt when all packets are transmitted and
there is not another descriptor ready. That is the only tx interrupt we
get without this patch. With this patch, we will get interrupts for
every N descriptors in addition to a tx complete/idle interrupt. This
patch is to avoid the transmitter from going idle and only refilling the
tx ring after finishing sending all frames. I can repost this patch and
make the commit message more clear, but I don't think there is any
functional change needed. This one is not so important compared to the
rest of the series, so you can just drop it if you still don't agree.

Rob

> 
> This cannot ever happen.  All TX SKBs must be freed up in a short,
> finite, amount of time.  Under all conditions, and in every situation.
> 
> Otherwise memory accounted to sockets is not liberated, and such
> sockets cannot be destroyed or closed.
> 
> SKBs also hold onto other kinds of resources, for which it is critical
> to liberate in a finite amount of time.
> 
> I'm not applying this series, it still needs more work.
> 

Re: [PATCH v3 5/6] net: calxedaxgmac: rework transmit ring handling

[David Miller]

From: Rob Herring <rob.herring@calxeda.com>
Date: Tue, 06 Nov 2012 18:29:11 -0600

> David,
> 
> On 11/06/2012 05:57 PM, David Miller wrote:
>> From: Rob Herring <robherring2@gmail.com>
>> Date: Mon,  5 Nov 2012 10:22:23 -0600
>> 
>>> Only generate tx interrupts on every ring size / 4 descriptors.
>> 
>> I thought we told you that you cannot do this.
>> 
>> With this change if we get a few packets, then stop generating any
>> traffic, there will be SKBs that just sit dead in your TX queue.
> 
> And as I previously mentioned, we do get a tx complete interrupt in
> addition. The h/w will interrupt when all packets are transmitted and
> there is not another descriptor ready.

Ok, in that case it's fine.  I'll keep reviewing this series then.

Re: [PATCH v3 0/6] Calxeda xgmac performance fixes

[David Miller]

From: Rob Herring <robherring2@gmail.com>
Date: Mon,  5 Nov 2012 10:22:18 -0600

> This is a series of performance improvements to the xgmac driver. The
> most significant changes are the alignment fixes to avoid alignment
> traps on received frames and using raw i/o accessors.

All applied to net-next, thanks.