[PATCH net-next-2.6 00/10] sfc changes for 2.6.37

[PATCH net-next-2.6 00/10] sfc changes for 2.6.37

[Ben Hutchings]

One new feature - variable ring sizes - and a few bug fixes.

Ben.

Ben Hutchings (9):
  sfc: Use MCDI RX_BAD_FCS_PKTS count as MAC rx_bad count
  sfc: Accumulate RX_NODESC_DROP count in rx_dropped, not
    rx_over_errors
  sfc: Remove declarations of functions that no longer exist
  sfc: Fix failure paths in efx_probe_port()
  sfc: Allocate DMA and event rings using GFP_KERNEL
  sfc: Abstract channel and index lookup for RX queues
  sfc: Refactor channel and queue lookup and iteration
  sfc: Allocate each channel separately, along with its RX and TX
    queues
  sfc: Allow changing the DMA ring sizes dynamically via ethtool

Steve Hodgson (1):
  sfc: Make the dmaq size a run-time setting (rather than compile-time)

 drivers/net/sfc/efx.c        |  302 +++++++++++++++++++++++++++++++-----------
 drivers/net/sfc/efx.h        |   22 ++-
 drivers/net/sfc/ethtool.c    |   53 +++++++-
 drivers/net/sfc/falcon.c     |   14 +--
 drivers/net/sfc/net_driver.h |  103 ++++++++++-----
 drivers/net/sfc/nic.c        |  182 ++++++++++++++-----------
 drivers/net/sfc/rx.c         |   71 ++++++-----
 drivers/net/sfc/selftest.c   |    7 +-
 drivers/net/sfc/siena.c      |    2 +-
 drivers/net/sfc/tx.c         |   78 ++++++-----
 10 files changed, 543 insertions(+), 291 deletions(-)

-- 
1.7.2.1


-- 
Ben Hutchings, Senior Software Engineer, Solarflare Communications
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.

[PATCH net-next-2.6 01/10] sfc: Use MCDI RX_BAD_FCS_PKTS count as MAC rx_bad count

[Ben Hutchings]

Calculating rx_bad as rx_packets - rx_good is unnecessary and
incorrect, since rx_good does not include control frames (e.g.
pause frames) and rx_packets does.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
---
 drivers/net/sfc/siena.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/drivers/net/sfc/siena.c b/drivers/net/sfc/siena.c
index 3fab030..9f53680 100644
--- a/drivers/net/sfc/siena.c
+++ b/drivers/net/sfc/siena.c
@@ -450,7 +450,7 @@ static int siena_try_update_nic_stats(struct efx_nic *efx)
 				    mac_stats->rx_bad_bytes);
 	MAC_STAT(rx_packets, RX_PKTS);
 	MAC_STAT(rx_good, RX_GOOD_PKTS);
-	mac_stats->rx_bad = mac_stats->rx_packets - mac_stats->rx_good;
+	MAC_STAT(rx_bad, RX_BAD_FCS_PKTS);
 	MAC_STAT(rx_pause, RX_PAUSE_PKTS);
 	MAC_STAT(rx_control, RX_CONTROL_PKTS);
 	MAC_STAT(rx_unicast, RX_UNICAST_PKTS);
-- 
1.7.2.1



-- 
Ben Hutchings, Senior Software Engineer, Solarflare Communications
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.

[PATCH net-next-2.6 02/10] sfc: Accumulate RX_NODESC_DROP count in rx_dropped, not rx_over_errors

[Ben Hutchings]

rx_over_errors appears to be intended as a count of packets that
overflow a packet buffer in the NIC.  Given that we implement a
cut-through receive path, this should always be 0.

rx_dropped appears to be the correct counter for packets dropped due
to lack of host buffers.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
---
 drivers/net/sfc/efx.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/drivers/net/sfc/efx.c b/drivers/net/sfc/efx.c
index ba674c5..b385f89 100644
--- a/drivers/net/sfc/efx.c
+++ b/drivers/net/sfc/efx.c
@@ -1546,11 +1546,11 @@ static struct rtnl_link_stats64 *efx_net_stats(struct net_device *net_dev, struc
 	stats->tx_packets = mac_stats->tx_packets;
 	stats->rx_bytes = mac_stats->rx_bytes;
 	stats->tx_bytes = mac_stats->tx_bytes;
+	stats->rx_dropped = efx->n_rx_nodesc_drop_cnt;
 	stats->multicast = mac_stats->rx_multicast;
 	stats->collisions = mac_stats->tx_collision;
 	stats->rx_length_errors = (mac_stats->rx_gtjumbo +
 				   mac_stats->rx_length_error);
-	stats->rx_over_errors = efx->n_rx_nodesc_drop_cnt;
 	stats->rx_crc_errors = mac_stats->rx_bad;
 	stats->rx_frame_errors = mac_stats->rx_align_error;
 	stats->rx_fifo_errors = mac_stats->rx_overflow;
-- 
1.7.2.1



-- 
Ben Hutchings, Senior Software Engineer, Solarflare Communications
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.

[PATCH net-next-2.6 03/10] sfc: Remove declarations of functions that no longer exist

[Ben Hutchings]

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
---
 drivers/net/sfc/efx.h |    2 --
 1 files changed, 0 insertions(+), 2 deletions(-)

diff --git a/drivers/net/sfc/efx.h b/drivers/net/sfc/efx.h
index 060dc95..d6b172b 100644
--- a/drivers/net/sfc/efx.h
+++ b/drivers/net/sfc/efx.h
@@ -81,8 +81,6 @@ extern int efx_reset_up(struct efx_nic *efx, enum reset_type method, bool ok);
 extern void efx_schedule_reset(struct efx_nic *efx, enum reset_type type);
 extern void efx_init_irq_moderation(struct efx_nic *efx, int tx_usecs,
 				    int rx_usecs, bool rx_adaptive);
-extern int efx_request_power(struct efx_nic *efx, int mw, const char *name);
-extern void efx_hex_dump(const u8 *, unsigned int, const char *);
 
 /* Dummy PHY ops for PHY drivers */
 extern int efx_port_dummy_op_int(struct efx_nic *efx);
-- 
1.7.2.1



-- 
Ben Hutchings, Senior Software Engineer, Solarflare Communications
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.

[PATCH net-next-2.6 04/10] sfc: Fix failure paths in efx_probe_port()

[Ben Hutchings]

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
---
 drivers/net/sfc/efx.c |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/net/sfc/efx.c b/drivers/net/sfc/efx.c
index b385f89..a576045 100644
--- a/drivers/net/sfc/efx.c
+++ b/drivers/net/sfc/efx.c
@@ -761,7 +761,7 @@ static int efx_probe_port(struct efx_nic *efx)
 	/* Connect up MAC/PHY operations table */
 	rc = efx->type->probe_port(efx);
 	if (rc)
-		goto err;
+		return rc;
 
 	/* Sanity check MAC address */
 	if (is_valid_ether_addr(efx->mac_address)) {
@@ -782,7 +782,7 @@ static int efx_probe_port(struct efx_nic *efx)
 	return 0;
 
  err:
-	efx_remove_port(efx);
+	efx->type->remove_port(efx);
 	return rc;
 }
 
-- 
1.7.2.1



-- 
Ben Hutchings, Senior Software Engineer, Solarflare Communications
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.

[PATCH net-next-2.6 05/10] sfc: Allocate DMA and event rings using GFP_KERNEL

[Ben Hutchings]

Currently we allocate DMA descriptor rings and event rings using
pci_alloc_consistent() which selects non-blocking behaviour from the
page allocator (GFP_ATOMIC). This is unnecessary, and since we
currently allocate a single contiguous block for each ring (up to 32
pages!) these allocations are likely to fail if there is any
significant memory pressure.  Use dma_alloc_coherent() and GFP_KERNEL
instead.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
---
 drivers/net/sfc/nic.c |    8 ++++----
 1 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/net/sfc/nic.c b/drivers/net/sfc/nic.c
index f595d92..8efe1ca 100644
--- a/drivers/net/sfc/nic.c
+++ b/drivers/net/sfc/nic.c
@@ -263,8 +263,8 @@ static int efx_alloc_special_buffer(struct efx_nic *efx,
 {
 	len = ALIGN(len, EFX_BUF_SIZE);
 
-	buffer->addr = pci_alloc_consistent(efx->pci_dev, len,
-					    &buffer->dma_addr);
+	buffer->addr = dma_alloc_coherent(&efx->pci_dev->dev, len,
+					  &buffer->dma_addr, GFP_KERNEL);
 	if (!buffer->addr)
 		return -ENOMEM;
 	buffer->len = len;
@@ -301,8 +301,8 @@ efx_free_special_buffer(struct efx_nic *efx, struct efx_special_buffer *buffer)
 		  (u64)buffer->dma_addr, buffer->len,
 		  buffer->addr, (u64)virt_to_phys(buffer->addr));
 
-	pci_free_consistent(efx->pci_dev, buffer->len, buffer->addr,
-			    buffer->dma_addr);
+	dma_free_coherent(&efx->pci_dev->dev, buffer->len, buffer->addr,
+			  buffer->dma_addr);
 	buffer->addr = NULL;
 	buffer->entries = 0;
 }
-- 
1.7.2.1



-- 
Ben Hutchings, Senior Software Engineer, Solarflare Communications
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.

[PATCH net-next-2.6 06/10] sfc: Abstract channel and index lookup for RX queues

[Ben Hutchings]

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
---
 drivers/net/sfc/net_driver.h |   11 +++++++++++
 drivers/net/sfc/nic.c        |   31 +++++++++++++++++--------------
 drivers/net/sfc/rx.c         |   38 +++++++++++++++++++-------------------
 3 files changed, 47 insertions(+), 33 deletions(-)

diff --git a/drivers/net/sfc/net_driver.h b/drivers/net/sfc/net_driver.h
index 64e7caa..89c6e02 100644
--- a/drivers/net/sfc/net_driver.h
+++ b/drivers/net/sfc/net_driver.h
@@ -943,6 +943,17 @@ struct efx_nic_type {
 			continue;					\
 		else
 
+static inline struct efx_channel *
+efx_rx_queue_channel(struct efx_rx_queue *rx_queue)
+{
+	return rx_queue->channel;
+}
+
+static inline int efx_rx_queue_index(struct efx_rx_queue *rx_queue)
+{
+	return rx_queue->queue;
+}
+
 /* Returns a pointer to the specified receive buffer in the RX
  * descriptor queue.
  */
diff --git a/drivers/net/sfc/nic.c b/drivers/net/sfc/nic.c
index 8efe1ca..be4d552 100644
--- a/drivers/net/sfc/nic.c
+++ b/drivers/net/sfc/nic.c
@@ -539,8 +539,8 @@ void efx_nic_notify_rx_desc(struct efx_rx_queue *rx_queue)
 	wmb();
 	write_ptr = rx_queue->added_count & EFX_RXQ_MASK;
 	EFX_POPULATE_DWORD_1(reg, FRF_AZ_RX_DESC_WPTR_DWORD, write_ptr);
-	efx_writed_page(rx_queue->efx, &reg,
-			FR_AZ_RX_DESC_UPD_DWORD_P0, rx_queue->queue);
+	efx_writed_page(rx_queue->efx, &reg, FR_AZ_RX_DESC_UPD_DWORD_P0,
+			efx_rx_queue_index(rx_queue));
 }
 
 int efx_nic_probe_rx(struct efx_rx_queue *rx_queue)
@@ -561,7 +561,7 @@ void efx_nic_init_rx(struct efx_rx_queue *rx_queue)
 
 	netif_dbg(efx, hw, efx->net_dev,
 		  "RX queue %d ring in special buffers %d-%d\n",
-		  rx_queue->queue, rx_queue->rxd.index,
+		  efx_rx_queue_index(rx_queue), rx_queue->rxd.index,
 		  rx_queue->rxd.index + rx_queue->rxd.entries - 1);
 
 	rx_queue->flushed = FLUSH_NONE;
@@ -575,9 +575,10 @@ void efx_nic_init_rx(struct efx_rx_queue *rx_queue)
 			      FRF_AZ_RX_ISCSI_HDIG_EN, iscsi_digest_en,
 			      FRF_AZ_RX_DESCQ_BUF_BASE_ID, rx_queue->rxd.index,
 			      FRF_AZ_RX_DESCQ_EVQ_ID,
-			      rx_queue->channel->channel,
+			      efx_rx_queue_channel(rx_queue)->channel,
 			      FRF_AZ_RX_DESCQ_OWNER_ID, 0,
-			      FRF_AZ_RX_DESCQ_LABEL, rx_queue->queue,
+			      FRF_AZ_RX_DESCQ_LABEL,
+			      efx_rx_queue_index(rx_queue),
 			      FRF_AZ_RX_DESCQ_SIZE,
 			      __ffs(rx_queue->rxd.entries),
 			      FRF_AZ_RX_DESCQ_TYPE, 0 /* kernel queue */ ,
@@ -585,7 +586,7 @@ void efx_nic_init_rx(struct efx_rx_queue *rx_queue)
 			      FRF_AZ_RX_DESCQ_JUMBO, !is_b0,
 			      FRF_AZ_RX_DESCQ_EN, 1);
 	efx_writeo_table(efx, &rx_desc_ptr, efx->type->rxd_ptr_tbl_base,
-			 rx_queue->queue);
+			 efx_rx_queue_index(rx_queue));
 }
 
 static void efx_flush_rx_queue(struct efx_rx_queue *rx_queue)
@@ -598,7 +599,8 @@ static void efx_flush_rx_queue(struct efx_rx_queue *rx_queue)
 	/* Post a flush command */
 	EFX_POPULATE_OWORD_2(rx_flush_descq,
 			     FRF_AZ_RX_FLUSH_DESCQ_CMD, 1,
-			     FRF_AZ_RX_FLUSH_DESCQ, rx_queue->queue);
+			     FRF_AZ_RX_FLUSH_DESCQ,
+			     efx_rx_queue_index(rx_queue));
 	efx_writeo(efx, &rx_flush_descq, FR_AZ_RX_FLUSH_DESCQ);
 }
 
@@ -613,7 +615,7 @@ void efx_nic_fini_rx(struct efx_rx_queue *rx_queue)
 	/* Remove RX descriptor ring from card */
 	EFX_ZERO_OWORD(rx_desc_ptr);
 	efx_writeo_table(efx, &rx_desc_ptr, efx->type->rxd_ptr_tbl_base,
-			 rx_queue->queue);
+			 efx_rx_queue_index(rx_queue));
 
 	/* Unpin RX descriptor ring */
 	efx_fini_special_buffer(efx, &rx_queue->rxd);
@@ -714,6 +716,7 @@ static void efx_handle_rx_not_ok(struct efx_rx_queue *rx_queue,
 				 bool *rx_ev_pkt_ok,
 				 bool *discard)
 {
+	struct efx_channel *channel = efx_rx_queue_channel(rx_queue);
 	struct efx_nic *efx = rx_queue->efx;
 	bool rx_ev_buf_owner_id_err, rx_ev_ip_hdr_chksum_err;
 	bool rx_ev_tcp_udp_chksum_err, rx_ev_eth_crc_err;
@@ -746,14 +749,14 @@ static void efx_handle_rx_not_ok(struct efx_rx_queue *rx_queue,
 	/* Count errors that are not in MAC stats.  Ignore expected
 	 * checksum errors during self-test. */
 	if (rx_ev_frm_trunc)
-		++rx_queue->channel->n_rx_frm_trunc;
+		++channel->n_rx_frm_trunc;
 	else if (rx_ev_tobe_disc)
-		++rx_queue->channel->n_rx_tobe_disc;
+		++channel->n_rx_tobe_disc;
 	else if (!efx->loopback_selftest) {
 		if (rx_ev_ip_hdr_chksum_err)
-			++rx_queue->channel->n_rx_ip_hdr_chksum_err;
+			++channel->n_rx_ip_hdr_chksum_err;
 		else if (rx_ev_tcp_udp_chksum_err)
-			++rx_queue->channel->n_rx_tcp_udp_chksum_err;
+			++channel->n_rx_tcp_udp_chksum_err;
 	}
 
 	/* The frame must be discarded if any of these are true. */
@@ -769,7 +772,7 @@ static void efx_handle_rx_not_ok(struct efx_rx_queue *rx_queue,
 		netif_dbg(efx, rx_err, efx->net_dev,
 			  " RX queue %d unexpected RX event "
 			  EFX_QWORD_FMT "%s%s%s%s%s%s%s%s\n",
-			  rx_queue->queue, EFX_QWORD_VAL(*event),
+			  efx_rx_queue_index(rx_queue), EFX_QWORD_VAL(*event),
 			  rx_ev_buf_owner_id_err ? " [OWNER_ID_ERR]" : "",
 			  rx_ev_ip_hdr_chksum_err ?
 			  " [IP_HDR_CHKSUM_ERR]" : "",
@@ -1269,7 +1272,7 @@ int efx_nic_flush_queues(struct efx_nic *efx)
 		if (rx_queue->flushed != FLUSH_DONE)
 			netif_err(efx, hw, efx->net_dev,
 				  "rx queue %d flush command timed out\n",
-				  rx_queue->queue);
+				  efx_rx_queue_index(rx_queue));
 		rx_queue->flushed = FLUSH_DONE;
 	}
 
diff --git a/drivers/net/sfc/rx.c b/drivers/net/sfc/rx.c
index acb372e..1e6c8cf 100644
--- a/drivers/net/sfc/rx.c
+++ b/drivers/net/sfc/rx.c
@@ -341,7 +341,7 @@ static void efx_recycle_rx_buffer(struct efx_channel *channel,
  */
 void efx_fast_push_rx_descriptors(struct efx_rx_queue *rx_queue)
 {
-	struct efx_channel *channel = rx_queue->channel;
+	struct efx_channel *channel = efx_rx_queue_channel(rx_queue);
 	unsigned fill_level;
 	int space, rc = 0;
 
@@ -364,7 +364,8 @@ void efx_fast_push_rx_descriptors(struct efx_rx_queue *rx_queue)
 	netif_vdbg(rx_queue->efx, rx_status, rx_queue->efx->net_dev,
 		   "RX queue %d fast-filling descriptor ring from"
 		   " level %d to level %d using %s allocation\n",
-		   rx_queue->queue, fill_level, rx_queue->fast_fill_limit,
+		   efx_rx_queue_index(rx_queue), fill_level,
+		   rx_queue->fast_fill_limit,
 		   channel->rx_alloc_push_pages ? "page" : "skb");
 
 	do {
@@ -382,7 +383,7 @@ void efx_fast_push_rx_descriptors(struct efx_rx_queue *rx_queue)
 
 	netif_vdbg(rx_queue->efx, rx_status, rx_queue->efx->net_dev,
 		   "RX queue %d fast-filled descriptor ring "
-		   "to level %d\n", rx_queue->queue,
+		   "to level %d\n", efx_rx_queue_index(rx_queue),
 		   rx_queue->added_count - rx_queue->removed_count);
 
  out:
@@ -393,7 +394,7 @@ void efx_fast_push_rx_descriptors(struct efx_rx_queue *rx_queue)
 void efx_rx_slow_fill(unsigned long context)
 {
 	struct efx_rx_queue *rx_queue = (struct efx_rx_queue *)context;
-	struct efx_channel *channel = rx_queue->channel;
+	struct efx_channel *channel = efx_rx_queue_channel(rx_queue);
 
 	/* Post an event to cause NAPI to run and refill the queue */
 	efx_nic_generate_fill_event(channel);
@@ -421,7 +422,7 @@ static void efx_rx_packet__check_len(struct efx_rx_queue *rx_queue,
 			netif_err(efx, rx_err, efx->net_dev,
 				  " RX queue %d seriously overlength "
 				  "RX event (0x%x > 0x%x+0x%x). Leaking\n",
-				  rx_queue->queue, len, max_len,
+				  efx_rx_queue_index(rx_queue), len, max_len,
 				  efx->type->rx_buffer_padding);
 		/* If this buffer was skb-allocated, then the meta
 		 * data at the end of the skb will be trashed. So
@@ -434,10 +435,10 @@ static void efx_rx_packet__check_len(struct efx_rx_queue *rx_queue,
 			netif_err(efx, rx_err, efx->net_dev,
 				  " RX queue %d overlength RX event "
 				  "(0x%x > 0x%x)\n",
-				  rx_queue->queue, len, max_len);
+				  efx_rx_queue_index(rx_queue), len, max_len);
 	}
 
-	rx_queue->channel->n_rx_overlength++;
+	efx_rx_queue_channel(rx_queue)->n_rx_overlength++;
 }
 
 /* Pass a received packet up through the generic LRO stack
@@ -507,7 +508,7 @@ void efx_rx_packet(struct efx_rx_queue *rx_queue, unsigned int index,
 		   unsigned int len, bool checksummed, bool discard)
 {
 	struct efx_nic *efx = rx_queue->efx;
-	struct efx_channel *channel = rx_queue->channel;
+	struct efx_channel *channel = efx_rx_queue_channel(rx_queue);
 	struct efx_rx_buffer *rx_buf;
 	bool leak_packet = false;
 
@@ -528,7 +529,7 @@ void efx_rx_packet(struct efx_rx_queue *rx_queue, unsigned int index,
 
 	netif_vdbg(efx, rx_status, efx->net_dev,
 		   "RX queue %d received id %x at %llx+%x %s%s\n",
-		   rx_queue->queue, index,
+		   efx_rx_queue_index(rx_queue), index,
 		   (unsigned long long)rx_buf->dma_addr, len,
 		   (checksummed ? " [SUMMED]" : ""),
 		   (discard ? " [DISCARD]" : ""));
@@ -560,12 +561,11 @@ void efx_rx_packet(struct efx_rx_queue *rx_queue, unsigned int index,
 	 */
 	rx_buf->len = len;
 out:
-	if (rx_queue->channel->rx_pkt)
-		__efx_rx_packet(rx_queue->channel,
-				rx_queue->channel->rx_pkt,
-				rx_queue->channel->rx_pkt_csummed);
-	rx_queue->channel->rx_pkt = rx_buf;
-	rx_queue->channel->rx_pkt_csummed = checksummed;
+	if (channel->rx_pkt)
+		__efx_rx_packet(channel,
+				channel->rx_pkt, channel->rx_pkt_csummed);
+	channel->rx_pkt = rx_buf;
+	channel->rx_pkt_csummed = checksummed;
 }
 
 /* Handle a received packet.  Second half: Touches packet payload. */
@@ -654,7 +654,7 @@ int efx_probe_rx_queue(struct efx_rx_queue *rx_queue)
 	int rc;
 
 	netif_dbg(efx, probe, efx->net_dev,
-		  "creating RX queue %d\n", rx_queue->queue);
+		  "creating RX queue %d\n", efx_rx_queue_index(rx_queue));
 
 	/* Allocate RX buffers */
 	rxq_size = EFX_RXQ_SIZE * sizeof(*rx_queue->buffer);
@@ -675,7 +675,7 @@ void efx_init_rx_queue(struct efx_rx_queue *rx_queue)
 	unsigned int max_fill, trigger, limit;
 
 	netif_dbg(rx_queue->efx, drv, rx_queue->efx->net_dev,
-		  "initialising RX queue %d\n", rx_queue->queue);
+		  "initialising RX queue %d\n", efx_rx_queue_index(rx_queue));
 
 	/* Initialise ptr fields */
 	rx_queue->added_count = 0;
@@ -703,7 +703,7 @@ void efx_fini_rx_queue(struct efx_rx_queue *rx_queue)
 	struct efx_rx_buffer *rx_buf;
 
 	netif_dbg(rx_queue->efx, drv, rx_queue->efx->net_dev,
-		  "shutting down RX queue %d\n", rx_queue->queue);
+		  "shutting down RX queue %d\n", efx_rx_queue_index(rx_queue));
 
 	del_timer_sync(&rx_queue->slow_fill);
 	efx_nic_fini_rx(rx_queue);
@@ -720,7 +720,7 @@ void efx_fini_rx_queue(struct efx_rx_queue *rx_queue)
 void efx_remove_rx_queue(struct efx_rx_queue *rx_queue)
 {
 	netif_dbg(rx_queue->efx, drv, rx_queue->efx->net_dev,
-		  "destroying RX queue %d\n", rx_queue->queue);
+		  "destroying RX queue %d\n", efx_rx_queue_index(rx_queue));
 
 	efx_nic_remove_rx(rx_queue);
 
-- 
1.7.2.1



-- 
Ben Hutchings, Senior Software Engineer, Solarflare Communications
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.

[PATCH net-next-2.6 07/10] sfc: Refactor channel and queue lookup and iteration

[Ben Hutchings]

In preparation for changes to the way channels and queue structures
are allocated, revise the macros and functions used to look up and
iterator over them.

- Replace efx_for_each_tx_queue() with iteration over channels then TX
  queues
- Replace efx_for_each_rx_queue() with iteration over channels then RX
  queues (with one exception, shortly to be removed)
- Introduce efx_get_{channel,rx_queue,tx_queue}() functions to look up
  channels and queues by index
- Introduce efx_channel_get_{rx,tx}_queue() functions to look up a
  channel's queues

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
---
 drivers/net/sfc/efx.c        |   28 ++++++++------
 drivers/net/sfc/ethtool.c    |   15 ++++---
 drivers/net/sfc/net_driver.h |   43 ++++++++++++++++++---
 drivers/net/sfc/nic.c        |   85 ++++++++++++++++++++++++------------------
 drivers/net/sfc/rx.c         |    2 +-
 drivers/net/sfc/selftest.c   |    5 +-
 drivers/net/sfc/tx.c         |   22 +++++-----
 7 files changed, 125 insertions(+), 75 deletions(-)

diff --git a/drivers/net/sfc/efx.c b/drivers/net/sfc/efx.c
index a576045..3dd71aa 100644
--- a/drivers/net/sfc/efx.c
+++ b/drivers/net/sfc/efx.c
@@ -248,7 +248,7 @@ static int efx_process_channel(struct efx_channel *channel, int budget)
 
 	efx_rx_strategy(channel);
 
-	efx_fast_push_rx_descriptors(&efx->rx_queue[channel->channel]);
+	efx_fast_push_rx_descriptors(efx_channel_get_rx_queue(channel));
 
 	return spent;
 }
@@ -1050,7 +1050,8 @@ static void efx_probe_interrupts(struct efx_nic *efx)
 				efx->n_rx_channels = efx->n_channels;
 			}
 			for (i = 0; i < n_channels; i++)
-				efx->channel[i].irq = xentries[i].vector;
+				efx_get_channel(efx, i)->irq =
+					xentries[i].vector;
 		} else {
 			/* Fall back to single channel MSI */
 			efx->interrupt_mode = EFX_INT_MODE_MSI;
@@ -1066,7 +1067,7 @@ static void efx_probe_interrupts(struct efx_nic *efx)
 		efx->n_tx_channels = 1;
 		rc = pci_enable_msi(efx->pci_dev);
 		if (rc == 0) {
-			efx->channel[0].irq = efx->pci_dev->irq;
+			efx_get_channel(efx, 0)->irq = efx->pci_dev->irq;
 		} else {
 			netif_err(efx, drv, efx->net_dev,
 				  "could not enable MSI\n");
@@ -1355,20 +1356,20 @@ static unsigned irq_mod_ticks(int usecs, int resolution)
 void efx_init_irq_moderation(struct efx_nic *efx, int tx_usecs, int rx_usecs,
 			     bool rx_adaptive)
 {
-	struct efx_tx_queue *tx_queue;
-	struct efx_rx_queue *rx_queue;
+	struct efx_channel *channel;
 	unsigned tx_ticks = irq_mod_ticks(tx_usecs, EFX_IRQ_MOD_RESOLUTION);
 	unsigned rx_ticks = irq_mod_ticks(rx_usecs, EFX_IRQ_MOD_RESOLUTION);
 
 	EFX_ASSERT_RESET_SERIALISED(efx);
 
-	efx_for_each_tx_queue(tx_queue, efx)
-		tx_queue->channel->irq_moderation = tx_ticks;
-
 	efx->irq_rx_adaptive = rx_adaptive;
 	efx->irq_rx_moderation = rx_ticks;
-	efx_for_each_rx_queue(rx_queue, efx)
-		rx_queue->channel->irq_moderation = rx_ticks;
+	efx_for_each_channel(channel, efx) {
+		if (efx_channel_get_rx_queue(channel))
+			channel->irq_moderation = rx_ticks;
+		else if (efx_channel_get_tx_queue(channel, 0))
+			channel->irq_moderation = tx_ticks;
+	}
 }
 
 /**************************************************************************
@@ -1767,6 +1768,7 @@ fail_registered:
 
 static void efx_unregister_netdev(struct efx_nic *efx)
 {
+	struct efx_channel *channel;
 	struct efx_tx_queue *tx_queue;
 
 	if (!efx->net_dev)
@@ -1777,8 +1779,10 @@ static void efx_unregister_netdev(struct efx_nic *efx)
 	/* Free up any skbs still remaining. This has to happen before
 	 * we try to unregister the netdev as running their destructors
 	 * may be needed to get the device ref. count to 0. */
-	efx_for_each_tx_queue(tx_queue, efx)
-		efx_release_tx_buffers(tx_queue);
+	efx_for_each_channel(channel, efx) {
+		efx_for_each_channel_tx_queue(tx_queue, channel)
+			efx_release_tx_buffers(tx_queue);
+	}
 
 	if (efx_dev_registered(efx)) {
 		strlcpy(efx->name, pci_name(efx->pci_dev), sizeof(efx->name));
diff --git a/drivers/net/sfc/ethtool.c b/drivers/net/sfc/ethtool.c
index fd19d6a..b9291db 100644
--- a/drivers/net/sfc/ethtool.c
+++ b/drivers/net/sfc/ethtool.c
@@ -328,9 +328,10 @@ static int efx_fill_loopback_test(struct efx_nic *efx,
 				  unsigned int test_index,
 				  struct ethtool_string *strings, u64 *data)
 {
+	struct efx_channel *channel = efx_get_channel(efx, 0);
 	struct efx_tx_queue *tx_queue;
 
-	efx_for_each_channel_tx_queue(tx_queue, &efx->channel[0]) {
+	efx_for_each_channel_tx_queue(tx_queue, channel) {
 		efx_fill_test(test_index++, strings, data,
 			      &lb_tests->tx_sent[tx_queue->queue],
 			      EFX_TX_QUEUE_NAME(tx_queue),
@@ -673,15 +674,15 @@ static int efx_ethtool_get_coalesce(struct net_device *net_dev,
 				    struct ethtool_coalesce *coalesce)
 {
 	struct efx_nic *efx = netdev_priv(net_dev);
-	struct efx_tx_queue *tx_queue;
 	struct efx_channel *channel;
 
 	memset(coalesce, 0, sizeof(*coalesce));
 
 	/* Find lowest IRQ moderation across all used TX queues */
 	coalesce->tx_coalesce_usecs_irq = ~((u32) 0);
-	efx_for_each_tx_queue(tx_queue, efx) {
-		channel = tx_queue->channel;
+	efx_for_each_channel(channel, efx) {
+		if (!efx_channel_get_tx_queue(channel, 0))
+			continue;
 		if (channel->irq_moderation < coalesce->tx_coalesce_usecs_irq) {
 			if (channel->channel < efx->n_rx_channels)
 				coalesce->tx_coalesce_usecs_irq =
@@ -708,7 +709,6 @@ static int efx_ethtool_set_coalesce(struct net_device *net_dev,
 {
 	struct efx_nic *efx = netdev_priv(net_dev);
 	struct efx_channel *channel;
-	struct efx_tx_queue *tx_queue;
 	unsigned tx_usecs, rx_usecs, adaptive;
 
 	if (coalesce->use_adaptive_tx_coalesce)
@@ -725,8 +725,9 @@ static int efx_ethtool_set_coalesce(struct net_device *net_dev,
 	adaptive = coalesce->use_adaptive_rx_coalesce;
 
 	/* If the channel is shared only allow RX parameters to be set */
-	efx_for_each_tx_queue(tx_queue, efx) {
-		if ((tx_queue->channel->channel < efx->n_rx_channels) &&
+	efx_for_each_channel(channel, efx) {
+		if (efx_channel_get_rx_queue(channel) &&
+		    efx_channel_get_tx_queue(channel, 0) &&
 		    tx_usecs) {
 			netif_err(efx, drv, efx->net_dev, "Channel is shared. "
 				  "Only RX coalescing may be set\n");
diff --git a/drivers/net/sfc/net_driver.h b/drivers/net/sfc/net_driver.h
index 89c6e02..eb35375 100644
--- a/drivers/net/sfc/net_driver.h
+++ b/drivers/net/sfc/net_driver.h
@@ -909,18 +909,34 @@ struct efx_nic_type {
  *
  *************************************************************************/
 
+static inline struct efx_channel *
+efx_get_channel(struct efx_nic *efx, unsigned index)
+{
+	EFX_BUG_ON_PARANOID(index >= efx->n_channels);
+	return &efx->channel[index];
+}
+
 /* Iterate over all used channels */
 #define efx_for_each_channel(_channel, _efx)				\
 	for (_channel = &((_efx)->channel[0]);				\
 	     _channel < &((_efx)->channel[(efx)->n_channels]);		\
 	     _channel++)
 
-/* Iterate over all used TX queues */
-#define efx_for_each_tx_queue(_tx_queue, _efx)				\
-	for (_tx_queue = &((_efx)->tx_queue[0]);			\
-	     _tx_queue < &((_efx)->tx_queue[EFX_TXQ_TYPES *		\
-					    (_efx)->n_tx_channels]);	\
-	     _tx_queue++)
+static inline struct efx_tx_queue *
+efx_get_tx_queue(struct efx_nic *efx, unsigned index, unsigned type)
+{
+	EFX_BUG_ON_PARANOID(index >= efx->n_tx_channels ||
+			    type >= EFX_TXQ_TYPES);
+	return &efx->tx_queue[index * EFX_TXQ_TYPES + type];
+}
+
+static inline struct efx_tx_queue *
+efx_channel_get_tx_queue(struct efx_channel *channel, unsigned type)
+{
+	struct efx_tx_queue *tx_queue = channel->tx_queue;
+	EFX_BUG_ON_PARANOID(type >= EFX_TXQ_TYPES);
+	return tx_queue ? tx_queue + type : NULL;
+}
 
 /* Iterate over all TX queues belonging to a channel */
 #define efx_for_each_channel_tx_queue(_tx_queue, _channel)		\
@@ -928,12 +944,27 @@ struct efx_nic_type {
 	     _tx_queue && _tx_queue < (_channel)->tx_queue + EFX_TXQ_TYPES; \
 	     _tx_queue++)
 
+static inline struct efx_rx_queue *
+efx_get_rx_queue(struct efx_nic *efx, unsigned index)
+{
+	EFX_BUG_ON_PARANOID(index >= efx->n_rx_channels);
+	return &efx->rx_queue[index];
+}
+
 /* Iterate over all used RX queues */
 #define efx_for_each_rx_queue(_rx_queue, _efx)				\
 	for (_rx_queue = &((_efx)->rx_queue[0]);			\
 	     _rx_queue < &((_efx)->rx_queue[(_efx)->n_rx_channels]);	\
 	     _rx_queue++)
 
+static inline struct efx_rx_queue *
+efx_channel_get_rx_queue(struct efx_channel *channel)
+{
+	struct efx_rx_queue *rx_queue =
+		&channel->efx->rx_queue[channel->channel];
+	return rx_queue->channel == channel ? rx_queue : NULL;
+}
+
 /* Iterate over all RX queues belonging to a channel */
 #define efx_for_each_channel_rx_queue(_rx_queue, _channel)		\
 	for (_rx_queue = &((_channel)->efx->rx_queue[(_channel)->channel]); \
diff --git a/drivers/net/sfc/nic.c b/drivers/net/sfc/nic.c
index be4d552..9e35633 100644
--- a/drivers/net/sfc/nic.c
+++ b/drivers/net/sfc/nic.c
@@ -682,7 +682,8 @@ efx_handle_tx_event(struct efx_channel *channel, efx_qword_t *event)
 		/* Transmit completion */
 		tx_ev_desc_ptr = EFX_QWORD_FIELD(*event, FSF_AZ_TX_EV_DESC_PTR);
 		tx_ev_q_label = EFX_QWORD_FIELD(*event, FSF_AZ_TX_EV_Q_LABEL);
-		tx_queue = &efx->tx_queue[tx_ev_q_label];
+		tx_queue = efx_channel_get_tx_queue(
+			channel, tx_ev_q_label % EFX_TXQ_TYPES);
 		tx_packets = ((tx_ev_desc_ptr - tx_queue->read_count) &
 			      EFX_TXQ_MASK);
 		channel->irq_mod_score += tx_packets;
@@ -690,7 +691,8 @@ efx_handle_tx_event(struct efx_channel *channel, efx_qword_t *event)
 	} else if (EFX_QWORD_FIELD(*event, FSF_AZ_TX_EV_WQ_FF_FULL)) {
 		/* Rewrite the FIFO write pointer */
 		tx_ev_q_label = EFX_QWORD_FIELD(*event, FSF_AZ_TX_EV_Q_LABEL);
-		tx_queue = &efx->tx_queue[tx_ev_q_label];
+		tx_queue = efx_channel_get_tx_queue(
+			channel, tx_ev_q_label % EFX_TXQ_TYPES);
 
 		if (efx_dev_registered(efx))
 			netif_tx_lock(efx->net_dev);
@@ -830,7 +832,7 @@ efx_handle_rx_event(struct efx_channel *channel, const efx_qword_t *event)
 	WARN_ON(EFX_QWORD_FIELD(*event, FSF_AZ_RX_EV_Q_LABEL) !=
 		channel->channel);
 
-	rx_queue = &efx->rx_queue[channel->channel];
+	rx_queue = efx_channel_get_rx_queue(channel);
 
 	rx_ev_desc_ptr = EFX_QWORD_FIELD(*event, FSF_AZ_RX_EV_DESC_PTR);
 	expected_ptr = rx_queue->removed_count & EFX_RXQ_MASK;
@@ -882,7 +884,7 @@ efx_handle_generated_event(struct efx_channel *channel, efx_qword_t *event)
 		/* The queue must be empty, so we won't receive any rx
 		 * events, so efx_process_channel() won't refill the
 		 * queue. Refill it here */
-		efx_fast_push_rx_descriptors(&efx->rx_queue[channel->channel]);
+		efx_fast_push_rx_descriptors(efx_channel_get_rx_queue(channel));
 	else
 		netif_dbg(efx, hw, efx->net_dev, "channel %d received "
 			  "generated event "EFX_QWORD_FMT"\n",
@@ -1166,7 +1168,7 @@ void efx_nic_generate_fill_event(struct efx_channel *channel)
 
 static void efx_poll_flush_events(struct efx_nic *efx)
 {
-	struct efx_channel *channel = &efx->channel[0];
+	struct efx_channel *channel = efx_get_channel(efx, 0);
 	struct efx_tx_queue *tx_queue;
 	struct efx_rx_queue *rx_queue;
 	unsigned int read_ptr = channel->eventq_read_ptr;
@@ -1188,7 +1190,9 @@ static void efx_poll_flush_events(struct efx_nic *efx)
 			ev_queue = EFX_QWORD_FIELD(*event,
 						   FSF_AZ_DRIVER_EV_SUBDATA);
 			if (ev_queue < EFX_TXQ_TYPES * efx->n_tx_channels) {
-				tx_queue = efx->tx_queue + ev_queue;
+				tx_queue = efx_get_tx_queue(
+					efx, ev_queue / EFX_TXQ_TYPES,
+					ev_queue % EFX_TXQ_TYPES);
 				tx_queue->flushed = FLUSH_DONE;
 			}
 		} else if (ev_code == FSE_AZ_EV_CODE_DRIVER_EV &&
@@ -1198,7 +1202,7 @@ static void efx_poll_flush_events(struct efx_nic *efx)
 			ev_failed = EFX_QWORD_FIELD(
 				*event, FSF_AZ_DRIVER_EV_RX_FLUSH_FAIL);
 			if (ev_queue < efx->n_rx_channels) {
-				rx_queue = efx->rx_queue + ev_queue;
+				rx_queue = efx_get_rx_queue(efx, ev_queue);
 				rx_queue->flushed =
 					ev_failed ? FLUSH_FAILED : FLUSH_DONE;
 			}
@@ -1219,6 +1223,7 @@ static void efx_poll_flush_events(struct efx_nic *efx)
  * serialise them */
 int efx_nic_flush_queues(struct efx_nic *efx)
 {
+	struct efx_channel *channel;
 	struct efx_rx_queue *rx_queue;
 	struct efx_tx_queue *tx_queue;
 	int i, tx_pending, rx_pending;
@@ -1227,29 +1232,35 @@ int efx_nic_flush_queues(struct efx_nic *efx)
 	efx->type->prepare_flush(efx);
 
 	/* Flush all tx queues in parallel */
-	efx_for_each_tx_queue(tx_queue, efx)
-		efx_flush_tx_queue(tx_queue);
+	efx_for_each_channel(channel, efx) {
+		efx_for_each_channel_tx_queue(tx_queue, channel)
+			efx_flush_tx_queue(tx_queue);
+	}
 
 	/* The hardware supports four concurrent rx flushes, each of which may
 	 * need to be retried if there is an outstanding descriptor fetch */
 	for (i = 0; i < EFX_FLUSH_POLL_COUNT; ++i) {
 		rx_pending = tx_pending = 0;
-		efx_for_each_rx_queue(rx_queue, efx) {
-			if (rx_queue->flushed == FLUSH_PENDING)
-				++rx_pending;
-		}
-		efx_for_each_rx_queue(rx_queue, efx) {
-			if (rx_pending == EFX_RX_FLUSH_COUNT)
-				break;
-			if (rx_queue->flushed == FLUSH_FAILED ||
-			    rx_queue->flushed == FLUSH_NONE) {
-				efx_flush_rx_queue(rx_queue);
-				++rx_pending;
+		efx_for_each_channel(channel, efx) {
+			efx_for_each_channel_rx_queue(rx_queue, channel) {
+				if (rx_queue->flushed == FLUSH_PENDING)
+					++rx_pending;
 			}
 		}
-		efx_for_each_tx_queue(tx_queue, efx) {
-			if (tx_queue->flushed != FLUSH_DONE)
-				++tx_pending;
+		efx_for_each_channel(channel, efx) {
+			efx_for_each_channel_rx_queue(rx_queue, channel) {
+				if (rx_pending == EFX_RX_FLUSH_COUNT)
+					break;
+				if (rx_queue->flushed == FLUSH_FAILED ||
+				    rx_queue->flushed == FLUSH_NONE) {
+					efx_flush_rx_queue(rx_queue);
+					++rx_pending;
+				}
+			}
+			efx_for_each_channel_tx_queue(tx_queue, channel) {
+				if (tx_queue->flushed != FLUSH_DONE)
+					++tx_pending;
+			}
 		}
 
 		if (rx_pending == 0 && tx_pending == 0)
@@ -1261,19 +1272,21 @@ int efx_nic_flush_queues(struct efx_nic *efx)
 
 	/* Mark the queues as all flushed. We're going to return failure
 	 * leading to a reset, or fake up success anyway */
-	efx_for_each_tx_queue(tx_queue, efx) {
-		if (tx_queue->flushed != FLUSH_DONE)
-			netif_err(efx, hw, efx->net_dev,
-				  "tx queue %d flush command timed out\n",
-				  tx_queue->queue);
-		tx_queue->flushed = FLUSH_DONE;
-	}
-	efx_for_each_rx_queue(rx_queue, efx) {
-		if (rx_queue->flushed != FLUSH_DONE)
-			netif_err(efx, hw, efx->net_dev,
-				  "rx queue %d flush command timed out\n",
-				  efx_rx_queue_index(rx_queue));
-		rx_queue->flushed = FLUSH_DONE;
+	efx_for_each_channel(channel, efx) {
+		efx_for_each_channel_tx_queue(tx_queue, channel) {
+			if (tx_queue->flushed != FLUSH_DONE)
+				netif_err(efx, hw, efx->net_dev,
+					  "tx queue %d flush command timed out\n",
+					  tx_queue->queue);
+			tx_queue->flushed = FLUSH_DONE;
+		}
+		efx_for_each_channel_rx_queue(rx_queue, channel) {
+			if (rx_queue->flushed != FLUSH_DONE)
+				netif_err(efx, hw, efx->net_dev,
+					  "rx queue %d flush command timed out\n",
+					  efx_rx_queue_index(rx_queue));
+			rx_queue->flushed = FLUSH_DONE;
+		}
 	}
 
 	return -ETIMEDOUT;
diff --git a/drivers/net/sfc/rx.c b/drivers/net/sfc/rx.c
index 1e6c8cf..6651d93 100644
--- a/drivers/net/sfc/rx.c
+++ b/drivers/net/sfc/rx.c
@@ -311,7 +311,7 @@ static void efx_recycle_rx_buffer(struct efx_channel *channel,
 				  struct efx_rx_buffer *rx_buf)
 {
 	struct efx_nic *efx = channel->efx;
-	struct efx_rx_queue *rx_queue = &efx->rx_queue[channel->channel];
+	struct efx_rx_queue *rx_queue = efx_channel_get_rx_queue(channel);
 	struct efx_rx_buffer *new_buf;
 	unsigned index;
 
diff --git a/drivers/net/sfc/selftest.c b/drivers/net/sfc/selftest.c
index 85f015f..11153d9 100644
--- a/drivers/net/sfc/selftest.c
+++ b/drivers/net/sfc/selftest.c
@@ -567,7 +567,7 @@ static int efx_wait_for_link(struct efx_nic *efx)
 			efx->type->monitor(efx);
 			mutex_unlock(&efx->mac_lock);
 		} else {
-			struct efx_channel *channel = &efx->channel[0];
+			struct efx_channel *channel = efx_get_channel(efx, 0);
 			if (channel->work_pending)
 				efx_process_channel_now(channel);
 		}
@@ -594,6 +594,7 @@ static int efx_test_loopbacks(struct efx_nic *efx, struct efx_self_tests *tests,
 {
 	enum efx_loopback_mode mode;
 	struct efx_loopback_state *state;
+	struct efx_channel *channel = efx_get_channel(efx, 0);
 	struct efx_tx_queue *tx_queue;
 	int rc = 0;
 
@@ -634,7 +635,7 @@ static int efx_test_loopbacks(struct efx_nic *efx, struct efx_self_tests *tests,
 		}
 
 		/* Test both types of TX queue */
-		efx_for_each_channel_tx_queue(tx_queue, &efx->channel[0]) {
+		efx_for_each_channel_tx_queue(tx_queue, channel) {
 			state->offload_csum = (tx_queue->queue &
 					       EFX_TXQ_TYPE_OFFLOAD);
 			rc = efx_test_loopback(tx_queue,
diff --git a/drivers/net/sfc/tx.c b/drivers/net/sfc/tx.c
index c6942da..6a6acc4 100644
--- a/drivers/net/sfc/tx.c
+++ b/drivers/net/sfc/tx.c
@@ -37,8 +37,9 @@
 void efx_stop_queue(struct efx_channel *channel)
 {
 	struct efx_nic *efx = channel->efx;
+	struct efx_tx_queue *tx_queue = efx_channel_get_tx_queue(channel, 0);
 
-	if (!channel->tx_queue)
+	if (!tx_queue)
 		return;
 
 	spin_lock_bh(&channel->tx_stop_lock);
@@ -46,9 +47,8 @@ void efx_stop_queue(struct efx_channel *channel)
 
 	atomic_inc(&channel->tx_stop_count);
 	netif_tx_stop_queue(
-		netdev_get_tx_queue(
-			efx->net_dev,
-			channel->tx_queue->queue / EFX_TXQ_TYPES));
+		netdev_get_tx_queue(efx->net_dev,
+				    tx_queue->queue / EFX_TXQ_TYPES));
 
 	spin_unlock_bh(&channel->tx_stop_lock);
 }
@@ -57,8 +57,9 @@ void efx_stop_queue(struct efx_channel *channel)
 void efx_wake_queue(struct efx_channel *channel)
 {
 	struct efx_nic *efx = channel->efx;
+	struct efx_tx_queue *tx_queue = efx_channel_get_tx_queue(channel, 0);
 
-	if (!channel->tx_queue)
+	if (!tx_queue)
 		return;
 
 	local_bh_disable();
@@ -66,9 +67,8 @@ void efx_wake_queue(struct efx_channel *channel)
 				&channel->tx_stop_lock)) {
 		netif_vdbg(efx, tx_queued, efx->net_dev, "waking TX queue\n");
 		netif_tx_wake_queue(
-			netdev_get_tx_queue(
-				efx->net_dev,
-				channel->tx_queue->queue / EFX_TXQ_TYPES));
+			netdev_get_tx_queue(efx->net_dev,
+					    tx_queue->queue / EFX_TXQ_TYPES));
 		spin_unlock(&channel->tx_stop_lock);
 	}
 	local_bh_enable();
@@ -390,9 +390,9 @@ netdev_tx_t efx_hard_start_xmit(struct sk_buff *skb,
 	if (unlikely(efx->port_inhibited))
 		return NETDEV_TX_BUSY;
 
-	tx_queue = &efx->tx_queue[EFX_TXQ_TYPES * skb_get_queue_mapping(skb)];
-	if (likely(skb->ip_summed == CHECKSUM_PARTIAL))
-		tx_queue += EFX_TXQ_TYPE_OFFLOAD;
+	tx_queue = efx_get_tx_queue(efx, skb_get_queue_mapping(skb),
+				    skb->ip_summed == CHECKSUM_PARTIAL ?
+				    EFX_TXQ_TYPE_OFFLOAD : 0);
 
 	return efx_enqueue_skb(tx_queue, skb);
 }
-- 
1.7.2.1



-- 
Ben Hutchings, Senior Software Engineer, Solarflare Communications
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.

[PATCH net-next-2.6 08/10] sfc: Allocate each channel separately, along with its RX and TX queues

[Ben Hutchings]

This will allow for reallocation of channel structures and rings.

Change module parameter separate_tx_channels to be read-only, since we
now require its value to be constant.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
---
 drivers/net/sfc/efx.c        |   61 +++++++++++++++++++++++------------------
 drivers/net/sfc/falcon.c     |   14 +++-------
 drivers/net/sfc/net_driver.h |   62 +++++++++++++++--------------------------
 3 files changed, 61 insertions(+), 76 deletions(-)

diff --git a/drivers/net/sfc/efx.c b/drivers/net/sfc/efx.c
index 3dd71aa..4b42e61 100644
--- a/drivers/net/sfc/efx.c
+++ b/drivers/net/sfc/efx.c
@@ -114,7 +114,7 @@ static struct workqueue_struct *reset_workqueue;
  * This is only used in MSI-X interrupt mode
  */
 static unsigned int separate_tx_channels;
-module_param(separate_tx_channels, uint, 0644);
+module_param(separate_tx_channels, uint, 0444);
 MODULE_PARM_DESC(separate_tx_channels,
 		 "Use separate channels for TX and RX");
 
@@ -334,6 +334,7 @@ void efx_process_channel_now(struct efx_channel *channel)
 {
 	struct efx_nic *efx = channel->efx;
 
+	BUG_ON(channel->channel >= efx->n_channels);
 	BUG_ON(!channel->enabled);
 
 	/* Disable interrupts and wait for ISRs to complete */
@@ -1098,26 +1099,32 @@ static void efx_remove_interrupts(struct efx_nic *efx)
 	efx->legacy_irq = 0;
 }
 
+struct efx_tx_queue *
+efx_get_tx_queue(struct efx_nic *efx, unsigned index, unsigned type)
+{
+	unsigned tx_channel_offset =
+		separate_tx_channels ? efx->n_channels - efx->n_tx_channels : 0;
+	EFX_BUG_ON_PARANOID(index >= efx->n_tx_channels ||
+			    type >= EFX_TXQ_TYPES);
+	return &efx->channel[tx_channel_offset + index]->tx_queue[type];
+}
+
 static void efx_set_channels(struct efx_nic *efx)
 {
 	struct efx_channel *channel;
 	struct efx_tx_queue *tx_queue;
-	struct efx_rx_queue *rx_queue;
 	unsigned tx_channel_offset =
 		separate_tx_channels ? efx->n_channels - efx->n_tx_channels : 0;
 
+	/* Channel pointers were set in efx_init_struct() but we now
+	 * need to clear them for TX queues in any RX-only channels. */
 	efx_for_each_channel(channel, efx) {
-		if (channel->channel - tx_channel_offset < efx->n_tx_channels) {
-			channel->tx_queue = &efx->tx_queue[
-				(channel->channel - tx_channel_offset) *
-				EFX_TXQ_TYPES];
+		if (channel->channel - tx_channel_offset >=
+		    efx->n_tx_channels) {
 			efx_for_each_channel_tx_queue(tx_queue, channel)
-				tx_queue->channel = channel;
+				tx_queue->channel = NULL;
 		}
 	}
-
-	efx_for_each_rx_queue(rx_queue, efx)
-		rx_queue->channel = &efx->channel[rx_queue->queue];
 }
 
 static int efx_probe_nic(struct efx_nic *efx)
@@ -2044,7 +2051,7 @@ static int efx_init_struct(struct efx_nic *efx, struct efx_nic_type *type,
 	struct efx_channel *channel;
 	struct efx_tx_queue *tx_queue;
 	struct efx_rx_queue *rx_queue;
-	int i;
+	int i, j;
 
 	/* Initialise common structures */
 	memset(efx, 0, sizeof(*efx));
@@ -2072,27 +2079,22 @@ static int efx_init_struct(struct efx_nic *efx, struct efx_nic_type *type,
 	INIT_WORK(&efx->mac_work, efx_mac_work);
 
 	for (i = 0; i < EFX_MAX_CHANNELS; i++) {
-		channel = &efx->channel[i];
+		efx->channel[i] = kzalloc(sizeof(*channel), GFP_KERNEL);
+		channel = efx->channel[i];
 		channel->efx = efx;
 		channel->channel = i;
-		channel->work_pending = false;
 		spin_lock_init(&channel->tx_stop_lock);
 		atomic_set(&channel->tx_stop_count, 1);
-	}
-	for (i = 0; i < EFX_MAX_TX_QUEUES; i++) {
-		tx_queue = &efx->tx_queue[i];
-		tx_queue->efx = efx;
-		tx_queue->queue = i;
-		tx_queue->buffer = NULL;
-		tx_queue->channel = &efx->channel[0]; /* for safety */
-		tx_queue->tso_headers_free = NULL;
-	}
-	for (i = 0; i < EFX_MAX_RX_QUEUES; i++) {
-		rx_queue = &efx->rx_queue[i];
+
+		for (j = 0; j < EFX_TXQ_TYPES; j++) {
+			tx_queue = &channel->tx_queue[j];
+			tx_queue->efx = efx;
+			tx_queue->queue = i * EFX_TXQ_TYPES + j;
+			tx_queue->channel = channel;
+		}
+
+		rx_queue = &channel->rx_queue;
 		rx_queue->efx = efx;
-		rx_queue->queue = i;
-		rx_queue->channel = &efx->channel[0]; /* for safety */
-		rx_queue->buffer = NULL;
 		setup_timer(&rx_queue->slow_fill, efx_rx_slow_fill,
 			    (unsigned long)rx_queue);
 	}
@@ -2120,6 +2122,11 @@ static int efx_init_struct(struct efx_nic *efx, struct efx_nic_type *type,
 
 static void efx_fini_struct(struct efx_nic *efx)
 {
+	int i;
+
+	for (i = 0; i < EFX_MAX_CHANNELS; i++)
+		kfree(efx->channel[i]);
+
 	if (efx->workqueue) {
 		destroy_workqueue(efx->workqueue);
 		efx->workqueue = NULL;
diff --git a/drivers/net/sfc/falcon.c b/drivers/net/sfc/falcon.c
index 4f9d33f..b4d8efe 100644
--- a/drivers/net/sfc/falcon.c
+++ b/drivers/net/sfc/falcon.c
@@ -159,7 +159,6 @@ irqreturn_t falcon_legacy_interrupt_a1(int irq, void *dev_id)
 {
 	struct efx_nic *efx = dev_id;
 	efx_oword_t *int_ker = efx->irq_status.addr;
-	struct efx_channel *channel;
 	int syserr;
 	int queues;
 
@@ -194,15 +193,10 @@ irqreturn_t falcon_legacy_interrupt_a1(int irq, void *dev_id)
 	wmb(); /* Ensure the vector is cleared before interrupt ack */
 	falcon_irq_ack_a1(efx);
 
-	/* Schedule processing of any interrupting queues */
-	channel = &efx->channel[0];
-	while (queues) {
-		if (queues & 0x01)
-			efx_schedule_channel(channel);
-		channel++;
-		queues >>= 1;
-	}
-
+	if (queues & 1)
+		efx_schedule_channel(efx_get_channel(efx, 0));
+	if (queues & 2)
+		efx_schedule_channel(efx_get_channel(efx, 1));
 	return IRQ_HANDLED;
 }
 /**************************************************************************
diff --git a/drivers/net/sfc/net_driver.h b/drivers/net/sfc/net_driver.h
index eb35375..cfc65f5 100644
--- a/drivers/net/sfc/net_driver.h
+++ b/drivers/net/sfc/net_driver.h
@@ -225,8 +225,6 @@ struct efx_rx_page_state {
 /**
  * struct efx_rx_queue - An Efx RX queue
  * @efx: The associated Efx NIC
- * @queue: DMA queue number
- * @channel: The associated channel
  * @buffer: The software buffer ring
  * @rxd: The hardware descriptor ring
  * @added_count: Number of buffers added to the receive queue.
@@ -250,8 +248,6 @@ struct efx_rx_page_state {
  */
 struct efx_rx_queue {
 	struct efx_nic *efx;
-	int queue;
-	struct efx_channel *channel;
 	struct efx_rx_buffer *buffer;
 	struct efx_special_buffer rxd;
 
@@ -327,9 +323,10 @@ enum efx_rx_alloc_method {
  * @n_rx_frm_trunc: Count of RX_FRM_TRUNC errors
  * @n_rx_overlength: Count of RX_OVERLENGTH errors
  * @n_skbuff_leaks: Count of skbuffs leaked due to RX overrun
- * @tx_queue: Pointer to first TX queue, or %NULL if not used for TX
+ * @rx_queue: RX queue for this channel
  * @tx_stop_count: Core TX queue stop count
  * @tx_stop_lock: Core TX queue stop lock
+ * @tx_queue: TX queues for this channel
  */
 struct efx_channel {
 	struct efx_nic *efx;
@@ -366,9 +363,12 @@ struct efx_channel {
 	struct efx_rx_buffer *rx_pkt;
 	bool rx_pkt_csummed;
 
-	struct efx_tx_queue *tx_queue;
+	struct efx_rx_queue rx_queue;
+
 	atomic_t tx_stop_count;
 	spinlock_t tx_stop_lock;
+
+	struct efx_tx_queue tx_queue[2];
 };
 
 enum efx_led_mode {
@@ -724,9 +724,7 @@ struct efx_nic {
 	enum nic_state state;
 	enum reset_type reset_pending;
 
-	struct efx_tx_queue tx_queue[EFX_MAX_TX_QUEUES];
-	struct efx_rx_queue rx_queue[EFX_MAX_RX_QUEUES];
-	struct efx_channel channel[EFX_MAX_CHANNELS];
+	struct efx_channel *channel[EFX_MAX_CHANNELS];
 
 	unsigned next_buffer_table;
 	unsigned n_channels;
@@ -913,34 +911,30 @@ static inline struct efx_channel *
 efx_get_channel(struct efx_nic *efx, unsigned index)
 {
 	EFX_BUG_ON_PARANOID(index >= efx->n_channels);
-	return &efx->channel[index];
+	return efx->channel[index];
 }
 
 /* Iterate over all used channels */
 #define efx_for_each_channel(_channel, _efx)				\
-	for (_channel = &((_efx)->channel[0]);				\
-	     _channel < &((_efx)->channel[(efx)->n_channels]);		\
-	     _channel++)
+	for (_channel = (_efx)->channel[0];				\
+	     _channel;							\
+	     _channel = (_channel->channel + 1 < (_efx)->n_channels) ?	\
+		     (_efx)->channel[_channel->channel + 1] : NULL)
 
-static inline struct efx_tx_queue *
-efx_get_tx_queue(struct efx_nic *efx, unsigned index, unsigned type)
-{
-	EFX_BUG_ON_PARANOID(index >= efx->n_tx_channels ||
-			    type >= EFX_TXQ_TYPES);
-	return &efx->tx_queue[index * EFX_TXQ_TYPES + type];
-}
+extern struct efx_tx_queue *
+efx_get_tx_queue(struct efx_nic *efx, unsigned index, unsigned type);
 
 static inline struct efx_tx_queue *
 efx_channel_get_tx_queue(struct efx_channel *channel, unsigned type)
 {
 	struct efx_tx_queue *tx_queue = channel->tx_queue;
 	EFX_BUG_ON_PARANOID(type >= EFX_TXQ_TYPES);
-	return tx_queue ? tx_queue + type : NULL;
+	return tx_queue->channel ? tx_queue + type : NULL;
 }
 
 /* Iterate over all TX queues belonging to a channel */
 #define efx_for_each_channel_tx_queue(_tx_queue, _channel)		\
-	for (_tx_queue = (_channel)->tx_queue;				\
+	for (_tx_queue = efx_channel_get_tx_queue(channel, 0);		\
 	     _tx_queue && _tx_queue < (_channel)->tx_queue + EFX_TXQ_TYPES; \
 	     _tx_queue++)
 
@@ -948,41 +942,31 @@ static inline struct efx_rx_queue *
 efx_get_rx_queue(struct efx_nic *efx, unsigned index)
 {
 	EFX_BUG_ON_PARANOID(index >= efx->n_rx_channels);
-	return &efx->rx_queue[index];
+	return &efx->channel[index]->rx_queue;
 }
 
-/* Iterate over all used RX queues */
-#define efx_for_each_rx_queue(_rx_queue, _efx)				\
-	for (_rx_queue = &((_efx)->rx_queue[0]);			\
-	     _rx_queue < &((_efx)->rx_queue[(_efx)->n_rx_channels]);	\
-	     _rx_queue++)
-
 static inline struct efx_rx_queue *
 efx_channel_get_rx_queue(struct efx_channel *channel)
 {
-	struct efx_rx_queue *rx_queue =
-		&channel->efx->rx_queue[channel->channel];
-	return rx_queue->channel == channel ? rx_queue : NULL;
+	return channel->channel < channel->efx->n_rx_channels ?
+		&channel->rx_queue : NULL;
 }
 
 /* Iterate over all RX queues belonging to a channel */
 #define efx_for_each_channel_rx_queue(_rx_queue, _channel)		\
-	for (_rx_queue = &((_channel)->efx->rx_queue[(_channel)->channel]); \
+	for (_rx_queue = efx_channel_get_rx_queue(channel);		\
 	     _rx_queue;							\
-	     _rx_queue = NULL)						\
-		if (_rx_queue->channel != (_channel))			\
-			continue;					\
-		else
+	     _rx_queue = NULL)
 
 static inline struct efx_channel *
 efx_rx_queue_channel(struct efx_rx_queue *rx_queue)
 {
-	return rx_queue->channel;
+	return container_of(rx_queue, struct efx_channel, rx_queue);
 }
 
 static inline int efx_rx_queue_index(struct efx_rx_queue *rx_queue)
 {
-	return rx_queue->queue;
+	return efx_rx_queue_channel(rx_queue)->channel;
 }
 
 /* Returns a pointer to the specified receive buffer in the RX
-- 
1.7.2.1



-- 
Ben Hutchings, Senior Software Engineer, Solarflare Communications
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.

[PATCH net-next-2.6 09/10] sfc: Make the dmaq size a run-time setting (rather than compile-time)

[Ben Hutchings]

From: Steve Hodgson <shodgson@solarflare.com>

- Allow the ring size to be specified in non
   power-of-two sizes (for instance to limit
   the amount of receive buffers).
 - Automatically size the event queue.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
---
 drivers/net/sfc/efx.c        |   15 ++++++++---
 drivers/net/sfc/efx.h        |   13 +++++----
 drivers/net/sfc/net_driver.h |   13 +++++++--
 drivers/net/sfc/nic.c        |   53 +++++++++++++++++++++------------------
 drivers/net/sfc/rx.c         |   33 +++++++++++++++---------
 drivers/net/sfc/selftest.c   |    2 +-
 drivers/net/sfc/tx.c         |   56 +++++++++++++++++++++++------------------
 7 files changed, 109 insertions(+), 76 deletions(-)

diff --git a/drivers/net/sfc/efx.c b/drivers/net/sfc/efx.c
index 4b42e61..6166e22 100644
--- a/drivers/net/sfc/efx.c
+++ b/drivers/net/sfc/efx.c
@@ -348,7 +348,7 @@ void efx_process_channel_now(struct efx_channel *channel)
 	napi_disable(&channel->napi_str);
 
 	/* Poll the channel */
-	efx_process_channel(channel, EFX_EVQ_SIZE);
+	efx_process_channel(channel, channel->eventq_mask + 1);
 
 	/* Ack the eventq. This may cause an interrupt to be generated
 	 * when they are reenabled */
@@ -365,9 +365,18 @@ void efx_process_channel_now(struct efx_channel *channel)
  */
 static int efx_probe_eventq(struct efx_channel *channel)
 {
+	struct efx_nic *efx = channel->efx;
+	unsigned long entries;
+
 	netif_dbg(channel->efx, probe, channel->efx->net_dev,
 		  "chan %d create event queue\n", channel->channel);
 
+	/* Build an event queue with room for one event per tx and rx buffer,
+	 * plus some extra for link state events and MCDI completions. */
+	entries = roundup_pow_of_two(efx->rxq_entries + efx->txq_entries + 128);
+	EFX_BUG_ON_PARANOID(entries > EFX_MAX_EVQ_SIZE);
+	channel->eventq_mask = max(entries, EFX_MIN_EVQ_SIZE) - 1;
+
 	return efx_nic_probe_eventq(channel);
 }
 
@@ -1191,6 +1200,7 @@ static int efx_probe_all(struct efx_nic *efx)
 	}
 
 	/* Create channels */
+	efx->rxq_entries = efx->txq_entries = EFX_DEFAULT_DMAQ_SIZE;
 	efx_for_each_channel(channel, efx) {
 		rc = efx_probe_channel(channel);
 		if (rc) {
@@ -2101,9 +2111,6 @@ static int efx_init_struct(struct efx_nic *efx, struct efx_nic_type *type,
 
 	efx->type = type;
 
-	/* As close as we can get to guaranteeing that we don't overflow */
-	BUILD_BUG_ON(EFX_EVQ_SIZE < EFX_TXQ_SIZE + EFX_RXQ_SIZE);
-
 	EFX_BUG_ON_PARANOID(efx->type->phys_addr_channels > EFX_MAX_CHANNELS);
 
 	/* Higher numbered interrupt modes are less capable! */
diff --git a/drivers/net/sfc/efx.h b/drivers/net/sfc/efx.h
index d6b172b..c15a2d3 100644
--- a/drivers/net/sfc/efx.h
+++ b/drivers/net/sfc/efx.h
@@ -37,8 +37,6 @@ efx_enqueue_skb(struct efx_tx_queue *tx_queue, struct sk_buff *skb);
 extern void efx_xmit_done(struct efx_tx_queue *tx_queue, unsigned int index);
 extern void efx_stop_queue(struct efx_channel *channel);
 extern void efx_wake_queue(struct efx_channel *channel);
-#define EFX_TXQ_SIZE 1024
-#define EFX_TXQ_MASK (EFX_TXQ_SIZE - 1)
 
 /* RX */
 extern int efx_probe_rx_queue(struct efx_rx_queue *rx_queue);
@@ -53,13 +51,16 @@ extern void __efx_rx_packet(struct efx_channel *channel,
 extern void efx_rx_packet(struct efx_rx_queue *rx_queue, unsigned int index,
 			  unsigned int len, bool checksummed, bool discard);
 extern void efx_schedule_slow_fill(struct efx_rx_queue *rx_queue);
-#define EFX_RXQ_SIZE 1024
-#define EFX_RXQ_MASK (EFX_RXQ_SIZE - 1)
+
+#define EFX_MAX_DMAQ_SIZE 4096UL
+#define EFX_DEFAULT_DMAQ_SIZE 1024UL
+#define EFX_MIN_DMAQ_SIZE 512UL
+
+#define EFX_MAX_EVQ_SIZE 16384UL
+#define EFX_MIN_EVQ_SIZE 512UL
 
 /* Channels */
 extern void efx_process_channel_now(struct efx_channel *channel);
-#define EFX_EVQ_SIZE 4096
-#define EFX_EVQ_MASK (EFX_EVQ_SIZE - 1)
 
 /* Ports */
 extern int efx_reconfigure_port(struct efx_nic *efx);
diff --git a/drivers/net/sfc/net_driver.h b/drivers/net/sfc/net_driver.h
index cfc65f5..ac622ab 100644
--- a/drivers/net/sfc/net_driver.h
+++ b/drivers/net/sfc/net_driver.h
@@ -137,6 +137,7 @@ struct efx_tx_buffer {
  * @channel: The associated channel
  * @buffer: The software buffer ring
  * @txd: The hardware descriptor ring
+ * @ptr_mask: The size of the ring minus 1.
  * @flushed: Used when handling queue flushing
  * @read_count: Current read pointer.
  *	This is the number of buffers that have been removed from both rings.
@@ -170,6 +171,7 @@ struct efx_tx_queue {
 	struct efx_nic *nic;
 	struct efx_tx_buffer *buffer;
 	struct efx_special_buffer txd;
+	unsigned int ptr_mask;
 	enum efx_flush_state flushed;
 
 	/* Members used mainly on the completion path */
@@ -227,6 +229,7 @@ struct efx_rx_page_state {
  * @efx: The associated Efx NIC
  * @buffer: The software buffer ring
  * @rxd: The hardware descriptor ring
+ * @ptr_mask: The size of the ring minus 1.
  * @added_count: Number of buffers added to the receive queue.
  * @notified_count: Number of buffers given to NIC (<= @added_count).
  * @removed_count: Number of buffers removed from the receive queue.
@@ -238,9 +241,6 @@ struct efx_rx_page_state {
  * @min_fill: RX descriptor minimum non-zero fill level.
  *	This records the minimum fill level observed when a ring
  *	refill was triggered.
- * @min_overfill: RX descriptor minimum overflow fill level.
- *	This records the minimum fill level at which RX queue
- *	overflow was observed.  It should never be set.
  * @alloc_page_count: RX allocation strategy counter.
  * @alloc_skb_count: RX allocation strategy counter.
  * @slow_fill: Timer used to defer efx_nic_generate_fill_event().
@@ -250,6 +250,7 @@ struct efx_rx_queue {
 	struct efx_nic *efx;
 	struct efx_rx_buffer *buffer;
 	struct efx_special_buffer rxd;
+	unsigned int ptr_mask;
 
 	int added_count;
 	int notified_count;
@@ -307,6 +308,7 @@ enum efx_rx_alloc_method {
  * @reset_work: Scheduled reset work thread
  * @work_pending: Is work pending via NAPI?
  * @eventq: Event queue buffer
+ * @eventq_mask: Event queue pointer mask
  * @eventq_read_ptr: Event queue read pointer
  * @last_eventq_read_ptr: Last event queue read pointer value.
  * @magic_count: Event queue test event count
@@ -339,6 +341,7 @@ struct efx_channel {
 	struct napi_struct napi_str;
 	bool work_pending;
 	struct efx_special_buffer eventq;
+	unsigned int eventq_mask;
 	unsigned int eventq_read_ptr;
 	unsigned int last_eventq_read_ptr;
 	unsigned int magic_count;
@@ -641,6 +644,8 @@ union efx_multicast_hash {
  * @tx_queue: TX DMA queues
  * @rx_queue: RX DMA queues
  * @channel: Channels
+ * @rxq_entries: Size of receive queues requested by user.
+ * @txq_entries: Size of transmit queues requested by user.
  * @next_buffer_table: First available buffer table id
  * @n_channels: Number of channels in use
  * @n_rx_channels: Number of channels used for RX (= number of RX queues)
@@ -726,6 +731,8 @@ struct efx_nic {
 
 	struct efx_channel *channel[EFX_MAX_CHANNELS];
 
+	unsigned rxq_entries;
+	unsigned txq_entries;
 	unsigned next_buffer_table;
 	unsigned n_channels;
 	unsigned n_rx_channels;
diff --git a/drivers/net/sfc/nic.c b/drivers/net/sfc/nic.c
index 9e35633..0deb5c3 100644
--- a/drivers/net/sfc/nic.c
+++ b/drivers/net/sfc/nic.c
@@ -356,7 +356,7 @@ static inline void efx_notify_tx_desc(struct efx_tx_queue *tx_queue)
 	unsigned write_ptr;
 	efx_dword_t reg;
 
-	write_ptr = tx_queue->write_count & EFX_TXQ_MASK;
+	write_ptr = tx_queue->write_count & tx_queue->ptr_mask;
 	EFX_POPULATE_DWORD_1(reg, FRF_AZ_TX_DESC_WPTR_DWORD, write_ptr);
 	efx_writed_page(tx_queue->efx, &reg,
 			FR_AZ_TX_DESC_UPD_DWORD_P0, tx_queue->queue);
@@ -377,7 +377,7 @@ void efx_nic_push_buffers(struct efx_tx_queue *tx_queue)
 	BUG_ON(tx_queue->write_count == tx_queue->insert_count);
 
 	do {
-		write_ptr = tx_queue->write_count & EFX_TXQ_MASK;
+		write_ptr = tx_queue->write_count & tx_queue->ptr_mask;
 		buffer = &tx_queue->buffer[write_ptr];
 		txd = efx_tx_desc(tx_queue, write_ptr);
 		++tx_queue->write_count;
@@ -398,10 +398,11 @@ void efx_nic_push_buffers(struct efx_tx_queue *tx_queue)
 int efx_nic_probe_tx(struct efx_tx_queue *tx_queue)
 {
 	struct efx_nic *efx = tx_queue->efx;
-	BUILD_BUG_ON(EFX_TXQ_SIZE < 512 || EFX_TXQ_SIZE > 4096 ||
-		     EFX_TXQ_SIZE & EFX_TXQ_MASK);
+	unsigned entries;
+
+	entries = tx_queue->ptr_mask + 1;
 	return efx_alloc_special_buffer(efx, &tx_queue->txd,
-					EFX_TXQ_SIZE * sizeof(efx_qword_t));
+					entries * sizeof(efx_qword_t));
 }
 
 void efx_nic_init_tx(struct efx_tx_queue *tx_queue)
@@ -526,30 +527,32 @@ efx_build_rx_desc(struct efx_rx_queue *rx_queue, unsigned index)
  */
 void efx_nic_notify_rx_desc(struct efx_rx_queue *rx_queue)
 {
+	struct efx_nic *efx = rx_queue->efx;
 	efx_dword_t reg;
 	unsigned write_ptr;
 
 	while (rx_queue->notified_count != rx_queue->added_count) {
-		efx_build_rx_desc(rx_queue,
-				  rx_queue->notified_count &
-				  EFX_RXQ_MASK);
+		efx_build_rx_desc(
+			rx_queue,
+			rx_queue->notified_count & rx_queue->ptr_mask);
 		++rx_queue->notified_count;
 	}
 
 	wmb();
-	write_ptr = rx_queue->added_count & EFX_RXQ_MASK;
+	write_ptr = rx_queue->added_count & rx_queue->ptr_mask;
 	EFX_POPULATE_DWORD_1(reg, FRF_AZ_RX_DESC_WPTR_DWORD, write_ptr);
-	efx_writed_page(rx_queue->efx, &reg, FR_AZ_RX_DESC_UPD_DWORD_P0,
+	efx_writed_page(efx, &reg, FR_AZ_RX_DESC_UPD_DWORD_P0,
 			efx_rx_queue_index(rx_queue));
 }
 
 int efx_nic_probe_rx(struct efx_rx_queue *rx_queue)
 {
 	struct efx_nic *efx = rx_queue->efx;
-	BUILD_BUG_ON(EFX_RXQ_SIZE < 512 || EFX_RXQ_SIZE > 4096 ||
-		     EFX_RXQ_SIZE & EFX_RXQ_MASK);
+	unsigned entries;
+
+	entries = rx_queue->ptr_mask + 1;
 	return efx_alloc_special_buffer(efx, &rx_queue->rxd,
-					EFX_RXQ_SIZE * sizeof(efx_qword_t));
+					entries * sizeof(efx_qword_t));
 }
 
 void efx_nic_init_rx(struct efx_rx_queue *rx_queue)
@@ -685,7 +688,7 @@ efx_handle_tx_event(struct efx_channel *channel, efx_qword_t *event)
 		tx_queue = efx_channel_get_tx_queue(
 			channel, tx_ev_q_label % EFX_TXQ_TYPES);
 		tx_packets = ((tx_ev_desc_ptr - tx_queue->read_count) &
-			      EFX_TXQ_MASK);
+			      tx_queue->ptr_mask);
 		channel->irq_mod_score += tx_packets;
 		efx_xmit_done(tx_queue, tx_ev_desc_ptr);
 	} else if (EFX_QWORD_FIELD(*event, FSF_AZ_TX_EV_WQ_FF_FULL)) {
@@ -796,8 +799,8 @@ efx_handle_rx_bad_index(struct efx_rx_queue *rx_queue, unsigned index)
 	struct efx_nic *efx = rx_queue->efx;
 	unsigned expected, dropped;
 
-	expected = rx_queue->removed_count & EFX_RXQ_MASK;
-	dropped = (index - expected) & EFX_RXQ_MASK;
+	expected = rx_queue->removed_count & rx_queue->ptr_mask;
+	dropped = (index - expected) & rx_queue->ptr_mask;
 	netif_info(efx, rx_err, efx->net_dev,
 		   "dropped %d events (index=%d expected=%d)\n",
 		   dropped, index, expected);
@@ -835,7 +838,7 @@ efx_handle_rx_event(struct efx_channel *channel, const efx_qword_t *event)
 	rx_queue = efx_channel_get_rx_queue(channel);
 
 	rx_ev_desc_ptr = EFX_QWORD_FIELD(*event, FSF_AZ_RX_EV_DESC_PTR);
-	expected_ptr = rx_queue->removed_count & EFX_RXQ_MASK;
+	expected_ptr = rx_queue->removed_count & rx_queue->ptr_mask;
 	if (unlikely(rx_ev_desc_ptr != expected_ptr))
 		efx_handle_rx_bad_index(rx_queue, rx_ev_desc_ptr);
 
@@ -1002,6 +1005,7 @@ efx_handle_driver_event(struct efx_channel *channel, efx_qword_t *event)
 
 int efx_nic_process_eventq(struct efx_channel *channel, int budget)
 {
+	struct efx_nic *efx = channel->efx;
 	unsigned int read_ptr;
 	efx_qword_t event, *p_event;
 	int ev_code;
@@ -1026,7 +1030,7 @@ int efx_nic_process_eventq(struct efx_channel *channel, int budget)
 		EFX_SET_QWORD(*p_event);
 
 		/* Increment read pointer */
-		read_ptr = (read_ptr + 1) & EFX_EVQ_MASK;
+		read_ptr = (read_ptr + 1) & channel->eventq_mask;
 
 		ev_code = EFX_QWORD_FIELD(event, FSF_AZ_EV_CODE);
 
@@ -1038,7 +1042,7 @@ int efx_nic_process_eventq(struct efx_channel *channel, int budget)
 			break;
 		case FSE_AZ_EV_CODE_TX_EV:
 			tx_packets += efx_handle_tx_event(channel, &event);
-			if (tx_packets >= EFX_TXQ_SIZE) {
+			if (tx_packets > efx->txq_entries) {
 				spent = budget;
 				goto out;
 			}
@@ -1073,10 +1077,11 @@ out:
 int efx_nic_probe_eventq(struct efx_channel *channel)
 {
 	struct efx_nic *efx = channel->efx;
-	BUILD_BUG_ON(EFX_EVQ_SIZE < 512 || EFX_EVQ_SIZE > 32768 ||
-		     EFX_EVQ_SIZE & EFX_EVQ_MASK);
+	unsigned entries;
+
+	entries = channel->eventq_mask + 1;
 	return efx_alloc_special_buffer(efx, &channel->eventq,
-					EFX_EVQ_SIZE * sizeof(efx_qword_t));
+					entries * sizeof(efx_qword_t));
 }
 
 void efx_nic_init_eventq(struct efx_channel *channel)
@@ -1172,7 +1177,7 @@ static void efx_poll_flush_events(struct efx_nic *efx)
 	struct efx_tx_queue *tx_queue;
 	struct efx_rx_queue *rx_queue;
 	unsigned int read_ptr = channel->eventq_read_ptr;
-	unsigned int end_ptr = (read_ptr - 1) & EFX_EVQ_MASK;
+	unsigned int end_ptr = (read_ptr - 1) & channel->eventq_mask;
 
 	do {
 		efx_qword_t *event = efx_event(channel, read_ptr);
@@ -1212,7 +1217,7 @@ static void efx_poll_flush_events(struct efx_nic *efx)
 		 * it's ok to throw away every non-flush event */
 		EFX_SET_QWORD(*event);
 
-		read_ptr = (read_ptr + 1) & EFX_EVQ_MASK;
+		read_ptr = (read_ptr + 1) & channel->eventq_mask;
 	} while (read_ptr != end_ptr);
 
 	channel->eventq_read_ptr = read_ptr;
diff --git a/drivers/net/sfc/rx.c b/drivers/net/sfc/rx.c
index 6651d93..6d0959b 100644
--- a/drivers/net/sfc/rx.c
+++ b/drivers/net/sfc/rx.c
@@ -133,7 +133,7 @@ static int efx_init_rx_buffers_skb(struct efx_rx_queue *rx_queue)
 	unsigned index, count;
 
 	for (count = 0; count < EFX_RX_BATCH; ++count) {
-		index = rx_queue->added_count & EFX_RXQ_MASK;
+		index = rx_queue->added_count & rx_queue->ptr_mask;
 		rx_buf = efx_rx_buffer(rx_queue, index);
 
 		rx_buf->skb = netdev_alloc_skb(net_dev, skb_len);
@@ -208,7 +208,7 @@ static int efx_init_rx_buffers_page(struct efx_rx_queue *rx_queue)
 		dma_addr += sizeof(struct efx_rx_page_state);
 
 	split:
-		index = rx_queue->added_count & EFX_RXQ_MASK;
+		index = rx_queue->added_count & rx_queue->ptr_mask;
 		rx_buf = efx_rx_buffer(rx_queue, index);
 		rx_buf->dma_addr = dma_addr + EFX_PAGE_IP_ALIGN;
 		rx_buf->skb = NULL;
@@ -285,7 +285,7 @@ static void efx_resurrect_rx_buffer(struct efx_rx_queue *rx_queue,
 	 * we'd like to insert an additional descriptor whilst leaving
 	 * EFX_RXD_HEAD_ROOM for the non-recycle path */
 	fill_level = (rx_queue->added_count - rx_queue->removed_count + 2);
-	if (unlikely(fill_level >= EFX_RXQ_SIZE - EFX_RXD_HEAD_ROOM)) {
+	if (unlikely(fill_level > rx_queue->max_fill)) {
 		/* We could place "state" on a list, and drain the list in
 		 * efx_fast_push_rx_descriptors(). For now, this will do. */
 		return;
@@ -294,7 +294,7 @@ static void efx_resurrect_rx_buffer(struct efx_rx_queue *rx_queue,
 	++state->refcnt;
 	get_page(rx_buf->page);
 
-	index = rx_queue->added_count & EFX_RXQ_MASK;
+	index = rx_queue->added_count & rx_queue->ptr_mask;
 	new_buf = efx_rx_buffer(rx_queue, index);
 	new_buf->dma_addr = rx_buf->dma_addr ^ (PAGE_SIZE >> 1);
 	new_buf->skb = NULL;
@@ -319,7 +319,7 @@ static void efx_recycle_rx_buffer(struct efx_channel *channel,
 	    page_count(rx_buf->page) == 1)
 		efx_resurrect_rx_buffer(rx_queue, rx_buf);
 
-	index = rx_queue->added_count & EFX_RXQ_MASK;
+	index = rx_queue->added_count & rx_queue->ptr_mask;
 	new_buf = efx_rx_buffer(rx_queue, index);
 
 	memcpy(new_buf, rx_buf, sizeof(*new_buf));
@@ -347,7 +347,7 @@ void efx_fast_push_rx_descriptors(struct efx_rx_queue *rx_queue)
 
 	/* Calculate current fill level, and exit if we don't need to fill */
 	fill_level = (rx_queue->added_count - rx_queue->removed_count);
-	EFX_BUG_ON_PARANOID(fill_level > EFX_RXQ_SIZE);
+	EFX_BUG_ON_PARANOID(fill_level > rx_queue->efx->rxq_entries);
 	if (fill_level >= rx_queue->fast_fill_trigger)
 		goto out;
 
@@ -650,15 +650,22 @@ void efx_rx_strategy(struct efx_channel *channel)
 int efx_probe_rx_queue(struct efx_rx_queue *rx_queue)
 {
 	struct efx_nic *efx = rx_queue->efx;
-	unsigned int rxq_size;
+	unsigned int entries;
 	int rc;
 
+	/* Create the smallest power-of-two aligned ring */
+	entries = max(roundup_pow_of_two(efx->rxq_entries), EFX_MIN_DMAQ_SIZE);
+	EFX_BUG_ON_PARANOID(entries > EFX_MAX_DMAQ_SIZE);
+	rx_queue->ptr_mask = entries - 1;
+
 	netif_dbg(efx, probe, efx->net_dev,
-		  "creating RX queue %d\n", efx_rx_queue_index(rx_queue));
+		  "creating RX queue %d size %#x mask %#x\n",
+		  efx_rx_queue_index(rx_queue), efx->rxq_entries,
+		  rx_queue->ptr_mask);
 
 	/* Allocate RX buffers */
-	rxq_size = EFX_RXQ_SIZE * sizeof(*rx_queue->buffer);
-	rx_queue->buffer = kzalloc(rxq_size, GFP_KERNEL);
+	rx_queue->buffer = kzalloc(entries * sizeof(*rx_queue->buffer),
+				   GFP_KERNEL);
 	if (!rx_queue->buffer)
 		return -ENOMEM;
 
@@ -672,6 +679,7 @@ int efx_probe_rx_queue(struct efx_rx_queue *rx_queue)
 
 void efx_init_rx_queue(struct efx_rx_queue *rx_queue)
 {
+	struct efx_nic *efx = rx_queue->efx;
 	unsigned int max_fill, trigger, limit;
 
 	netif_dbg(rx_queue->efx, drv, rx_queue->efx->net_dev,
@@ -682,10 +690,9 @@ void efx_init_rx_queue(struct efx_rx_queue *rx_queue)
 	rx_queue->notified_count = 0;
 	rx_queue->removed_count = 0;
 	rx_queue->min_fill = -1U;
-	rx_queue->min_overfill = -1U;
 
 	/* Initialise limit fields */
-	max_fill = EFX_RXQ_SIZE - EFX_RXD_HEAD_ROOM;
+	max_fill = efx->rxq_entries - EFX_RXD_HEAD_ROOM;
 	trigger = max_fill * min(rx_refill_threshold, 100U) / 100U;
 	limit = max_fill * min(rx_refill_limit, 100U) / 100U;
 
@@ -710,7 +717,7 @@ void efx_fini_rx_queue(struct efx_rx_queue *rx_queue)
 
 	/* Release RX buffers NB start at index 0 not current HW ptr */
 	if (rx_queue->buffer) {
-		for (i = 0; i <= EFX_RXQ_MASK; i++) {
+		for (i = 0; i <= rx_queue->ptr_mask; i++) {
 			rx_buf = efx_rx_buffer(rx_queue, i);
 			efx_fini_rx_buffer(rx_queue, rx_buf);
 		}
diff --git a/drivers/net/sfc/selftest.c b/drivers/net/sfc/selftest.c
index 11153d9..da4473b 100644
--- a/drivers/net/sfc/selftest.c
+++ b/drivers/net/sfc/selftest.c
@@ -506,7 +506,7 @@ efx_test_loopback(struct efx_tx_queue *tx_queue,
 
 	for (i = 0; i < 3; i++) {
 		/* Determine how many packets to send */
-		state->packet_count = EFX_TXQ_SIZE / 3;
+		state->packet_count = efx->txq_entries / 3;
 		state->packet_count = min(1 << (i << 2), state->packet_count);
 		state->skbs = kzalloc(sizeof(state->skbs[0]) *
 				      state->packet_count, GFP_KERNEL);
diff --git a/drivers/net/sfc/tx.c b/drivers/net/sfc/tx.c
index 6a6acc4..1172698 100644
--- a/drivers/net/sfc/tx.c
+++ b/drivers/net/sfc/tx.c
@@ -28,7 +28,7 @@
  * The tx_queue descriptor ring fill-level must fall below this value
  * before we restart the netif queue
  */
-#define EFX_TXQ_THRESHOLD (EFX_TXQ_MASK / 2u)
+#define EFX_TXQ_THRESHOLD(_efx) ((_efx)->txq_entries / 2u)
 
 /* We need to be able to nest calls to netif_tx_stop_queue(), partly
  * because of the 2 hardware queues associated with each core queue,
@@ -207,7 +207,7 @@ netdev_tx_t efx_enqueue_skb(struct efx_tx_queue *tx_queue, struct sk_buff *skb)
 	}
 
 	fill_level = tx_queue->insert_count - tx_queue->old_read_count;
-	q_space = EFX_TXQ_MASK - 1 - fill_level;
+	q_space = efx->txq_entries - 1 - fill_level;
 
 	/* Map for DMA.  Use pci_map_single rather than pci_map_page
 	 * since this is more efficient on machines with sparse
@@ -244,14 +244,14 @@ netdev_tx_t efx_enqueue_skb(struct efx_tx_queue *tx_queue, struct sk_buff *skb)
 					&tx_queue->read_count;
 				fill_level = (tx_queue->insert_count
 					      - tx_queue->old_read_count);
-				q_space = EFX_TXQ_MASK - 1 - fill_level;
+				q_space = efx->txq_entries - 1 - fill_level;
 				if (unlikely(q_space-- <= 0))
 					goto stop;
 				smp_mb();
 				--tx_queue->stopped;
 			}
 
-			insert_ptr = tx_queue->insert_count & EFX_TXQ_MASK;
+			insert_ptr = tx_queue->insert_count & tx_queue->ptr_mask;
 			buffer = &tx_queue->buffer[insert_ptr];
 			efx_tsoh_free(tx_queue, buffer);
 			EFX_BUG_ON_PARANOID(buffer->tsoh);
@@ -320,7 +320,7 @@ netdev_tx_t efx_enqueue_skb(struct efx_tx_queue *tx_queue, struct sk_buff *skb)
 	/* Work backwards until we hit the original insert pointer value */
 	while (tx_queue->insert_count != tx_queue->write_count) {
 		--tx_queue->insert_count;
-		insert_ptr = tx_queue->insert_count & EFX_TXQ_MASK;
+		insert_ptr = tx_queue->insert_count & tx_queue->ptr_mask;
 		buffer = &tx_queue->buffer[insert_ptr];
 		efx_dequeue_buffer(tx_queue, buffer);
 		buffer->len = 0;
@@ -350,8 +350,8 @@ static void efx_dequeue_buffers(struct efx_tx_queue *tx_queue,
 	struct efx_nic *efx = tx_queue->efx;
 	unsigned int stop_index, read_ptr;
 
-	stop_index = (index + 1) & EFX_TXQ_MASK;
-	read_ptr = tx_queue->read_count & EFX_TXQ_MASK;
+	stop_index = (index + 1) & tx_queue->ptr_mask;
+	read_ptr = tx_queue->read_count & tx_queue->ptr_mask;
 
 	while (read_ptr != stop_index) {
 		struct efx_tx_buffer *buffer = &tx_queue->buffer[read_ptr];
@@ -368,7 +368,7 @@ static void efx_dequeue_buffers(struct efx_tx_queue *tx_queue,
 		buffer->len = 0;
 
 		++tx_queue->read_count;
-		read_ptr = tx_queue->read_count & EFX_TXQ_MASK;
+		read_ptr = tx_queue->read_count & tx_queue->ptr_mask;
 	}
 }
 
@@ -402,7 +402,7 @@ void efx_xmit_done(struct efx_tx_queue *tx_queue, unsigned int index)
 	unsigned fill_level;
 	struct efx_nic *efx = tx_queue->efx;
 
-	EFX_BUG_ON_PARANOID(index > EFX_TXQ_MASK);
+	EFX_BUG_ON_PARANOID(index > tx_queue->ptr_mask);
 
 	efx_dequeue_buffers(tx_queue, index);
 
@@ -412,7 +412,7 @@ void efx_xmit_done(struct efx_tx_queue *tx_queue, unsigned int index)
 	smp_mb();
 	if (unlikely(tx_queue->stopped) && likely(efx->port_enabled)) {
 		fill_level = tx_queue->insert_count - tx_queue->read_count;
-		if (fill_level < EFX_TXQ_THRESHOLD) {
+		if (fill_level < EFX_TXQ_THRESHOLD(efx)) {
 			EFX_BUG_ON_PARANOID(!efx_dev_registered(efx));
 
 			/* Do this under netif_tx_lock(), to avoid racing
@@ -430,18 +430,24 @@ void efx_xmit_done(struct efx_tx_queue *tx_queue, unsigned int index)
 int efx_probe_tx_queue(struct efx_tx_queue *tx_queue)
 {
 	struct efx_nic *efx = tx_queue->efx;
-	unsigned int txq_size;
+	unsigned int entries;
 	int i, rc;
 
-	netif_dbg(efx, probe, efx->net_dev, "creating TX queue %d\n",
-		  tx_queue->queue);
+	/* Create the smallest power-of-two aligned ring */
+	entries = max(roundup_pow_of_two(efx->txq_entries), EFX_MIN_DMAQ_SIZE);
+	EFX_BUG_ON_PARANOID(entries > EFX_MAX_DMAQ_SIZE);
+	tx_queue->ptr_mask = entries - 1;
+
+	netif_dbg(efx, probe, efx->net_dev,
+		  "creating TX queue %d size %#x mask %#x\n",
+		  tx_queue->queue, efx->txq_entries, tx_queue->ptr_mask);
 
 	/* Allocate software ring */
-	txq_size = EFX_TXQ_SIZE * sizeof(*tx_queue->buffer);
-	tx_queue->buffer = kzalloc(txq_size, GFP_KERNEL);
+	tx_queue->buffer = kzalloc(entries * sizeof(*tx_queue->buffer),
+				   GFP_KERNEL);
 	if (!tx_queue->buffer)
 		return -ENOMEM;
-	for (i = 0; i <= EFX_TXQ_MASK; ++i)
+	for (i = 0; i <= tx_queue->ptr_mask; ++i)
 		tx_queue->buffer[i].continuation = true;
 
 	/* Allocate hardware ring */
@@ -481,7 +487,7 @@ void efx_release_tx_buffers(struct efx_tx_queue *tx_queue)
 
 	/* Free any buffers left in the ring */
 	while (tx_queue->read_count != tx_queue->write_count) {
-		buffer = &tx_queue->buffer[tx_queue->read_count & EFX_TXQ_MASK];
+		buffer = &tx_queue->buffer[tx_queue->read_count & tx_queue->ptr_mask];
 		efx_dequeue_buffer(tx_queue, buffer);
 		buffer->continuation = true;
 		buffer->len = 0;
@@ -741,7 +747,7 @@ static int efx_tx_queue_insert(struct efx_tx_queue *tx_queue,
 
 	fill_level = tx_queue->insert_count - tx_queue->old_read_count;
 	/* -1 as there is no way to represent all descriptors used */
-	q_space = EFX_TXQ_MASK - 1 - fill_level;
+	q_space = efx->txq_entries - 1 - fill_level;
 
 	while (1) {
 		if (unlikely(q_space-- <= 0)) {
@@ -757,7 +763,7 @@ static int efx_tx_queue_insert(struct efx_tx_queue *tx_queue,
 				*(volatile unsigned *)&tx_queue->read_count;
 			fill_level = (tx_queue->insert_count
 				      - tx_queue->old_read_count);
-			q_space = EFX_TXQ_MASK - 1 - fill_level;
+			q_space = efx->txq_entries - 1 - fill_level;
 			if (unlikely(q_space-- <= 0)) {
 				*final_buffer = NULL;
 				return 1;
@@ -766,13 +772,13 @@ static int efx_tx_queue_insert(struct efx_tx_queue *tx_queue,
 			--tx_queue->stopped;
 		}
 
-		insert_ptr = tx_queue->insert_count & EFX_TXQ_MASK;
+		insert_ptr = tx_queue->insert_count & tx_queue->ptr_mask;
 		buffer = &tx_queue->buffer[insert_ptr];
 		++tx_queue->insert_count;
 
 		EFX_BUG_ON_PARANOID(tx_queue->insert_count -
-				    tx_queue->read_count >
-				    EFX_TXQ_MASK);
+				    tx_queue->read_count >=
+				    efx->txq_entries);
 
 		efx_tsoh_free(tx_queue, buffer);
 		EFX_BUG_ON_PARANOID(buffer->len);
@@ -813,7 +819,7 @@ static void efx_tso_put_header(struct efx_tx_queue *tx_queue,
 {
 	struct efx_tx_buffer *buffer;
 
-	buffer = &tx_queue->buffer[tx_queue->insert_count & EFX_TXQ_MASK];
+	buffer = &tx_queue->buffer[tx_queue->insert_count & tx_queue->ptr_mask];
 	efx_tsoh_free(tx_queue, buffer);
 	EFX_BUG_ON_PARANOID(buffer->len);
 	EFX_BUG_ON_PARANOID(buffer->unmap_len);
@@ -838,7 +844,7 @@ static void efx_enqueue_unwind(struct efx_tx_queue *tx_queue)
 	while (tx_queue->insert_count != tx_queue->write_count) {
 		--tx_queue->insert_count;
 		buffer = &tx_queue->buffer[tx_queue->insert_count &
-					   EFX_TXQ_MASK];
+					   tx_queue->ptr_mask];
 		efx_tsoh_free(tx_queue, buffer);
 		EFX_BUG_ON_PARANOID(buffer->skb);
 		if (buffer->unmap_len) {
@@ -1168,7 +1174,7 @@ static void efx_fini_tso(struct efx_tx_queue *tx_queue)
 	unsigned i;
 
 	if (tx_queue->buffer) {
-		for (i = 0; i <= EFX_TXQ_MASK; ++i)
+		for (i = 0; i <= tx_queue->ptr_mask; ++i)
 			efx_tsoh_free(tx_queue, &tx_queue->buffer[i]);
 	}
 
-- 
1.7.2.1



-- 
Ben Hutchings, Senior Software Engineer, Solarflare Communications
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.

[PATCH net-next-2.6 10/10] sfc: Allow changing the DMA ring sizes dynamically via ethtool

[Ben Hutchings]

This requires some reorganisation of channel setup and teardown to
ensure that we can always roll-back a failed change.

Based on work by Steve Hodgson <shodgson@solarflare.com>

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
---
 drivers/net/sfc/efx.c        |  218 +++++++++++++++++++++++++++++++++---------
 drivers/net/sfc/efx.h        |    7 ++
 drivers/net/sfc/ethtool.c    |   38 +++++++
 drivers/net/sfc/net_driver.h |    4 +-
 drivers/net/sfc/nic.c        |    9 +-
 5 files changed, 224 insertions(+), 52 deletions(-)

diff --git a/drivers/net/sfc/efx.c b/drivers/net/sfc/efx.c
index 6166e22..f702f1f 100644
--- a/drivers/net/sfc/efx.c
+++ b/drivers/net/sfc/efx.c
@@ -201,10 +201,13 @@ MODULE_PARM_DESC(debug, "Bitmapped debugging message enable value");
  * Utility functions and prototypes
  *
  *************************************************************************/
-static void efx_remove_channel(struct efx_channel *channel);
+
+static void efx_remove_channels(struct efx_nic *efx);
 static void efx_remove_port(struct efx_nic *efx);
 static void efx_fini_napi(struct efx_nic *efx);
-static void efx_fini_channels(struct efx_nic *efx);
+static void efx_fini_struct(struct efx_nic *efx);
+static void efx_start_all(struct efx_nic *efx);
+static void efx_stop_all(struct efx_nic *efx);
 
 #define EFX_ASSERT_RESET_SERIALISED(efx)		\
 	do {						\
@@ -413,6 +416,63 @@ static void efx_remove_eventq(struct efx_channel *channel)
  *
  *************************************************************************/
 
+/* Allocate and initialise a channel structure, optionally copying
+ * parameters (but not resources) from an old channel structure. */
+static struct efx_channel *
+efx_alloc_channel(struct efx_nic *efx, int i, struct efx_channel *old_channel)
+{
+	struct efx_channel *channel;
+	struct efx_rx_queue *rx_queue;
+	struct efx_tx_queue *tx_queue;
+	int j;
+
+	if (old_channel) {
+		channel = kmalloc(sizeof(*channel), GFP_KERNEL);
+		if (!channel)
+			return NULL;
+
+		*channel = *old_channel;
+
+		memset(&channel->eventq, 0, sizeof(channel->eventq));
+
+		rx_queue = &channel->rx_queue;
+		rx_queue->buffer = NULL;
+		memset(&rx_queue->rxd, 0, sizeof(rx_queue->rxd));
+
+		for (j = 0; j < EFX_TXQ_TYPES; j++) {
+			tx_queue = &channel->tx_queue[j];
+			if (tx_queue->channel)
+				tx_queue->channel = channel;
+			tx_queue->buffer = NULL;
+			memset(&tx_queue->txd, 0, sizeof(tx_queue->txd));
+		}
+	} else {
+		channel = kzalloc(sizeof(*channel), GFP_KERNEL);
+		if (!channel)
+			return NULL;
+
+		channel->efx = efx;
+		channel->channel = i;
+
+		for (j = 0; j < EFX_TXQ_TYPES; j++) {
+			tx_queue = &channel->tx_queue[j];
+			tx_queue->efx = efx;
+			tx_queue->queue = i * EFX_TXQ_TYPES + j;
+			tx_queue->channel = channel;
+		}
+	}
+
+	spin_lock_init(&channel->tx_stop_lock);
+	atomic_set(&channel->tx_stop_count, 1);
+
+	rx_queue = &channel->rx_queue;
+	rx_queue->efx = efx;
+	setup_timer(&rx_queue->slow_fill, efx_rx_slow_fill,
+		    (unsigned long)rx_queue);
+
+	return channel;
+}
+
 static int efx_probe_channel(struct efx_channel *channel)
 {
 	struct efx_tx_queue *tx_queue;
@@ -469,11 +529,38 @@ static void efx_set_channel_names(struct efx_nic *efx)
 				number -= efx->n_rx_channels;
 			}
 		}
-		snprintf(channel->name, sizeof(channel->name),
+		snprintf(efx->channel_name[channel->channel],
+			 sizeof(efx->channel_name[0]),
 			 "%s%s-%d", efx->name, type, number);
 	}
 }
 
+static int efx_probe_channels(struct efx_nic *efx)
+{
+	struct efx_channel *channel;
+	int rc;
+
+	/* Restart special buffer allocation */
+	efx->next_buffer_table = 0;
+
+	efx_for_each_channel(channel, efx) {
+		rc = efx_probe_channel(channel);
+		if (rc) {
+			netif_err(efx, probe, efx->net_dev,
+				  "failed to create channel %d\n",
+				  channel->channel);
+			goto fail;
+		}
+	}
+	efx_set_channel_names(efx);
+
+	return 0;
+
+fail:
+	efx_remove_channels(efx);
+	return rc;
+}
+
 /* Channels are shutdown and reinitialised whilst the NIC is running
  * to propagate configuration changes (mtu, checksum offload), or
  * to clear hardware error conditions
@@ -611,6 +698,75 @@ static void efx_remove_channel(struct efx_channel *channel)
 	efx_remove_eventq(channel);
 }
 
+static void efx_remove_channels(struct efx_nic *efx)
+{
+	struct efx_channel *channel;
+
+	efx_for_each_channel(channel, efx)
+		efx_remove_channel(channel);
+}
+
+int
+efx_realloc_channels(struct efx_nic *efx, u32 rxq_entries, u32 txq_entries)
+{
+	struct efx_channel *other_channel[EFX_MAX_CHANNELS], *channel;
+	u32 old_rxq_entries, old_txq_entries;
+	unsigned i;
+	int rc;
+
+	efx_stop_all(efx);
+	efx_fini_channels(efx);
+
+	/* Clone channels */
+	memset(other_channel, 0, sizeof(other_channel));
+	for (i = 0; i < efx->n_channels; i++) {
+		channel = efx_alloc_channel(efx, i, efx->channel[i]);
+		if (!channel) {
+			rc = -ENOMEM;
+			goto out;
+		}
+		other_channel[i] = channel;
+	}
+
+	/* Swap entry counts and channel pointers */
+	old_rxq_entries = efx->rxq_entries;
+	old_txq_entries = efx->txq_entries;
+	efx->rxq_entries = rxq_entries;
+	efx->txq_entries = txq_entries;
+	for (i = 0; i < efx->n_channels; i++) {
+		channel = efx->channel[i];
+		efx->channel[i] = other_channel[i];
+		other_channel[i] = channel;
+	}
+
+	rc = efx_probe_channels(efx);
+	if (rc)
+		goto rollback;
+
+	/* Destroy old channels */
+	for (i = 0; i < efx->n_channels; i++)
+		efx_remove_channel(other_channel[i]);
+out:
+	/* Free unused channel structures */
+	for (i = 0; i < efx->n_channels; i++)
+		kfree(other_channel[i]);
+
+	efx_init_channels(efx);
+	efx_start_all(efx);
+	return rc;
+
+rollback:
+	/* Swap back */
+	efx->rxq_entries = old_rxq_entries;
+	efx->txq_entries = old_txq_entries;
+	for (i = 0; i < efx->n_channels; i++) {
+		channel = efx->channel[i];
+		efx->channel[i] = other_channel[i];
+		other_channel[i] = channel;
+	}
+	goto out;
+}
+
 void efx_schedule_slow_fill(struct efx_rx_queue *rx_queue)
 {
 	mod_timer(&rx_queue->slow_fill, jiffies + msecs_to_jiffies(100));
@@ -1182,41 +1338,28 @@ static void efx_remove_nic(struct efx_nic *efx)
 
 static int efx_probe_all(struct efx_nic *efx)
 {
-	struct efx_channel *channel;
 	int rc;
 
-	/* Create NIC */
 	rc = efx_probe_nic(efx);
 	if (rc) {
 		netif_err(efx, probe, efx->net_dev, "failed to create NIC\n");
 		goto fail1;
 	}
 
-	/* Create port */
 	rc = efx_probe_port(efx);
 	if (rc) {
 		netif_err(efx, probe, efx->net_dev, "failed to create port\n");
 		goto fail2;
 	}
 
-	/* Create channels */
 	efx->rxq_entries = efx->txq_entries = EFX_DEFAULT_DMAQ_SIZE;
-	efx_for_each_channel(channel, efx) {
-		rc = efx_probe_channel(channel);
-		if (rc) {
-			netif_err(efx, probe, efx->net_dev,
-				  "failed to create channel %d\n",
-				  channel->channel);
-			goto fail3;
-		}
-	}
-	efx_set_channel_names(efx);
+	rc = efx_probe_channels(efx);
+	if (rc)
+		goto fail3;
 
 	return 0;
 
  fail3:
-	efx_for_each_channel(channel, efx)
-		efx_remove_channel(channel);
 	efx_remove_port(efx);
  fail2:
 	efx_remove_nic(efx);
@@ -1346,10 +1489,7 @@ static void efx_stop_all(struct efx_nic *efx)
 
 static void efx_remove_all(struct efx_nic *efx)
 {
-	struct efx_channel *channel;
-
-	efx_for_each_channel(channel, efx)
-		efx_remove_channel(channel);
+	efx_remove_channels(efx);
 	efx_remove_port(efx);
 	efx_remove_nic(efx);
 }
@@ -2058,10 +2198,7 @@ static struct efx_phy_operations efx_dummy_phy_operations = {
 static int efx_init_struct(struct efx_nic *efx, struct efx_nic_type *type,
 			   struct pci_dev *pci_dev, struct net_device *net_dev)
 {
-	struct efx_channel *channel;
-	struct efx_tx_queue *tx_queue;
-	struct efx_rx_queue *rx_queue;
-	int i, j;
+	int i;
 
 	/* Initialise common structures */
 	memset(efx, 0, sizeof(*efx));
@@ -2089,24 +2226,9 @@ static int efx_init_struct(struct efx_nic *efx, struct efx_nic_type *type,
 	INIT_WORK(&efx->mac_work, efx_mac_work);
 
 	for (i = 0; i < EFX_MAX_CHANNELS; i++) {
-		efx->channel[i] = kzalloc(sizeof(*channel), GFP_KERNEL);
-		channel = efx->channel[i];
-		channel->efx = efx;
-		channel->channel = i;
-		spin_lock_init(&channel->tx_stop_lock);
-		atomic_set(&channel->tx_stop_count, 1);
-
-		for (j = 0; j < EFX_TXQ_TYPES; j++) {
-			tx_queue = &channel->tx_queue[j];
-			tx_queue->efx = efx;
-			tx_queue->queue = i * EFX_TXQ_TYPES + j;
-			tx_queue->channel = channel;
-		}
-
-		rx_queue = &channel->rx_queue;
-		rx_queue->efx = efx;
-		setup_timer(&rx_queue->slow_fill, efx_rx_slow_fill,
-			    (unsigned long)rx_queue);
+		efx->channel[i] = efx_alloc_channel(efx, i, NULL);
+		if (!efx->channel[i])
+			goto fail;
 	}
 
 	efx->type = type;
@@ -2122,9 +2244,13 @@ static int efx_init_struct(struct efx_nic *efx, struct efx_nic_type *type,
 		 pci_name(pci_dev));
 	efx->workqueue = create_singlethread_workqueue(efx->workqueue_name);
 	if (!efx->workqueue)
-		return -ENOMEM;
+		goto fail;
 
 	return 0;
+
+fail:
+	efx_fini_struct(efx);
+	return -ENOMEM;
 }
 
 static void efx_fini_struct(struct efx_nic *efx)
diff --git a/drivers/net/sfc/efx.h b/drivers/net/sfc/efx.h
index c15a2d3..e783c0f 100644
--- a/drivers/net/sfc/efx.h
+++ b/drivers/net/sfc/efx.h
@@ -59,8 +59,15 @@ extern void efx_schedule_slow_fill(struct efx_rx_queue *rx_queue);
 #define EFX_MAX_EVQ_SIZE 16384UL
 #define EFX_MIN_EVQ_SIZE 512UL
 
+/* The smallest [rt]xq_entries that the driver supports. Callers of
+ * efx_wake_queue() assume that they can subsequently send at least one
+ * skb. Falcon/A1 may require up to three descriptors per skb_frag. */
+#define EFX_MIN_RING_SIZE (roundup_pow_of_two(2 * 3 * MAX_SKB_FRAGS))
+
 /* Channels */
 extern void efx_process_channel_now(struct efx_channel *channel);
+extern int
+efx_realloc_channels(struct efx_nic *efx, u32 rxq_entries, u32 txq_entries);
 
 /* Ports */
 extern int efx_reconfigure_port(struct efx_nic *efx);
diff --git a/drivers/net/sfc/ethtool.c b/drivers/net/sfc/ethtool.c
index b9291db..7f735d8 100644
--- a/drivers/net/sfc/ethtool.c
+++ b/drivers/net/sfc/ethtool.c
@@ -742,6 +742,42 @@ static int efx_ethtool_set_coalesce(struct net_device *net_dev,
 	return 0;
 }
 
+static void efx_ethtool_get_ringparam(struct net_device *net_dev,
+				      struct ethtool_ringparam *ring)
+{
+	struct efx_nic *efx = netdev_priv(net_dev);
+
+	ring->rx_max_pending = EFX_MAX_DMAQ_SIZE;
+	ring->tx_max_pending = EFX_MAX_DMAQ_SIZE;
+	ring->rx_mini_max_pending = 0;
+	ring->rx_jumbo_max_pending = 0;
+	ring->rx_pending = efx->rxq_entries;
+	ring->tx_pending = efx->txq_entries;
+	ring->rx_mini_pending = 0;
+	ring->rx_jumbo_pending = 0;
+}
+
+static int efx_ethtool_set_ringparam(struct net_device *net_dev,
+				     struct ethtool_ringparam *ring)
+{
+	struct efx_nic *efx = netdev_priv(net_dev);
+
+	if (ring->rx_mini_pending || ring->rx_jumbo_pending ||
+	    ring->rx_pending > EFX_MAX_DMAQ_SIZE ||
+	    ring->tx_pending > EFX_MAX_DMAQ_SIZE)
+		return -EINVAL;
+
+	if (ring->rx_pending < EFX_MIN_RING_SIZE ||
+	    ring->tx_pending < EFX_MIN_RING_SIZE) {
+		netif_err(efx, drv, efx->net_dev,
+			  "TX and RX queues cannot be smaller than %ld\n",
+			  EFX_MIN_RING_SIZE);
+		return -EINVAL;
+	}
+
+	return efx_realloc_channels(efx, ring->rx_pending, ring->tx_pending);
+}
+
 static int efx_ethtool_set_pauseparam(struct net_device *net_dev,
 				      struct ethtool_pauseparam *pause)
 {
@@ -972,6 +1008,8 @@ const struct ethtool_ops efx_ethtool_ops = {
 	.set_eeprom		= efx_ethtool_set_eeprom,
 	.get_coalesce		= efx_ethtool_get_coalesce,
 	.set_coalesce		= efx_ethtool_set_coalesce,
+	.get_ringparam		= efx_ethtool_get_ringparam,
+	.set_ringparam		= efx_ethtool_set_ringparam,
 	.get_pauseparam         = efx_ethtool_get_pauseparam,
 	.set_pauseparam         = efx_ethtool_set_pauseparam,
 	.get_rx_csum		= efx_ethtool_get_rx_csum,
diff --git a/drivers/net/sfc/net_driver.h b/drivers/net/sfc/net_driver.h
index ac622ab..4b3f680 100644
--- a/drivers/net/sfc/net_driver.h
+++ b/drivers/net/sfc/net_driver.h
@@ -299,7 +299,6 @@ enum efx_rx_alloc_method {
  *
  * @efx: Associated Efx NIC
  * @channel: Channel instance number
- * @name: Name for channel and IRQ
  * @enabled: Channel enabled indicator
  * @irq: IRQ number (MSI and MSI-X only)
  * @irq_moderation: IRQ moderation value (in hardware ticks)
@@ -333,7 +332,6 @@ enum efx_rx_alloc_method {
 struct efx_channel {
 	struct efx_nic *efx;
 	int channel;
-	char name[IFNAMSIZ + 6];
 	bool enabled;
 	int irq;
 	unsigned int irq_moderation;
@@ -644,6 +642,7 @@ union efx_multicast_hash {
  * @tx_queue: TX DMA queues
  * @rx_queue: RX DMA queues
  * @channel: Channels
+ * @channel_name: Names for channels and their IRQs
  * @rxq_entries: Size of receive queues requested by user.
  * @txq_entries: Size of transmit queues requested by user.
  * @next_buffer_table: First available buffer table id
@@ -730,6 +729,7 @@ struct efx_nic {
 	enum reset_type reset_pending;
 
 	struct efx_channel *channel[EFX_MAX_CHANNELS];
+	char channel_name[IFNAMSIZ + 6][EFX_MAX_CHANNELS];
 
 	unsigned rxq_entries;
 	unsigned txq_entries;
diff --git a/drivers/net/sfc/nic.c b/drivers/net/sfc/nic.c
index 0deb5c3..6c5c0ce 100644
--- a/drivers/net/sfc/nic.c
+++ b/drivers/net/sfc/nic.c
@@ -1478,7 +1478,7 @@ static irqreturn_t efx_legacy_interrupt(int irq, void *dev_id)
  */
 static irqreturn_t efx_msi_interrupt(int irq, void *dev_id)
 {
-	struct efx_channel *channel = dev_id;
+	struct efx_channel *channel = *(struct efx_channel **)dev_id;
 	struct efx_nic *efx = channel->efx;
 	efx_oword_t *int_ker = efx->irq_status.addr;
 	int syserr;
@@ -1553,7 +1553,8 @@ int efx_nic_init_interrupt(struct efx_nic *efx)
 	efx_for_each_channel(channel, efx) {
 		rc = request_irq(channel->irq, efx_msi_interrupt,
 				 IRQF_PROBE_SHARED, /* Not shared */
-				 channel->name, channel);
+				 efx->channel_name[channel->channel],
+				 &efx->channel[channel->channel]);
 		if (rc) {
 			netif_err(efx, drv, efx->net_dev,
 				  "failed to hook IRQ %d\n", channel->irq);
@@ -1565,7 +1566,7 @@ int efx_nic_init_interrupt(struct efx_nic *efx)
 
  fail2:
 	efx_for_each_channel(channel, efx)
-		free_irq(channel->irq, channel);
+		free_irq(channel->irq, &efx->channel[channel->channel]);
  fail1:
 	return rc;
 }
@@ -1578,7 +1579,7 @@ void efx_nic_fini_interrupt(struct efx_nic *efx)
 	/* Disable MSI/MSI-X interrupts */
 	efx_for_each_channel(channel, efx) {
 		if (channel->irq)
-			free_irq(channel->irq, channel);
+			free_irq(channel->irq, &efx->channel[channel->channel]);
 	}
 
 	/* ACK legacy interrupt */
-- 
1.7.2.1


-- 
Ben Hutchings, Senior Software Engineer, Solarflare Communications
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.

Re: [PATCH net-next-2.6 00/10] sfc changes for 2.6.37

[David Miller]

From: Ben Hutchings <bhutchings@solarflare.com>
Date: Fri, 10 Sep 2010 17:40:17 +0100

> One new feature - variable ring sizes - and a few bug fixes.
> 
> Ben.
> 
> Ben Hutchings (9):
>   sfc: Use MCDI RX_BAD_FCS_PKTS count as MAC rx_bad count
>   sfc: Accumulate RX_NODESC_DROP count in rx_dropped, not
>     rx_over_errors
>   sfc: Remove declarations of functions that no longer exist
>   sfc: Fix failure paths in efx_probe_port()
>   sfc: Allocate DMA and event rings using GFP_KERNEL
>   sfc: Abstract channel and index lookup for RX queues
>   sfc: Refactor channel and queue lookup and iteration
>   sfc: Allocate each channel separately, along with its RX and TX
>     queues
>   sfc: Allow changing the DMA ring sizes dynamically via ethtool
> 
> Steve Hodgson (1):
>   sfc: Make the dmaq size a run-time setting (rather than compile-time)

All applied, thanks Ben.

It could be argued that rx_over_errors is meant to count internal chip
FIFO overflows during receive, which occur when DMA cannot meet demand
for long periods of time.  Indicating that FIFO settings need to be
increased or PCI settings adjusted.

Anyways, just my view.

Re: [PATCH net-next-2.6 00/10] sfc changes for 2.6.37

[Ben Hutchings]

On Fri, 2010-09-10 at 12:28 -0700, David Miller wrote:
> From: Ben Hutchings <bhutchings@solarflare.com>
> Date: Fri, 10 Sep 2010 17:40:17 +0100
> 
> > One new feature - variable ring sizes - and a few bug fixes.
> > 
> > Ben.
> > 
> > Ben Hutchings (9):
> >   sfc: Use MCDI RX_BAD_FCS_PKTS count as MAC rx_bad count
> >   sfc: Accumulate RX_NODESC_DROP count in rx_dropped, not
> >     rx_over_errors
> >   sfc: Remove declarations of functions that no longer exist
> >   sfc: Fix failure paths in efx_probe_port()
> >   sfc: Allocate DMA and event rings using GFP_KERNEL
> >   sfc: Abstract channel and index lookup for RX queues
> >   sfc: Refactor channel and queue lookup and iteration
> >   sfc: Allocate each channel separately, along with its RX and TX
> >     queues
> >   sfc: Allow changing the DMA ring sizes dynamically via ethtool
> > 
> > Steve Hodgson (1):
> >   sfc: Make the dmaq size a run-time setting (rather than compile-time)
> 
> All applied, thanks Ben.
> 
> It could be argued that rx_over_errors is meant to count internal chip
> FIFO overflows during receive, which occur when DMA cannot meet demand
> for long periods of time.  Indicating that FIFO settings need to be
> increased or PCI settings adjusted.
> 
> Anyways, just my view.

It would really be nice to have more verbose definitions of these in
rtnetlink.h.  The decider for us was that ifconfig includes
rx_over_errors in RX frame errors i.e. bad packets.

Ben.

-- 
Ben Hutchings, Senior Software Engineer, Solarflare Communications
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.

Re: [PATCH net-next-2.6 00/10] sfc changes for 2.6.37

[David Miller]

From: Ben Hutchings <bhutchings@solarflare.com>
Date: Fri, 10 Sep 2010 20:59:50 +0100

[ talking about standard netdevice statistic meanings ]

> It would really be nice to have more verbose definitions of these in
> rtnetlink.h.  The decider for us was that ifconfig includes
> rx_over_errors in RX frame errors i.e. bad packets.

I completely agree.