The two architectures that define CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS (powerpc and x86) now both define NET_IP_ALIGN as 0, so there is no need for this optimisation any more. Signed-off-by: Ben Hutchings <bhutchings@solarflare.com> --- drivers/net/ethernet/sfc/efx.c | 4 ++-- drivers/net/ethernet/sfc/net_driver.h | 15 +-------------- drivers/net/ethernet/sfc/rx.c | 6 +++--- 3 files changed, 6 insertions(+), 19 deletions(-) diff --git a/drivers/net/ethernet/sfc/efx.c b/drivers/net/ethernet/sfc/efx.c index 01b9920..999289b 100644 --- a/drivers/net/ethernet/sfc/efx.c +++ b/drivers/net/ethernet/sfc/efx.c @@ -638,13 +638,13 @@ static void efx_start_datapath(struct efx_nic *efx) EFX_MAX_FRAME_LEN(efx->net_dev->mtu) + efx->type->rx_buffer_padding); rx_buf_len = (sizeof(struct efx_rx_page_state) + - EFX_PAGE_IP_ALIGN + efx->rx_dma_len); + NET_IP_ALIGN + efx->rx_dma_len); if (rx_buf_len <= PAGE_SIZE) { efx->rx_scatter = false; efx->rx_buffer_order = 0; } else if (efx->type->can_rx_scatter) { BUILD_BUG_ON(sizeof(struct efx_rx_page_state) + - EFX_PAGE_IP_ALIGN + EFX_RX_USR_BUF_SIZE > + NET_IP_ALIGN + EFX_RX_USR_BUF_SIZE > PAGE_SIZE / 2); efx->rx_scatter = true; efx->rx_dma_len = EFX_RX_USR_BUF_SIZE; diff --git a/drivers/net/ethernet/sfc/net_driver.h b/drivers/net/ethernet/sfc/net_driver.h index 9bd433a..5efddf3 100644 --- a/drivers/net/ethernet/sfc/net_driver.h +++ b/drivers/net/ethernet/sfc/net_driver.h @@ -468,24 +468,11 @@ enum nic_state { }; /* - * Alignment of page-allocated RX buffers - * - * Controls the number of bytes inserted at the start of an RX buffer. - * This is the equivalent of NET_IP_ALIGN [which controls the alignment - * of the skb->head for hardware DMA]. - */ -#ifdef CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS -#define EFX_PAGE_IP_ALIGN 0 -#else -#define EFX_PAGE_IP_ALIGN NET_IP_ALIGN -#endif - -/* * Alignment of the skb->head which wraps a page-allocated RX buffer * * The skb allocated to wrap an rx_buffer can have this alignment. Since * the data is memcpy'd from the rx_buf, it does not need to be equal to - * EFX_PAGE_IP_ALIGN. + * NET_IP_ALIGN. */ #define EFX_PAGE_SKB_ALIGN 2 diff --git a/drivers/net/ethernet/sfc/rx.c b/drivers/net/ethernet/sfc/rx.c index e73e30b..99f70dd 100644 --- a/drivers/net/ethernet/sfc/rx.c +++ b/drivers/net/ethernet/sfc/rx.c @@ -93,7 +93,7 @@ static inline void efx_sync_rx_buffer(struct efx_nic *efx, void efx_rx_config_page_split(struct efx_nic *efx) { - efx->rx_page_buf_step = ALIGN(efx->rx_dma_len + EFX_PAGE_IP_ALIGN, + efx->rx_page_buf_step = ALIGN(efx->rx_dma_len + NET_IP_ALIGN, L1_CACHE_BYTES); efx->rx_bufs_per_page = efx->rx_buffer_order ? 1 : ((PAGE_SIZE - sizeof(struct efx_rx_page_state)) / @@ -188,9 +188,9 @@ static int efx_init_rx_buffers(struct efx_rx_queue *rx_queue) do { index = rx_queue->added_count & rx_queue->ptr_mask; rx_buf = efx_rx_buffer(rx_queue, index); - rx_buf->dma_addr = dma_addr + EFX_PAGE_IP_ALIGN; + rx_buf->dma_addr = dma_addr + NET_IP_ALIGN; rx_buf->page = page; - rx_buf->page_offset = page_offset + EFX_PAGE_IP_ALIGN; + rx_buf->page_offset = page_offset + NET_IP_ALIGN; rx_buf->len = efx->rx_dma_len; rx_buf->flags = 0; ++rx_queue->added_count; -- Ben Hutchings, Staff Engineer, Solarflare Not speaking for my employer; that's the marketing department's job. They asked us to note that Solarflare product names are trademarked.
efx_start_datapath() asserts that we can fit 2 RX scatter buffers plus a software structure, each appropriately aligned, into a single page. Where L1_CACHE_BYTES == 256 and PAGE_SIZE == 4096, which is the case on s390, this assertion fails. The current scatter buffer size is also not a multiple of 64 or 128, which are more common cache line sizes. If we can make both the start and end of a scatter buffer cache-aligned, this will reduce the need for read-modify-write operations on inter- processor links. Fix the alignment by reducing EFX_RX_USR_BUF_SIZE to 2048 - 256 == 1792. (We could use 2048 - L1_CACHE_BYTES, but EFX_RX_USR_BUF_SIZE also affects user-level networking where a larger amount of housekeeping data may be needed. Although this version of the driver does not support user-level networking, I prefer to keep scattering behaviour consistent with the out-of-tree version.) This still doesn't fix the s390 build because like most architectures it has NET_IP_ALIGN == 2. When NET_IP_ALIGN != 0 we cannot achieve cache line alignment at either the start or end of a scatter buffer, so there is actually no point in padding the buffers to a multiple of the cache line size. All we need is 4-byte alignment of the network header, so do that. Adjust the assertions accordingly. Reported-by: Geert Uytterhoeven <geert@linux-m68k.org> Reported-by: Heiko Carstens <heiko.carstens@de.ibm.com> Signed-off-by: Ben Hutchings <bhutchings@solarflare.com> --- drivers/net/ethernet/sfc/efx.c | 6 ++++-- drivers/net/ethernet/sfc/net_driver.h | 16 ++++++++++++++-- drivers/net/ethernet/sfc/rx.c | 2 +- 3 files changed, 19 insertions(+), 5 deletions(-) diff --git a/drivers/net/ethernet/sfc/efx.c b/drivers/net/ethernet/sfc/efx.c index 999289b..39e4cb3 100644 --- a/drivers/net/ethernet/sfc/efx.c +++ b/drivers/net/ethernet/sfc/efx.c @@ -643,9 +643,11 @@ static void efx_start_datapath(struct efx_nic *efx) efx->rx_scatter = false; efx->rx_buffer_order = 0; } else if (efx->type->can_rx_scatter) { + BUILD_BUG_ON(EFX_RX_USR_BUF_SIZE % L1_CACHE_BYTES); BUILD_BUG_ON(sizeof(struct efx_rx_page_state) + - NET_IP_ALIGN + EFX_RX_USR_BUF_SIZE > - PAGE_SIZE / 2); + 2 * ALIGN(NET_IP_ALIGN + EFX_RX_USR_BUF_SIZE, + EFX_RX_BUF_ALIGNMENT) > + PAGE_SIZE); efx->rx_scatter = true; efx->rx_dma_len = EFX_RX_USR_BUF_SIZE; efx->rx_buffer_order = 0; diff --git a/drivers/net/ethernet/sfc/net_driver.h b/drivers/net/ethernet/sfc/net_driver.h index 5efddf3..39d6bd77 100644 --- a/drivers/net/ethernet/sfc/net_driver.h +++ b/drivers/net/ethernet/sfc/net_driver.h @@ -72,8 +72,20 @@ /* Maximum possible MTU the driver supports */ #define EFX_MAX_MTU (9 * 1024) -/* Size of an RX scatter buffer. Small enough to pack 2 into a 4K page. */ -#define EFX_RX_USR_BUF_SIZE 1824 +/* Size of an RX scatter buffer. Small enough to pack 2 into a 4K page, + * and should be a multiple of the cache line size. + */ +#define EFX_RX_USR_BUF_SIZE (2048 - 256) + +/* If possible, we should ensure cache line alignment at start and end + * of every buffer. Otherwise, we just need to ensure 4-byte + * alignment of the network header. + */ +#if NET_IP_ALIGN == 0 +#define EFX_RX_BUF_ALIGNMENT L1_CACHE_BYTES +#else +#define EFX_RX_BUF_ALIGNMENT 4 +#endif /* Forward declare Precision Time Protocol (PTP) support structure. */ struct efx_ptp_data; diff --git a/drivers/net/ethernet/sfc/rx.c b/drivers/net/ethernet/sfc/rx.c index 99f70dd..a7dfe36 100644 --- a/drivers/net/ethernet/sfc/rx.c +++ b/drivers/net/ethernet/sfc/rx.c @@ -94,7 +94,7 @@ static inline void efx_sync_rx_buffer(struct efx_nic *efx, void efx_rx_config_page_split(struct efx_nic *efx) { efx->rx_page_buf_step = ALIGN(efx->rx_dma_len + NET_IP_ALIGN, - L1_CACHE_BYTES); + EFX_RX_BUF_ALIGNMENT); efx->rx_bufs_per_page = efx->rx_buffer_order ? 1 : ((PAGE_SIZE - sizeof(struct efx_rx_page_state)) / efx->rx_page_buf_step); -- Ben Hutchings, Staff Engineer, Solarflare Not speaking for my employer; that's the marketing department's job. They asked us to note that Solarflare product names are trademarked.
On Tue, May 14, 2013 at 12:01 AM, Ben Hutchings
<bhutchings@solarflare.com> wrote:
> efx_start_datapath() asserts that we can fit 2 RX scatter buffers plus
> a software structure, each appropriately aligned, into a single page.
> Where L1_CACHE_BYTES == 256 and PAGE_SIZE == 4096, which is the case
> on s390, this assertion fails.
>
> The current scatter buffer size is also not a multiple of 64 or 128,
> which are more common cache line sizes. If we can make both the start
> and end of a scatter buffer cache-aligned, this will reduce the need
> for read-modify-write operations on inter- processor links.
>
> Fix the alignment by reducing EFX_RX_USR_BUF_SIZE to 2048 - 256 ==
> 1792. (We could use 2048 - L1_CACHE_BYTES, but EFX_RX_USR_BUF_SIZE
> also affects user-level networking where a larger amount of
> housekeeping data may be needed. Although this version of the driver
> does not support user-level networking, I prefer to keep scattering
> behaviour consistent with the out-of-tree version.)
>
> This still doesn't fix the s390 build because like most architectures
> it has NET_IP_ALIGN == 2. When NET_IP_ALIGN != 0 we cannot achieve
> cache line alignment at either the start or end of a scatter buffer,
> so there is actually no point in padding the buffers to a multiple of
> the cache line size. All we need is 4-byte alignment of the network
> header, so do that.
>
> Adjust the assertions accordingly.
>
> Reported-by: Geert Uytterhoeven <geert@linux-m68k.org>
> Reported-by: Heiko Carstens <heiko.carstens@de.ibm.com>
> Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
Fixes the s390 build, so
Acked-by: Geert Uytterhoeven <geert@linux-m68k.org>
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
> The two architectures that define CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS
> (powerpc and x86) now both define NET_IP_ALIGN as 0, so there is no
> need for this optimisation any more.
Hmmm.... even on x86 there will be a measurable cost
in misaligned accesses - at least for some workloads.
If the DMA is able to write to a mis-aligned buffer and
still perform aligned burst transfers mid-frame then
4n+2 aligning the rx buffer should be a win even on x86.
Note to hardware engineers: add an option to write two
bytes of junk before the rx data :-)
David
On Tue, 2013-05-14 at 09:32 +0100, David Laight wrote: > > The two architectures that define CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS > > (powerpc and x86) now both define NET_IP_ALIGN as 0, so there is no > > need for this optimisation any more. > > Hmmm.... even on x86 there will be a measurable cost > in misaligned accesses - at least for some workloads. When they cross cache-line boundaries, yes. Maybe napi_get_frags() should be adding the 2 byte offset on all architectures, as the skbs it allocates are never RX DMA buffers. > If the DMA is able to write to a mis-aligned buffer and > still perform aligned burst transfers mid-frame then > 4n+2 aligning the rx buffer should be a win even on x86. I don't think so. > Note to hardware engineers: add an option to write two > bytes of junk before the rx data :-) There is some hardware with that option. Ben. -- Ben Hutchings, Staff Engineer, Solarflare Not speaking for my employer; that's the marketing department's job. They asked us to note that Solarflare product names are trademarked.
From: Ben Hutchings <bhutchings@solarflare.com>
Date: Mon, 13 May 2013 22:58:31 +0100
> The two architectures that define CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS
> (powerpc and x86) now both define NET_IP_ALIGN as 0, so there is no
> need for this optimisation any more.
>
> Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
Applied.
From: Ben Hutchings <bhutchings@solarflare.com>
Date: Mon, 13 May 2013 23:01:22 +0100
> efx_start_datapath() asserts that we can fit 2 RX scatter buffers plus
> a software structure, each appropriately aligned, into a single page.
> Where L1_CACHE_BYTES == 256 and PAGE_SIZE == 4096, which is the case
> on s390, this assertion fails.
>
> The current scatter buffer size is also not a multiple of 64 or 128,
> which are more common cache line sizes. If we can make both the start
> and end of a scatter buffer cache-aligned, this will reduce the need
> for read-modify-write operations on inter- processor links.
>
> Fix the alignment by reducing EFX_RX_USR_BUF_SIZE to 2048 - 256 ==
> 1792. (We could use 2048 - L1_CACHE_BYTES, but EFX_RX_USR_BUF_SIZE
> also affects user-level networking where a larger amount of
> housekeeping data may be needed. Although this version of the driver
> does not support user-level networking, I prefer to keep scattering
> behaviour consistent with the out-of-tree version.)
>
> This still doesn't fix the s390 build because like most architectures
> it has NET_IP_ALIGN == 2. When NET_IP_ALIGN != 0 we cannot achieve
> cache line alignment at either the start or end of a scatter buffer,
> so there is actually no point in padding the buffers to a multiple of
> the cache line size. All we need is 4-byte alignment of the network
> header, so do that.
>
> Adjust the assertions accordingly.
>
> Reported-by: Geert Uytterhoeven <geert@linux-m68k.org>
> Reported-by: Heiko Carstens <heiko.carstens@de.ibm.com>
> Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
Applied.
On Mon, 2013-05-13 at 23:01 +0100, Ben Hutchings wrote:
[...]
> This still doesn't fix the s390 build because like most architectures
> it has NET_IP_ALIGN == 2. When NET_IP_ALIGN != 0 we cannot achieve
> cache line alignment at either the start or end of a scatter buffer,
> so there is actually no point in padding the buffers to a multiple of
> the cache line size. All we need is 4-byte alignment of the network
> header, so do that.
[...]
I was just thinking about this again. Do we need to care about
alignment in a page fragment at all? Won't all the headers be pulled
into the header area before being accessed with any instructions that
would require alignment? (We aren't using build_skb().)
Ben.
--
Ben Hutchings, Staff Engineer, Solarflare
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.