RE: [PATCH v2] bnx2x: Alloc 4k fragment for each rx ring buffer element

[Yuval Mintz <Yuval.Mintz@qlogic.com>]

 The driver allocates one page for each buffer on the rx ring, which is too much
> on architectures like ppc64 and can cause unexpected allocation failures when
> the system is under stress.  Now, we keep a memory pool per queue, and if the
> architecture's PAGE_SIZE is greater than 4k, we fragment pages and assign each
> 4k segment to a ring element, which reduces the overall memory consumption
> on such architectures.  This helps avoiding errors like the example below:
> 
> [bnx2x_alloc_rx_sge:435(eth1)]Can't alloc sge [c00000037ffeb900]
> [d000000075eddeb4] .bnx2x_alloc_rx_sge+0x44/0x200 [bnx2x]
> [c00000037ffeb9b0] [d000000075ee0b34] .bnx2x_fill_frag_skb+0x1ac/0x460
> [bnx2x] [c00000037ffebac0] [d000000075ee11f0]
> .bnx2x_tpa_stop+0x160/0x2e8 [bnx2x] [c00000037ffebb90]
> [d000000075ee1560] .bnx2x_rx_int+0x1e8/0xc30 [bnx2x] [c00000037ffebcd0]
> [d000000075ee2084] .bnx2x_poll+0xdc/0x3d8 [bnx2x] (unreliable)
> 
> Signed-off-by: Gabriel Krisman Bertazi <krisman@linux.vnet.ibm.com>
> ---
>  drivers/net/ethernet/broadcom/bnx2x/bnx2x.h     | 19 ++++++--
>  drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.c | 61 +++++++++++++++++-
> -------  drivers/net/ethernet/broadcom/bnx2x/bnx2x_cmn.h | 33 +++++++++++--
>  3 files changed, 89 insertions(+), 24 deletions(-)
> 
> diff --git a/drivers/net/ethernet/broadcom/bnx2x/bnx2x.h
> b/drivers/net/ethernet/broadcom/bnx2x/bnx2x.h
> index 4085c4b..d0c8ed0 100644
> --- a/drivers/net/ethernet/broadcom/bnx2x/bnx2x.h
> +++ b/drivers/net/ethernet/broadcom/bnx2x/bnx2x.h
> @@ -356,6 +356,8 @@ struct sw_tx_bd {
> 
 ...
> +struct bnx2x_alloc_pool {
> +	struct page	*page;
> +	dma_addr_t	dma;
> +	int		len;

Isn't len always set to PAGE_SIZE? If so it can be dropped

> +	int		offset;
> +	int		last_offset;

Looks like you last_offset is used only for comparisons with offset;
Can we make it into a counter of possible allocations instead?

I.e., set it to (PAGE_SIZE / SGE_PAGE_SIZE - 1) during allocation
And decrement it each time an SGE_PAGE_SIZE chunk is taken.
That would turn (pool-> offset > pool->last_offset) into (!remain).