From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pf0-f196.google.com ([209.85.192.196]:36711 "EHLO mail-pf0-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933390AbeBVRen (ORCPT ); Thu, 22 Feb 2018 12:34:43 -0500 Received: by mail-pf0-f196.google.com with SMTP id 68so2357327pfx.3 for ; Thu, 22 Feb 2018 09:34:42 -0800 (PST) Date: Thu, 22 Feb 2018 10:34:40 -0700 From: Jason Gunthorpe To: Saeed Mahameed Cc: "David S. Miller" , Doug Ledford , Leon Romanovsky , netdev@vger.kernel.org, linux-rdma@vger.kernel.org, Yonatan Cohen , Leon Romanovsky Subject: Re: [for-next 7/7] IB/mlx5: Implement fragmented completion queue (CQ) Message-ID: <20180222173440.d5pjqbrjps5g2czk@ziepe.ca> References: <20180221201354.8802-1-saeedm@mellanox.com> <20180221201354.8802-8-saeedm@mellanox.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20180221201354.8802-8-saeedm@mellanox.com> Sender: netdev-owner@vger.kernel.org List-ID: On Wed, Feb 21, 2018 at 12:13:54PM -0800, Saeed Mahameed wrote: > From: Yonatan Cohen > > The current implementation of create CQ requires contiguous > memory, such requirement is problematic once the memory is > fragmented or the system is low in memory, it causes for > failures in dma_zalloc_coherent(). > > This patch implements new scheme of fragmented CQ to overcome > this issue by introducing new type: 'struct mlx5_frag_buf_ctrl' > to allocate fragmented buffers, rather than contiguous ones. > > Base the Completion Queues (CQs) on this new fragmented buffer. > > It fixes following crashes: > kworker/29:0: page allocation failure: order:6, mode:0x80d0 > CPU: 29 PID: 8374 Comm: kworker/29:0 Tainted: G OE 3.10.0 > Workqueue: ib_cm cm_work_handler [ib_cm] > Call Trace: > [<>] dump_stack+0x19/0x1b > [<>] warn_alloc_failed+0x110/0x180 > [<>] __alloc_pages_slowpath+0x6b7/0x725 > [<>] __alloc_pages_nodemask+0x405/0x420 > [<>] dma_generic_alloc_coherent+0x8f/0x140 > [<>] x86_swiotlb_alloc_coherent+0x21/0x50 > [<>] mlx5_dma_zalloc_coherent_node+0xad/0x110 [mlx5_core] > [<>] ? mlx5_db_alloc_node+0x69/0x1b0 [mlx5_core] > [<>] mlx5_buf_alloc_node+0x3e/0xa0 [mlx5_core] > [<>] mlx5_buf_alloc+0x14/0x20 [mlx5_core] > [<>] create_cq_kernel+0x90/0x1f0 [mlx5_ib] > [<>] mlx5_ib_create_cq+0x3b0/0x4e0 [mlx5_ib] > > Signed-off-by: Yonatan Cohen > Reviewed-by: Tariq Toukan > Signed-off-by: Leon Romanovsky > Signed-off-by: Saeed Mahameed > drivers/infiniband/hw/mlx5/cq.c | 64 +++++++++++++++---------- > drivers/infiniband/hw/mlx5/mlx5_ib.h | 6 +-- > drivers/net/ethernet/mellanox/mlx5/core/alloc.c | 37 +++++++++----- > drivers/net/ethernet/mellanox/mlx5/core/en_rx.c | 11 +++-- > drivers/net/ethernet/mellanox/mlx5/core/wq.c | 18 +++---- > drivers/net/ethernet/mellanox/mlx5/core/wq.h | 22 +++------ > include/linux/mlx5/driver.h | 51 ++++++++++++++------ > 7 files changed, 124 insertions(+), 85 deletions(-) For the drivers/infiniband stuff: Acked-by: Jason Gunthorpe Thanks, Jason