From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.2 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A4836C4361B for ; Wed, 16 Dec 2020 08:19:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 48EFC23356 for ; Wed, 16 Dec 2020 08:19:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725831AbgLPIS7 (ORCPT ); Wed, 16 Dec 2020 03:18:59 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39404 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725274AbgLPIS7 (ORCPT ); Wed, 16 Dec 2020 03:18:59 -0500 Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D75B8C0613D6; Wed, 16 Dec 2020 00:18:18 -0800 (PST) Received: by mail-wm1-x330.google.com with SMTP id 3so1520187wmg.4; Wed, 16 Dec 2020 00:18:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:mail-followup-to:references :mime-version:content-disposition:in-reply-to:user-agent; bh=OsOzas/w/eSfqOtHOGgepo3zxkPHG/qg3EkWGzcgIY8=; b=WesLM46cYqkvDM11I0wuqigkm5AWitGsfCK7aWBAgxMicsGw6+HNROPhOlCWYL+iEJ WiDhj9N+GG03RaFy4poJjsDtzf5XwkjDgOeeSXXsgOENSBA0pG/eWY3o9XGRIcLeqfHS pHW2ts22TPRs1ra5hjHSqEov/Qw+eMVbMcueZ3mMZA+P7+NDc5eYsAsMVme8AQUKKqgN Cfpkb245c7TRWR+rVWSEUYIk1KBXNRIiE/TRkSNy8TZtNiXW5V4gEP87FX/2DodreYcB M/rDiCOCjxyhJfLX/aOklf0FK8yTfNrMMEDlP+FCURLVxfr7HRP2DQk6Ke6JsFAq5bqJ Nejw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id :mail-followup-to:references:mime-version:content-disposition :in-reply-to:user-agent; bh=OsOzas/w/eSfqOtHOGgepo3zxkPHG/qg3EkWGzcgIY8=; b=sD57vdezXni8rOh4zHo1BIeHATw/d14GscKaAV7dLiNDh+AkTKY4kP0bF8Ck3S2mYl IU4WOuvH/XPIkYWzEa5QdIxIfS6PbSQiwbemjtef2P5+R/yjcDcGyQCRU3Q8QUvLUOsA qW05ea2LJv3fDbzHF06VN9+twdrgntVArzh2Qld5JfWAKjfn/K0lwENUmu0cy4YMoKLo IXKzT8C4SpSHPAxl3kqtgM6SWUVTcRXJto59MYHwwXy4xAnfHPH+FyzGzjaxNNduwSeR NZ84by2u9AZrirWR/mehRZIdItdDwLYO3DrEUMUwYlrZ9hZLPcquYRz+h+9RjV97D1tH aJ5Q== X-Gm-Message-State: AOAM532EtyOZkg2TnhANbrOxNLLLOJLIK8uxXsUucfjfdxx4kSo5gGxx 2XOy0hQw58gY8Byt8WbInU8= X-Google-Smtp-Source: ABdhPJzHeFoP5KZ4/1PKPsAcS1CL9F0trZ83qjPMopC7aPN6AW/HZ02O8UE++K+AEq681Fu5ROling== X-Received: by 2002:a1c:6208:: with SMTP id w8mr2068448wmb.96.1608106697688; Wed, 16 Dec 2020 00:18:17 -0800 (PST) Received: from gmail.com ([81.168.73.77]) by smtp.gmail.com with ESMTPSA id i16sm1908890wrx.89.2020.12.16.00.18.16 (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 16 Dec 2020 00:18:17 -0800 (PST) Date: Wed, 16 Dec 2020 08:18:14 +0000 From: Martin Habets To: Ivan Babrou Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org, kernel-team@cloudflare.com, Edward Cree , "David S. Miller" , Jakub Kicinski , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend Subject: Re: [PATCH net-next] sfc: reduce the number of requested xdp ev queues Message-ID: <20201216081814.jcwq6xzdxur5xm4l@gmail.com> Mail-Followup-To: Ivan Babrou , netdev@vger.kernel.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org, kernel-team@cloudflare.com, Edward Cree , "David S. Miller" , Jakub Kicinski , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend References: <20201215012907.3062-1-ivan@cloudflare.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20201215012907.3062-1-ivan@cloudflare.com> User-Agent: NeoMutt/20170113 (1.7.2) Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org On Mon, Dec 14, 2020 at 05:29:06PM -0800, Ivan Babrou wrote: > Without this change the driver tries to allocate too many queues, > breaching the number of available msi-x interrupts on machines > with many logical cpus and default adapter settings: > > Insufficient resources for 12 XDP event queues (24 other channels, max 32) > > Which in turn triggers EINVAL on XDP processing: > > sfc 0000:86:00.0 ext0: XDP TX failed (-22) > > Signed-off-by: Ivan Babrou Acked-by: Martin Habets > --- > drivers/net/ethernet/sfc/efx_channels.c | 6 ++++-- > 1 file changed, 4 insertions(+), 2 deletions(-) > > diff --git a/drivers/net/ethernet/sfc/efx_channels.c b/drivers/net/ethernet/sfc/efx_channels.c > index a4a626e9cd9a..1bfeee283ea9 100644 > --- a/drivers/net/ethernet/sfc/efx_channels.c > +++ b/drivers/net/ethernet/sfc/efx_channels.c > @@ -17,6 +17,7 @@ > #include "rx_common.h" > #include "nic.h" > #include "sriov.h" > +#include "workarounds.h" > > /* This is the first interrupt mode to try out of: > * 0 => MSI-X > @@ -137,6 +138,7 @@ static int efx_allocate_msix_channels(struct efx_nic *efx, > { > unsigned int n_channels = parallelism; > int vec_count; > + int tx_per_ev; > int n_xdp_tx; > int n_xdp_ev; > > @@ -149,9 +151,9 @@ static int efx_allocate_msix_channels(struct efx_nic *efx, > * multiple tx queues, assuming tx and ev queues are both > * maximum size. > */ > - > + tx_per_ev = EFX_MAX_EVQ_SIZE / EFX_TXQ_MAX_ENT(efx); > n_xdp_tx = num_possible_cpus(); > - n_xdp_ev = DIV_ROUND_UP(n_xdp_tx, EFX_MAX_TXQ_PER_CHANNEL); > + n_xdp_ev = DIV_ROUND_UP(n_xdp_tx, tx_per_ev); > > vec_count = pci_msix_vec_count(efx->pci_dev); > if (vec_count < 0) > -- > 2.29.2 -- Martin Habets