From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1D42C4CECE for ; Sat, 12 Oct 2019 11:53:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id BAACB2087E for ; Sat, 12 Oct 2019 11:53:58 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=dlink.ru header.i=@dlink.ru header.b="R1cXAHL0" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729080AbfJLLx4 (ORCPT ); Sat, 12 Oct 2019 07:53:56 -0400 Received: from fd.dlink.ru ([178.170.168.18]:46642 "EHLO fd.dlink.ru" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727231AbfJLLv4 (ORCPT ); Sat, 12 Oct 2019 07:51:56 -0400 Received: by fd.dlink.ru (Postfix, from userid 5000) id 944891B20983; Sat, 12 Oct 2019 14:51:52 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 fd.dlink.ru 944891B20983 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dlink.ru; s=mail; t=1570881112; bh=GVdyMAkJoaCzEF7enJYVSyxNIEivWNJHyH4jL2SwDe8=; h=Date:From:To:Cc:Subject:In-Reply-To:References; b=R1cXAHL0FigCYkYf211zc5POTUXLGBUKygLqCdjJb0OCLYxcHzAAIjkjk1ywkK+g1 T3iDsNnAVd8/6r/UT04PVGfoXRm/p9P6NslcMsy3mZ9VzalPDyZPU4h6uopD78ewsk N70f9jDbkBTYjPlnG1fjMPkUNIv+qF8Bg/d9hWSI= Received: from mail.rzn.dlink.ru (mail.rzn.dlink.ru [178.170.168.13]) by fd.dlink.ru (Postfix) with ESMTP id 388411B202D2; Sat, 12 Oct 2019 14:51:49 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 fd.dlink.ru 388411B202D2 Received: by mail.rzn.dlink.ru (Postfix, from userid 5000) id 232351B21890; Sat, 12 Oct 2019 14:51:49 +0300 (MSK) Received: from mail.rzn.dlink.ru (localhost [127.0.0.1]) by mail.rzn.dlink.ru (Postfix) with ESMTPA id 7A6C71B2120F; Sat, 12 Oct 2019 14:51:41 +0300 (MSK) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Sat, 12 Oct 2019 14:51:41 +0300 From: Alexander Lobakin To: Eric Dumazet Cc: Edward Cree , "David S. Miller" , Jiri Pirko , Ido Schimmel , Paolo Abeni , Petr Machata , Sabrina Dubroca , Florian Fainelli , Jassi Brar , Ilias Apalodimas , netdev , LKML Subject: Re: [PATCH net-next 2/2] net: core: increase the default size of GRO_NORMAL skb lists to flush In-Reply-To: References: <20191010144226.4115-1-alobakin@dlink.ru> <20191010144226.4115-3-alobakin@dlink.ru> <1eaac2e1f1d65194a4a39232d7e45870@dlink.ru> <3c459c84df86f79b593632d3f08d5f4c@dlink.ru> Message-ID: X-Sender: alobakin@dlink.ru User-Agent: Roundcube Webmail/1.3.6 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Hi Eric, Eric Dumazet wrote 12.10.2019 14:18: > On Sat, Oct 12, 2019 at 2:22 AM Alexander Lobakin > wrote: > >> >> I've generated an another solution. Considering that gro_normal_batch >> is very individual for every single case, maybe it would be better to >> make it per-NAPI (or per-netdevice) variable rather than a global >> across the kernel? >> I think most of all network-capable configurations and systems has >> more >> than one network device nowadays, and they might need different values >> for achieving their bests. >> >> One possible variant is: >> >> #define THIS_DRIVER_GRO_NORMAL_BATCH 16 >> >> /* ... */ >> >> netif_napi_add(dev, napi, this_driver_rx_poll, NAPI_POLL_WEIGHT); /* >> napi->gro_normal_batch will be set to the systcl value during NAPI >> context initialization */ >> napi_set_gro_normal_batch(napi, THIS_DRIVER_GRO_NORMAL_BATCH); /* new >> static inline helper, napi->gro_normal_batch will be set to the >> driver-speficic value of 16 */ >> >> The second possible variant is to make gro_normal_batch sysctl >> per-netdevice to tune it from userspace. >> Or we can combine them into one to make it available for tweaking from >> both driver and userspace, just like it's now with XPS CPUs setting. >> >> If you'll find any of this reasonable and worth implementing, I'll >> come >> with it in v2 after a proper testing. > > Most likely the optimal tuning is also a function of the host cpu > caches. > > Building a too big list can also lead to premature cache evictions. > > Tuning the value on your test machines does not mean the value will be > good > for other systems. Oh, I missed that it might be a lot more machine-dependent than netdevice-dependent. Thank you for explanation. The best I can do in that case is to leave batch control in its current. I'll publish v2 containing only the acked first part of the series on Monday if nothing serious will happen. Addition of listified Rx to napi_gro_receive() was the main goal anyway. > > Adding yet another per device value should only be done if you > demonstrate > a significant performance increase compared to the conservative value > Edward chose. > > Also the behavior can be quite different depending on the protocols, > make sure you test handling of TCP pure ACK packets. > > Accumulating 64 (in case the device uses standard NAPI_POLL_WEIGHT) > of them before entering upper stacks seems not a good choice, since 64 > skbs > will need to be kept in the GRO system, compared to only 8 with Edward > value. Regards, ᚷ ᛖ ᚢ ᚦ ᚠ ᚱ