From mboxrd@z Thu Jan 1 00:00:00 1970 From: Eric Dumazet Subject: Re: [PATCH net-next 2/2] udp: implement and use per cpu rx skbs cache Date: Thu, 19 Apr 2018 06:47:10 -0700 Message-ID: <0e3abeb5-8081-f9ea-4de6-cc1a7edfc5a5@gmail.com> References: <890db004-4dfe-7f77-61ee-1ac0d7d2a24c@gmail.com> <1524071712.2599.60.camel@redhat.com> <3270c995-4eea-b3e1-128c-82921d89eb79@gmail.com> <1524123637.3160.16.camel@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Cc: "David S. Miller" To: Paolo Abeni , Eric Dumazet , netdev@vger.kernel.org Return-path: Received: from mail-pl0-f43.google.com ([209.85.160.43]:37520 "EHLO mail-pl0-f43.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752455AbeDSNrM (ORCPT ); Thu, 19 Apr 2018 09:47:12 -0400 Received: by mail-pl0-f43.google.com with SMTP id f7-v6so3287573plr.4 for ; Thu, 19 Apr 2018 06:47:12 -0700 (PDT) In-Reply-To: <1524123637.3160.16.camel@redhat.com> Content-Language: en-US Sender: netdev-owner@vger.kernel.org List-ID: On 04/19/2018 12:40 AM, Paolo Abeni wrote: > Hi, > > On Wed, 2018-04-18 at 12:21 -0700, Eric Dumazet wrote: >> >> On 04/18/2018 10:15 AM, Paolo Abeni wrote: >> is not appealing to me :/ >>> >>> Thank you for the feedback. >>> Sorry for not being clear about it, but knotd is using SO_REUSEPORT and >>> the above tests are leveraging it. >>> >>> That 5% is on top of that 300%. >> >> Then there is something wrong. >> >> Adding copies should not increase performance. > > The skb and data are copied into the UDP skb cache only if the socket > is under memory pressure, and that happens if and only if the receiver > is slower than the BH/IP receive path. Which is going to happen under attack. Bimodal behavior is dangerous for system stability.. > > The copy slows down the RX path - which was dropping packets - and > makes the udp_recvmsg() considerably faster, as consuming skb becomes > almost a no-op. > > AFAICS, this is similar to the strategy you used in: > > ommit c8c8b127091b758f5768f906bcdeeb88bc9951ca > Author: Eric Dumazet > Date: Wed Dec 7 09:19:33 2016 -0800 > > udp: under rx pressure, try to condense skbs > > with the difference that with the UDP skb cache there is an hard limit > to the amount of memory the BH is allowed to copy. > Very different strategy really. We do not copy 500 bytes per skb :/ and the total amount of memory is tunable (socket rcvbuf) instead of hard coded in the kernel :/ >> If it does, there is certainly another way, reaching 10% instead of 5% > > I benchmarked vs a DNS server to test and verify that we get measurable > benefits in real life scenario. The measured performance gain for the > RX path with reasonable configurations is ~20%. Then we probably can make +40% without copies. > > Any suggestions for better results are more than welcome! Yes, remote skb freeing. I mentioned this idea to Jesper and Tariq in Seoul (netdev conference) Not tied to UDP, but a generic solution. You are adding more and more code only that only helps in some benchmarks really. UDP stack is becoming a very complex beast, while heavy duty UDP servers have alternatives, and can not cope with arbitrary flood anyway.