From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8DA11C433EF for ; Fri, 6 May 2022 09:44:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1390603AbiEFJsI convert rfc822-to-8bit (ORCPT ); Fri, 6 May 2022 05:48:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58716 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235926AbiEFJsH (ORCPT ); Fri, 6 May 2022 05:48:07 -0400 Received: from mout.kundenserver.de (mout.kundenserver.de [212.227.126.131]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E310E309 for ; Fri, 6 May 2022 02:44:23 -0700 (PDT) Received: from mail-wm1-f42.google.com ([209.85.128.42]) by mrelayeu.kundenserver.de (mreue012 [213.165.67.97]) with ESMTPSA (Nemesis) id 1MQy0N-1nQa823vpn-00NwsF for ; Fri, 06 May 2022 11:44:21 +0200 Received: by mail-wm1-f42.google.com with SMTP id k126so4095213wme.2 for ; Fri, 06 May 2022 02:44:21 -0700 (PDT) X-Gm-Message-State: AOAM530luWF96E1qO84tTpiqEi0KOV3c/nwklGQziLDJZLaJjUg7pG/9 2bHVRbxJRA//lyP/JDhV7OwFlrFq50OF4HUN09M= X-Google-Smtp-Source: ABdhPJw674cI+pdA3oV/AEeEt4eKbh3MctCt9mOfVrjt0xhdmh9s7Xx+DtmywUr9l/up+ycGJVjrsipvLvigAJlKg0A= X-Received: by 2002:a05:600c:4f0f:b0:394:54c1:f5b3 with SMTP id l15-20020a05600c4f0f00b0039454c1f5b3mr9324422wmq.33.1651830261595; Fri, 06 May 2022 02:44:21 -0700 (PDT) MIME-Version: 1.0 References: <84f25f73-1fab-fe43-70eb-45d25b614b4c@gmail.com> <20220427125658.3127816-1-alexandr.lobakin@intel.com> <066fc320-dc04-11a4-476e-b0d11f3b17e6@gmail.com> <510bd08b-3d46-2fc8-3974-9d99fd53430e@gmail.com> <306e9713-5c37-8c6a-488b-bc07f8b8b274@gmail.com> In-Reply-To: <306e9713-5c37-8c6a-488b-bc07f8b8b274@gmail.com> From: Arnd Bergmann Date: Fri, 6 May 2022 11:44:05 +0200 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Optimizing kernel compilation / alignments for network performance To: =?UTF-8?B?UmFmYcWCIE1pxYJlY2tp?= Cc: Arnd Bergmann , Andrew Lunn , Alexander Lobakin , Network Development , linux-arm-kernel , Russell King , Felix Fietkau , "openwrt-devel@lists.openwrt.org" , Florian Fainelli Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8BIT X-Provags-ID: V03:K1:ml63v7AdlgWKa0P92ncS8hNHCP1EVq5PM1o9W9yiugBYXWdYPev 8EqipSTzTvJPVxizc6KrRV9OsqqxJ7pWypge7m8mgCLCzYcyq3XRGb2d4jk0mpdhJL58GSx mCyrwoZFX3G9dTarEWRHpvtmOBMgG+M1azAoKiHHGnuA0pKEKol4iRcTwMiurrWclfgjRYh +dVgs1IDFM4/fBF5KZEgA== X-UI-Out-Filterresults: notjunk:1;V03:K0:F7lxRvfy3uM=:RuM1sbTPhdgHi++1pNl5Yl fpGFbuGpG2fG7GkCSVKCKicwf4LTDAmyjIAlLb6wCnNULt/yfAfCRBuqXqUTeXc55xH46SL2U XqoOk801csv+1AjkxieE4oDewgRn5PQ8Ek9XZOK6lR9zpvfAh4FalPCJI0hswgRBnJ6kvyG4S Pdb7Xwz5qAKffO4wXiQDpG/teTC/JPXFwF/FFkL2CIf0BPVltzir3P7sZB5H4ZxJcPauyL18E H7XUA+2Eiitl9rPhKaxiymJ7uW1UQjq6JAGVPePRP9lXg2zzDTx0jAm5ghAZ9ExrxE8Cj9XPo 0qG3P04/4r1w45QoFezhkhFOxbYpXzZFHZL/fngyGcbiRhi28fJAfQEZJ6hZIrX2mznLK5hYV H4JSc8Q6/gaBxE0Ws1B0ZkOphZmzsTrwmZcgpIJ+smCUxn1cz7uEwsRWCoCQwnnnN8MhvDhEA fGz7XAk5uucuINAQea1riUJRRg7PW7zLTFz5tuk0cYYdlz1JXFBTvXYXC+dHhY0+2bYyZ0MNw ATKxyYbUuYmapc/O5qsIMVRwblWFO7HvVwmtBlceV8pPprY0rphEbIZFt7I9BVOeG1u+WkV2D fcqQmf/qomrpszCXcawld0NoPdZf7zg3J4fdwj4PVKeQDSWWNEuFW3TGxLMXModTaScN9Ydc7 9YU8kV00sjHd5fF6qp8ZUWoz3yBXbmHX9TdWptqPjEBEDIcaJ7+PEib8lhqxI+2ppS8TJuOsH 6KQBMq12BYUEjJP/YfQszGI6sr+G9k+3hsmExyHs5a84d7N0pvijJW1KqqfA9vHgzf8fRWv+g +qcbouO8nI35zzgmDzkuZGfoCWxEduLPkwLzKyrmnoHcWsFbwY= Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org On Fri, May 6, 2022 at 10:55 AM Rafał Miłecki wrote: > On 6.05.2022 10:45, Arnd Bergmann wrote: > > On Fri, May 6, 2022 at 9:44 AM Rafał Miłecki wrote: > >> With > >> echo 1 > /sys/class/net/eth0/queues/rx-0/rps_cpus > >> my NAT speeds were jumping between 2 speeds: > >> 284 Mbps / 408 Mbps > > > > Can you try using 'numactl -C' to pin the iperf processes to > > a particular CPU core? This may be related to the locality of > > the user process relative to where the interrupts end up. > > I run iperf on x86 machines connected to router's WAN and LAN ports. > It's meant to emulate end user just downloading from / uploading to > Internet some data. > > Router's only task is doing masquarade NAT here. Ah, makes sense. Can you observe the CPU usage to be on a particular core in the slow vs fast case then? Arnd