From: Paul Menzel <pmenzel@molgen.mpg.de> To: Bartek Kois <bartek.kois@gmail.com> Cc: intel-wired-lan@osuosl.org, regressions@lists.linux.dev Subject: Re: [Intel-wired-lan] Supermicro AOC-STGN-I1S (Intel 82599EN based 10G adapter) - poor network perfomance after moving to Debian 11.5 Date: Thu, 19 Jan 2023 11:17:42 +0100 [thread overview] Message-ID: <652bf236-d97e-832c-e0f3-24927a46d7ad@molgen.mpg.de> (raw) In-Reply-To: <d1530cba-1a72-cae8-6a04-ed8ec0f82e6e@gmail.com> #regzbot ^introduced: 4.9.88..5.10.149 Dear Bartek, Am 14.01.23 um 11:23 schrieb Bartek Kois: > After moving from Debian 9.7 to 11.5 as soon as I perform "ip link set > enp1s0 up" for my 10G adapter (AOC-STGN-I1S - Intel 82599EN based 10G > adapter) I am experiencing high cpu load (even if no traffic is passing > through the adapter) and network performance is low (when network is > connected). How do you test the network performance? Please give exact numbers for comparison. > The cpu load is oscillating between 0.1 and 0.3 on vanilla system > with no network attached. The problem can be observed on the > following platforms: Supermicro X9SCL (Intel C202 PCH) and > Supermicro X10SLL+-F (Intel C222 Express PCH), but for the Supermicro > X11SSL-F (Intel® C232 chipset) everything is working well. > > Tested environments: > Debian 9.7 - Linux 4.9.0-6-amd64 #1 SMP Debian 4.9.88-1+deb9u1 > (2018-05-07) x86_64 GNU/Linux [all platforms working well with no > problems: Supermicro X9SCL (Intel C202 PCH), Supermicro X10SLL+-F (Intel > C222 Express PCH), Supermicro X11SSL-F (Intel® C232 chipset)] > Debian 11.5 - Linux 5.10.0-19-amd64 #1 SMP Debian 5.10.149-2 > (2022-10-21) x86_64 GNU/Linux [older platforms: Supermicro X9SCL (Intel > C202 PCH), Supermicro X10SLL+-F (Intel C222 Express PCH) behave > problematic as described above | newer platform: Supermicro X11SSL-F > (Intel® C232 chipset) working well with no problems] Maybe create a bug at the Linux kernel bug tracker [1], where you can attach all the logs (`dmesg`, `lspci -nnk -s …`, …). > So far to solve the problem I was trying to upgrade system to the newest > stable version, upgrade kernel to version 6.x, upgrade ixgbe driver to > the newest version but with no luck. Thank you for checking that. Too bad it’s still present. To rule out some user space problem, could you test Debian 9.7 with a stable Linux release, currently 6.1.7? What does `sudo perf top --sort comm,dso` show, where the time is spent? > Supermicro support suggested as follows: > it might be kernel related debian 11.5 has kernel 5.10 which is a > recent kernel it might not properly support the chipsets for X9 > therefore i suggest to use RHEL or CentOS as they use much older kernel > versions. I expect that with ubuntu 20.04 you see the same problem it > uses kernel 5.4 Testing another GNU/Linux distribution for another data point, might be a good idea. As nobody has responded yet, bisecting the issue is probably the fastest way to get to the bottom of this. Luckily the problem seems reproducible and you seem to be able to build a Linux kernel yourself, so that should work. (For testing purposes you could also test with Ubuntu, as they provide Linux kernel builds for (almost) all releases in their Linux kernel mainline PPA [2].) Kind regards, Paul [1]: https://bugzilla.kernel.org/ [2]: https://kernel.ubuntu.com/~kernel-ppa/mainline/
WARNING: multiple messages have this Message-ID (diff)
From: Paul Menzel <pmenzel@molgen.mpg.de> To: Bartek Kois <bartek.kois@gmail.com> Cc: intel-wired-lan@osuosl.org, regressions@lists.linux.dev Subject: Re: [Intel-wired-lan] Supermicro AOC-STGN-I1S (Intel 82599EN based 10G adapter) - poor network perfomance after moving to Debian 11.5 Date: Thu, 19 Jan 2023 11:17:42 +0100 [thread overview] Message-ID: <652bf236-d97e-832c-e0f3-24927a46d7ad@molgen.mpg.de> (raw) In-Reply-To: <d1530cba-1a72-cae8-6a04-ed8ec0f82e6e@gmail.com> #regzbot ^introduced: 4.9.88..5.10.149 Dear Bartek, Am 14.01.23 um 11:23 schrieb Bartek Kois: > After moving from Debian 9.7 to 11.5 as soon as I perform "ip link set > enp1s0 up" for my 10G adapter (AOC-STGN-I1S - Intel 82599EN based 10G > adapter) I am experiencing high cpu load (even if no traffic is passing > through the adapter) and network performance is low (when network is > connected). How do you test the network performance? Please give exact numbers for comparison. > The cpu load is oscillating between 0.1 and 0.3 on vanilla system > with no network attached. The problem can be observed on the > following platforms: Supermicro X9SCL (Intel C202 PCH) and > Supermicro X10SLL+-F (Intel C222 Express PCH), but for the Supermicro > X11SSL-F (Intel® C232 chipset) everything is working well. > > Tested environments: > Debian 9.7 - Linux 4.9.0-6-amd64 #1 SMP Debian 4.9.88-1+deb9u1 > (2018-05-07) x86_64 GNU/Linux [all platforms working well with no > problems: Supermicro X9SCL (Intel C202 PCH), Supermicro X10SLL+-F (Intel > C222 Express PCH), Supermicro X11SSL-F (Intel® C232 chipset)] > Debian 11.5 - Linux 5.10.0-19-amd64 #1 SMP Debian 5.10.149-2 > (2022-10-21) x86_64 GNU/Linux [older platforms: Supermicro X9SCL (Intel > C202 PCH), Supermicro X10SLL+-F (Intel C222 Express PCH) behave > problematic as described above | newer platform: Supermicro X11SSL-F > (Intel® C232 chipset) working well with no problems] Maybe create a bug at the Linux kernel bug tracker [1], where you can attach all the logs (`dmesg`, `lspci -nnk -s …`, …). > So far to solve the problem I was trying to upgrade system to the newest > stable version, upgrade kernel to version 6.x, upgrade ixgbe driver to > the newest version but with no luck. Thank you for checking that. Too bad it’s still present. To rule out some user space problem, could you test Debian 9.7 with a stable Linux release, currently 6.1.7? What does `sudo perf top --sort comm,dso` show, where the time is spent? > Supermicro support suggested as follows: > it might be kernel related debian 11.5 has kernel 5.10 which is a > recent kernel it might not properly support the chipsets for X9 > therefore i suggest to use RHEL or CentOS as they use much older kernel > versions. I expect that with ubuntu 20.04 you see the same problem it > uses kernel 5.4 Testing another GNU/Linux distribution for another data point, might be a good idea. As nobody has responded yet, bisecting the issue is probably the fastest way to get to the bottom of this. Luckily the problem seems reproducible and you seem to be able to build a Linux kernel yourself, so that should work. (For testing purposes you could also test with Ubuntu, as they provide Linux kernel builds for (almost) all releases in their Linux kernel mainline PPA [2].) Kind regards, Paul [1]: https://bugzilla.kernel.org/ [2]: https://kernel.ubuntu.com/~kernel-ppa/mainline/ _______________________________________________ Intel-wired-lan mailing list Intel-wired-lan@osuosl.org https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
next prev parent reply other threads:[~2023-01-19 10:17 UTC|newest] Thread overview: 31+ messages / expand[flat|nested] mbox.gz Atom feed top 2023-01-14 10:23 [Intel-wired-lan] Supermicro AOC-STGN-I1S (Intel 82599EN based 10G adapter) - poor network perfomance after moving to Debian 11.5 Bartek Kois 2023-01-19 9:59 ` Bartek Kois 2023-01-19 10:17 ` Paul Menzel [this message] 2023-01-19 10:17 ` Paul Menzel 2023-01-19 10:22 ` [Intel-wired-lan] Supermicro AOC-STGN-I1S (Intel 82599EN based 10G adapter) - poor network performance " Paul Menzel 2023-01-19 10:22 ` Paul Menzel 2023-01-19 12:24 ` [Intel-wired-lan] Supermicro AOC-STGN-I1S (Intel 82599EN based 10G adapter) - poor network perfomance " Bartek Kois 2023-01-19 12:24 ` Bartek Kois 2023-01-19 16:58 ` Bartek Kois 2023-01-19 16:58 ` Bartek Kois 2023-01-19 17:09 ` Paul Menzel 2023-01-19 17:09 ` Paul Menzel 2023-01-19 17:17 ` Bartek Kois 2023-01-19 17:17 ` Bartek Kois 2023-01-22 20:28 ` Paul Menzel 2023-01-22 20:28 ` Paul Menzel 2023-01-23 18:38 ` Bartek Kois 2023-01-23 18:38 ` Bartek Kois 2023-01-23 18:53 ` Paul Menzel 2023-01-23 18:53 ` Paul Menzel 2023-01-23 18:58 ` Bartek Kois 2023-01-23 18:58 ` Bartek Kois 2023-01-23 19:03 ` Paul Menzel 2023-01-23 19:03 ` Paul Menzel 2023-01-24 9:33 ` Linux kernel regression tracking (Thorsten Leemhuis) 2023-01-24 9:33 ` Linux kernel regression tracking (Thorsten Leemhuis) 2023-01-24 9:40 ` Bartek Kois 2023-01-24 9:40 ` Bartek Kois 2023-03-23 13:46 ` Linux regression tracking (Thorsten Leemhuis) 2023-03-23 13:46 ` Linux regression tracking (Thorsten Leemhuis) -- strict thread matches above, loose matches on Subject: below -- 2023-01-04 8:39 Bartek Kois
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=652bf236-d97e-832c-e0f3-24927a46d7ad@molgen.mpg.de \ --to=pmenzel@molgen.mpg.de \ --cc=bartek.kois@gmail.com \ --cc=intel-wired-lan@osuosl.org \ --cc=regressions@lists.linux.dev \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.