bnxt_en NIC driver crashes IO_PAGE_FAULT

bnxt_en NIC driver crashes IO_PAGE_FAULT

[Roman Steinhart]

Hi all,

You receive this mail because I raised a bug report against the
bnxt_en driver in the Linux kernel on launchpad.net:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1931106
I was advised there to get in touch with you here.

We received a bunch of new servers with a Supermicro H12SSL-NT
mainboard that has an embedded Broadcom BCM57416 NIC.

On all those servers we observe crashes of the NIC driver (bnxt_en)
from time to time. We're not able to manually reproduce this issue, it
just occurs at some point. Also our monitoring does not show any
irregularities(high traffic flow or sth. like this).

All servers are running with up-to-date packages:
$ lsb_release -rd
Description: Ubuntu 20.04.2 LTS
Release: 20.04

We tested the kernel versions 5.4.0-73 back to -66, the current HWE
kernel 5.8.0-55 as well as the latest mainline kernel
5.13.0-051300rc5.
On those 20 servers the crash occurs like ~1-2 times a week.
Just with the 5.13.0 kernel the driver crashed on all 5 servers
running that version within 1-2 hours after installing that kernel
version.

Syslog 5.4.0-73 kernel: https://pastebin.com/yDAyjHvF
Syslog 5.13-rc5 kernel: https://pastebin.com/GWqtVaA3
Apport file: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1931106/+attachment/5502930/+files/apport.linux-image-5.8.0-55-generic.cime34c6.apport

related Launchpad.net Bug report:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1931106


Thanks in advance.
~ Roman

Re: bnxt_en NIC driver crashes IO_PAGE_FAULT

[Michael Chan]

[-- Attachment #1: Type: text/plain, Size: 713 bytes --]

On Tue, Jun 8, 2021 at 10:53 AM Roman Steinhart <roman@aternos.org> wrote:
> We received a bunch of new servers with a Supermicro H12SSL-NT
> mainboard that has an embedded Broadcom BCM57416 NIC.
>
> On all those servers we observe crashes of the NIC driver (bnxt_en) from
> time to time. We're not able to manually reproduce this issue, it just occurs at
> some point. Also our monitoring does not show any irregularities(high traffic
> flow or sth. like this).
>

These IOMMU faults are seen on AMD systems, right?  We have also seen
similar issues on some AMD systems and have worked with AMD to debug
the issues.  I'll likely have someone who's more familiar with these
AMD IOMMU issues contact you.  Thanks.

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4209 bytes --]