From: Bjorn Helgaas <helgaas@kernel.org>
To: Paul Menzel <pmenzel@molgen.mpg.de>
Cc: "Jörg Rödel" <joro@8bytes.org>,
"Suravee Suthikulpanit" <suravee.suthikulpanit@amd.com>,
"Bjorn Helgaas" <bhelgaas@google.com>,
iommu@lists.linux-foundation.org,
"Thomas Gleixner" <tglx@linutronix.de>,
"Ingo Molnar" <mingo@redhat.com>,
"Borislav Petkov" <bp@alien8.de>,
x86@kernel.org, LKML <linux-kernel@vger.kernel.org>,
linux-pci@vger.kernel.org
Subject: Re: How to reduce PCI initialization from 5 s (1.5 s adding them to IOMMU groups)
Date: Fri, 5 Nov 2021 13:53:04 -0500 [thread overview]
Message-ID: <20211105185304.GA936068@bhelgaas> (raw)
In-Reply-To: <de6706b2-4ea5-ce68-6b72-02090b98630f@molgen.mpg.de>
On Fri, Nov 05, 2021 at 12:56:09PM +0100, Paul Menzel wrote:
> Dear Linux folks,
>
>
> On a PowerEdge T440/021KCD, BIOS 2.11.2 04/22/2021, Linux 5.10.70 takes
> almost five seconds to initialize PCI. According to the timestamps, 1.5 s
> are from assigning the PCI devices to the 142 IOMMU groups.
>
> ```
> $ lspci | wc -l
> 281
> $ dmesg
> […]
> [ 2.918411] PCI: Using host bridge windows from ACPI; if necessary, use
> "pci=nocrs" and report a bug
> [ 2.933841] ACPI: Enabled 5 GPEs in block 00 to 7F
> [ 2.973739] ACPI: PCI Root Bridge [PC00] (domain 0000 [bus 00-16])
> [ 2.980398] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM
> ClockPM Segments MSI HPX-Type3]
> [ 2.989457] acpi PNP0A08:00: _OSC: platform does not support [LTR]
> [ 2.995451] acpi PNP0A08:00: _OSC: OS now controls [PME PCIeCapability]
> [ 3.001394] acpi PNP0A08:00: FADT indicates ASPM is unsupported, using
> BIOS configuration
> [ 3.010511] PCI host bridge to bus 0000:00
> […]
> [ 6.233508] system 00:05: [io 0x1000-0x10fe] has been reserved
> [ 6.239420] system 00:05: Plug and Play ACPI device, IDs PNP0c02 (active)
> [ 6.239906] pnp: PnP ACPI: found 6 devices
For ~280 PCI devices, (6.24-2.92)/280 = 0.012 s/dev. On my laptop I
have about (.66-.37)/36 = 0.008 s/dev (on v5.4), so about the same
ballpark.
Faster would always be better, of course. I assume this is not really
a regression?
> [ 6.989016] pci 0000:d7:05.0: disabled boot interrupts on device
> [8086:2034]
> [ 6.996063] PCI: CLS 0 bytes, default 64
> [ 7.000008] Trying to unpack rootfs image as initramfs...
> [ 7.065281] Freeing initrd memory: 5136K
> […]
> [ 7.079098] DMAR: dmar7: Using Queued invalidation
> [ 7.083983] pci 0000:00:00.0: Adding to iommu group 0
> […]
> [ 8.537808] pci 0000:d7:17.1: Adding to iommu group 141
I don't have this iommu stuff turned on and don't know what's
happening here.
> Is there anything that could be done to reduce the time?
>
>
> Kind regards,
>
> Paul
WARNING: multiple messages have this Message-ID (diff)
From: Bjorn Helgaas <helgaas@kernel.org>
To: Paul Menzel <pmenzel@molgen.mpg.de>
Cc: linux-pci@vger.kernel.org, x86@kernel.org,
LKML <linux-kernel@vger.kernel.org>,
iommu@lists.linux-foundation.org, Ingo Molnar <mingo@redhat.com>,
Borislav Petkov <bp@alien8.de>,
Bjorn Helgaas <bhelgaas@google.com>,
Thomas Gleixner <tglx@linutronix.de>
Subject: Re: How to reduce PCI initialization from 5 s (1.5 s adding them to IOMMU groups)
Date: Fri, 5 Nov 2021 13:53:04 -0500 [thread overview]
Message-ID: <20211105185304.GA936068@bhelgaas> (raw)
In-Reply-To: <de6706b2-4ea5-ce68-6b72-02090b98630f@molgen.mpg.de>
On Fri, Nov 05, 2021 at 12:56:09PM +0100, Paul Menzel wrote:
> Dear Linux folks,
>
>
> On a PowerEdge T440/021KCD, BIOS 2.11.2 04/22/2021, Linux 5.10.70 takes
> almost five seconds to initialize PCI. According to the timestamps, 1.5 s
> are from assigning the PCI devices to the 142 IOMMU groups.
>
> ```
> $ lspci | wc -l
> 281
> $ dmesg
> […]
> [ 2.918411] PCI: Using host bridge windows from ACPI; if necessary, use
> "pci=nocrs" and report a bug
> [ 2.933841] ACPI: Enabled 5 GPEs in block 00 to 7F
> [ 2.973739] ACPI: PCI Root Bridge [PC00] (domain 0000 [bus 00-16])
> [ 2.980398] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM
> ClockPM Segments MSI HPX-Type3]
> [ 2.989457] acpi PNP0A08:00: _OSC: platform does not support [LTR]
> [ 2.995451] acpi PNP0A08:00: _OSC: OS now controls [PME PCIeCapability]
> [ 3.001394] acpi PNP0A08:00: FADT indicates ASPM is unsupported, using
> BIOS configuration
> [ 3.010511] PCI host bridge to bus 0000:00
> […]
> [ 6.233508] system 00:05: [io 0x1000-0x10fe] has been reserved
> [ 6.239420] system 00:05: Plug and Play ACPI device, IDs PNP0c02 (active)
> [ 6.239906] pnp: PnP ACPI: found 6 devices
For ~280 PCI devices, (6.24-2.92)/280 = 0.012 s/dev. On my laptop I
have about (.66-.37)/36 = 0.008 s/dev (on v5.4), so about the same
ballpark.
Faster would always be better, of course. I assume this is not really
a regression?
> [ 6.989016] pci 0000:d7:05.0: disabled boot interrupts on device
> [8086:2034]
> [ 6.996063] PCI: CLS 0 bytes, default 64
> [ 7.000008] Trying to unpack rootfs image as initramfs...
> [ 7.065281] Freeing initrd memory: 5136K
> […]
> [ 7.079098] DMAR: dmar7: Using Queued invalidation
> [ 7.083983] pci 0000:00:00.0: Adding to iommu group 0
> […]
> [ 8.537808] pci 0000:d7:17.1: Adding to iommu group 141
I don't have this iommu stuff turned on and don't know what's
happening here.
> Is there anything that could be done to reduce the time?
>
>
> Kind regards,
>
> Paul
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
next prev parent reply other threads:[~2021-11-05 18:53 UTC|newest]
Thread overview: 20+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-11-05 11:56 How to reduce PCI initialization from 5 s (1.5 s adding them to IOMMU groups) Paul Menzel
2021-11-05 11:56 ` Paul Menzel
2021-11-05 12:04 ` How to reduce PCI initialization from 5 s (1.5 s adding them to IOMMU groups)s Paul Menzel
2021-11-05 12:04 ` Paul Menzel
2021-11-05 18:53 ` Bjorn Helgaas [this message]
2021-11-05 18:53 ` How to reduce PCI initialization from 5 s (1.5 s adding them to IOMMU groups) Bjorn Helgaas
2021-11-06 10:42 ` Paul Menzel
2021-11-06 10:42 ` Paul Menzel
2021-11-09 15:31 ` Robin Murphy
2021-11-09 15:31 ` Robin Murphy
2021-11-09 20:32 ` Paul Menzel
2021-11-09 20:32 ` Paul Menzel
2021-11-08 17:18 ` Krzysztof Wilczyński
2021-11-08 17:18 ` Krzysztof Wilczyński
2021-11-09 20:25 ` Paul Menzel
2021-11-09 20:25 ` Paul Menzel
2021-11-09 23:10 ` Krzysztof Wilczyński
2021-11-09 23:10 ` Krzysztof Wilczyński
2021-11-19 14:43 ` Paul Menzel
2021-11-19 14:43 ` Paul Menzel
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20211105185304.GA936068@bhelgaas \
--to=helgaas@kernel.org \
--cc=bhelgaas@google.com \
--cc=bp@alien8.de \
--cc=iommu@lists.linux-foundation.org \
--cc=joro@8bytes.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=pmenzel@molgen.mpg.de \
--cc=suravee.suthikulpanit@amd.com \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.