All of lore.kernel.org
 help / color / mirror / Atom feed
From: David Hildenbrand <david@redhat.com>
To: "Deucher, Alexander" <Alexander.Deucher@amd.com>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	amd-gfx list <amd-gfx@lists.freedesktop.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	"Liang, Liang (Leo)" <Liang.Liang@amd.com>
Cc: "Huang, Ray" <Ray.Huang@amd.com>,
	"Koenig, Christian" <Christian.Koenig@amd.com>,
	Mike Rapoport <rppt@linux.ibm.com>,
	"Rafael J. Wysocki" <rafael@kernel.org>,
	George Kennedy <george.kennedy@oracle.com>
Subject: Re: slow boot with 7fef431be9c9 ("mm/page_alloc: place pages to tail in __free_pages_core()")
Date: Fri, 12 Mar 2021 16:48:28 +0100	[thread overview]
Message-ID: <b5faa9b9-6f40-f8e4-cf7c-795b429e02a8@redhat.com> (raw)
In-Reply-To: <MN2PR12MB44884289499B6B16A0E9017DF76F9@MN2PR12MB4488.namprd12.prod.outlook.com>

> 8G (with some carve out for the integrated GPU).
> [    0.044181] Memory: 6858688K/7200304K available (14345K kernel code, 9659K rwdata, 4980K rodata, 2484K init, 12292K bss, 341360K reserved, 0K cma-reserved)
> 
> Nothing particularly special about these systems that I am aware of.  I'll see if we can repro this issue on any other platforms, but so far, not one has noticed any problems.
> 
>>
>> Increasing the boot time from a few seconds to 2-3 minutes does not smell
>> like some corner case cache effects we might be hitting in this particular
>> instance - there have been minor reports that it either slightly increased or
>> slightly decreases initial system performance, but that was about it.
>>
>> Either, yet another latent BUG (but why? why should memory access
>> suddenly be that slow? I could only guess that we are now making sooner
>> use of very slow memory), or there is really something else weird going on.
> 
> Looks like pretty much everything is slower based on the timestamps in the dmesg output.  There is a big jump here:

If we're really dealing with some specific slow memory regions and that 
memory gets allocated for something that gets used regularly, then we 
might get a general slowdown. Hard to identify, though :)

> 
>> [    3.758596] ACPI: Added _OSI(Linux-Lenovo-NV-HDMI-Audio)
>> [    3.759372] ACPI: Added _OSI(Linux-HPI-Hybrid-Graphics)
>> [   16.177983] ACPI: 13 ACPI AML tables successfully acquired and loaded
>> [   17.099316] ACPI: [Firmware Bug]: BIOS _OSI(Linux) query ignored
>> [   18.969959] ACPI: EC: EC started
> 
> And here:
> 
>> [   36.566608] PCI: CLS 64 bytes, default 64
>> [   36.575383] Trying to unpack rootfs image as initramfs...
>> [   44.594348] Initramfs unpacking failed: Decoding failed
>> [   44.765141] Freeing initrd memory: 46348K
> 
> Also seeing soft lockups:
>> [  124.588634] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [swapper/1:0]

Yes, I noticed that -- there is a heavy slowdown somewhere.

As that patch is v5.10 already (and we're close to v5.12) I assume 
something is particularly weird about the platform you are running on - 
because this is the first time I see a report like that.

-- 
Thanks,

David / dhildenb


WARNING: multiple messages have this Message-ID (diff)
From: David Hildenbrand <david@redhat.com>
To: "Deucher, Alexander" <Alexander.Deucher@amd.com>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	amd-gfx list <amd-gfx@lists.freedesktop.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	"Liang, Liang (Leo)" <Liang.Liang@amd.com>
Cc: "Rafael J. Wysocki" <rafael@kernel.org>,
	George Kennedy <george.kennedy@oracle.com>,
	"Huang, Ray" <Ray.Huang@amd.com>,
	"Koenig, Christian" <Christian.Koenig@amd.com>,
	Mike Rapoport <rppt@linux.ibm.com>
Subject: Re: slow boot with 7fef431be9c9 ("mm/page_alloc: place pages to tail in __free_pages_core()")
Date: Fri, 12 Mar 2021 16:48:28 +0100	[thread overview]
Message-ID: <b5faa9b9-6f40-f8e4-cf7c-795b429e02a8@redhat.com> (raw)
In-Reply-To: <MN2PR12MB44884289499B6B16A0E9017DF76F9@MN2PR12MB4488.namprd12.prod.outlook.com>

> 8G (with some carve out for the integrated GPU).
> [    0.044181] Memory: 6858688K/7200304K available (14345K kernel code, 9659K rwdata, 4980K rodata, 2484K init, 12292K bss, 341360K reserved, 0K cma-reserved)
> 
> Nothing particularly special about these systems that I am aware of.  I'll see if we can repro this issue on any other platforms, but so far, not one has noticed any problems.
> 
>>
>> Increasing the boot time from a few seconds to 2-3 minutes does not smell
>> like some corner case cache effects we might be hitting in this particular
>> instance - there have been minor reports that it either slightly increased or
>> slightly decreases initial system performance, but that was about it.
>>
>> Either, yet another latent BUG (but why? why should memory access
>> suddenly be that slow? I could only guess that we are now making sooner
>> use of very slow memory), or there is really something else weird going on.
> 
> Looks like pretty much everything is slower based on the timestamps in the dmesg output.  There is a big jump here:

If we're really dealing with some specific slow memory regions and that 
memory gets allocated for something that gets used regularly, then we 
might get a general slowdown. Hard to identify, though :)

> 
>> [    3.758596] ACPI: Added _OSI(Linux-Lenovo-NV-HDMI-Audio)
>> [    3.759372] ACPI: Added _OSI(Linux-HPI-Hybrid-Graphics)
>> [   16.177983] ACPI: 13 ACPI AML tables successfully acquired and loaded
>> [   17.099316] ACPI: [Firmware Bug]: BIOS _OSI(Linux) query ignored
>> [   18.969959] ACPI: EC: EC started
> 
> And here:
> 
>> [   36.566608] PCI: CLS 64 bytes, default 64
>> [   36.575383] Trying to unpack rootfs image as initramfs...
>> [   44.594348] Initramfs unpacking failed: Decoding failed
>> [   44.765141] Freeing initrd memory: 46348K
> 
> Also seeing soft lockups:
>> [  124.588634] watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [swapper/1:0]

Yes, I noticed that -- there is a heavy slowdown somewhere.

As that patch is v5.10 already (and we're close to v5.12) I assume 
something is particularly weird about the platform you are running on - 
because this is the first time I see a report like that.

-- 
Thanks,

David / dhildenb

_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

  reply	other threads:[~2021-03-12 15:49 UTC|newest]

Thread overview: 48+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-03-11 14:41 slow boot with 7fef431be9c9 ("mm/page_alloc: place pages to tail in __free_pages_core()") Deucher, Alexander
2021-03-11 14:41 ` Deucher, Alexander
2021-03-11 15:02 ` David Hildenbrand
2021-03-11 15:02   ` David Hildenbrand
2021-03-12 14:06   ` Deucher, Alexander
2021-03-12 14:06     ` Deucher, Alexander
2021-03-12 14:12     ` David Hildenbrand
2021-03-12 14:12       ` David Hildenbrand
2021-03-12 15:39       ` Deucher, Alexander
2021-03-12 15:39         ` Deucher, Alexander
2021-03-12 15:48         ` David Hildenbrand [this message]
2021-03-12 15:48           ` David Hildenbrand
2021-03-12 16:15           ` Deucher, Alexander
2021-03-12 16:15             ` Deucher, Alexander
2021-03-12 16:19         ` Liang, Liang (Leo)
2021-03-12 16:19           ` Liang, Liang (Leo)
2021-03-12 16:46           ` David Hildenbrand
2021-03-12 16:46             ` David Hildenbrand
2021-03-13  4:04             ` Liang, Liang (Leo)
2021-03-13  4:04               ` Liang, Liang (Leo)
2021-03-13  9:05               ` David Hildenbrand
2021-03-13  9:05                 ` David Hildenbrand
2021-03-13 13:48                 ` Mike Rapoport
2021-03-13 13:48                   ` Mike Rapoport
2021-03-15 13:04                   ` David Hildenbrand
2021-03-15 13:04                     ` David Hildenbrand
2021-03-16  0:36                     ` Liang, Liang (Leo)
2021-03-16  0:36                       ` Liang, Liang (Leo)
2021-03-16  6:50                       ` Mike Rapoport
2021-03-16  6:50                         ` Mike Rapoport
2021-03-16  8:00                         ` Liang, Liang (Leo)
2021-03-16  8:00                           ` Liang, Liang (Leo)
2021-03-16  8:37                           ` David Hildenbrand
2021-03-16  8:37                             ` David Hildenbrand
2021-03-16  8:43                             ` Liang, Liang (Leo)
2021-03-16  8:43                               ` Liang, Liang (Leo)
2021-03-16  8:54                               ` David Hildenbrand
2021-03-16  8:54                                 ` David Hildenbrand
2021-03-16  8:58                                 ` Liang, Liang (Leo)
2021-03-16  8:58                                   ` Liang, Liang (Leo)
2021-03-16  9:08                                   ` David Hildenbrand
2021-03-16  9:08                                     ` David Hildenbrand
2021-03-16 10:30                                     ` Mike Rapoport
2021-03-16 10:30                                       ` Mike Rapoport
2021-03-16 11:02                                       ` Liang, Liang (Leo)
2021-03-16 11:02                                         ` Liang, Liang (Leo)
2021-03-16 11:10                                         ` David Hildenbrand
2021-03-16 11:10                                           ` David Hildenbrand

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=b5faa9b9-6f40-f8e4-cf7c-795b429e02a8@redhat.com \
    --to=david@redhat.com \
    --cc=Alexander.Deucher@amd.com \
    --cc=Christian.Koenig@amd.com \
    --cc=Liang.Liang@amd.com \
    --cc=Ray.Huang@amd.com \
    --cc=akpm@linux-foundation.org \
    --cc=amd-gfx@lists.freedesktop.org \
    --cc=george.kennedy@oracle.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=rafael@kernel.org \
    --cc=rppt@linux.ibm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.