* [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Hi!
This is a complete rework of the parallel bringup patch series (V17)
https://lore.kernel.org/lkml/20230328195758.1049469-1-usama.arif@bytedance.com
to address the issues which were discovered in review:
1) The X86 microcode loader serialization requirement
https://lore.kernel.org/lkml/87v8iirxun.ffs@tglx
Microcode loading on HT enabled X86 CPUs requires that the microcode is
loaded on the primary thread. The sibling thread(s) must be in
quiescent state; either looping in a place which is aware of potential
changes by the microcode update (see late loading) or in fully quiescent
state, i.e. waiting for INIT/SIPI.
This is required by hardware/firmware on Intel. Aside of that it's a
vendor independent software correctness issue. Assume the following
sequence:
CPU1.0 CPU1.1
CPUID($A)
Load microcode.
Changes CPUID($A, $B)
CPUID($B)
CPU1.1 makes a decision on $A and $B which might be inconsistent due
to the microcode update.
The solution for this is to bringup the primary threads first and after
that the siblings. Loading microcode on the siblings is a NOOP on Intel
and on AMD it is guaranteed to only modify thread local state.
This ensures that the APs can load microcode before reaching the alive
synchronization point w/o doing any further x86 specific
synchronization between the core siblings.
2) The general design issues discussed in V16
https://lore.kernel.org/lkml/87pm8y6yme.ffs@tglx
The previous parallel bringup patches just glued this mechanism into
the existing code without a deeper analysis of the synchronization
mechanisms and without generalizing it so that the control logic is
mostly in the core code and not made an architecture specific tinker
space.
Much of that had been pointed out 2 years ago in the discussions about
the early versions of parallel bringup already.
The series is based on:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip x86/apic
and also available from git:
git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git hotplug
Background
----------
The reason why people are interested in parallel bringup is to shorten
the (kexec) reboot time of cloud servers to reduce the downtime of the
VM tenants. There are obviously other interesting use cases for this
like VM startup time, embedded devices...
The current fully serialized bringup does the following per AP:
1) Prepare callbacks (allocate, intialize, create threads)
2) Kick the AP alive (e.g. INIT/SIPI on x86)
3) Wait for the AP to report alive state
4) Let the AP continue through the atomic bringup
5) Let the AP run the threaded bringup to full online state
There are two significant delays:
#3 The time for an AP to report alive state in start_secondary() on x86
has been measured in the range between 350us and 3.5ms depending on
vendor and CPU type, BIOS microcode size etc.
#4 The atomic bringup does the microcode update. This has been measured
to take up to ~8ms on the primary threads depending on the microcode
patch size to apply.
On a two socket SKL server with 56 cores (112 threads) the boot CPU spends
on current mainline about 800ms busy waiting for the APs to come up and
apply microcode. That's more than 80% of the actual onlining procedure.
By splitting the actual bringup mechanism into two parts this can be
reduced to waiting for the first AP to report alive or if the system is
large enough the first AP is already waiting when the boot CPU finished the
wake-up of the last AP.
The actual solution comes in several parts
------------------------------------------
1) [P 1-2] General cleanups (init annotations, kernel doc...)
2) [P 3] The obvious
Avoid pointless delay calibration when TSC is synchronized across
sockets. That removes a whopping 100ms delay for the first CPU of a
socket. This is an improvement independent of parallel bringup and had
been discussed two years ago already.
2) [P 3-6] Removal of the CPU0 hotplug hack.
This was added 11 years ago with the promise to make this a real
hardware mechanism, but that never materialized. As physical CPU
hotplug is not really supported and the physical unplugging of CPU0
never materialized there is no reason to keep this cruft around. It's
just maintenance ballast for no value and the removal makes
implementing the parallel bringup feature way simpler.
3) [P 7-16] Cleanup of the existing bringup mechanism:
a) Code reorganisation so that the general hotplug specific code is
in smpboot.c and not sprinkled all over the place
b) Decouple MTRR/PAT initialization from smp_callout_mask to prepare
for replacing that mask with a hotplug core code synchronization
mechanism.
c) Make TSC synchronization function call based so that the control CPU
does not have to busy wait for nothing if synchronization is not
required.
d) Remove the smp_callin_mask synchronization point as its not longer
required due to #3c.
e) Rework the sparse_irq_lock held region in the core code so that the
next polling synchronization point in the x86 code can be removed to.
f) Due to #3e it's not longer required to spin wait for the AP to set
it's online bit. Remove wait_cpu_online() and the XENPV
counterpart. So the control CPU can directly wait for the online
idle completion by the AP and free the control CPU up for other
work.
This reduces the synchronization points in the x86 code to one, which
is the AP alive one. This synchronization will be moved to core
infrastructure in the next section.
4) [P 17-27] Replace the disconnected CPU state tracking
The extra CPU state tracking which is used by a few architectures is
completely separate from the CPU hotplug core code.
Replacing it by a variant integrated in the core hotplug machinery
allows to reduce architecture specific code and provides a generic
synchronization mechanism for (parallel) CPU bringup/teardown.
- Convert x86 over and replace the AP alive synchronization on x86 with
the core variant which removes the remaining x86 hotplug
synchronization masks.
- Convert the other architectures usage and remove the old interface
and code.
5) [P 28-30] Split the bringup into two steps
First step invokes the wakeup function on the BP, e.g. SIPI/STARTUP on
x86. The second one waits on the BP for the AP to report alive and
releases it for the complete onlining.
As the hotplug state machine allows partial bringup this allows later
to kick all APs alive in a first iteration and then bring them up
completely one by one afterwards.
6) [P 31] Switch the primary thread detection to a cpumask
This makes the parallel bringup a simple cpumask based mechanism
without tons of conditionals and checks for primary threads.
7) [P 32] Implement the parallel bringup core code
The parallel bringup looks like this:
1) Bring up the primary SMT threads to the CPUHP_KICK_AP_ALIVE step
one by one
2) Bring up the primary SMT threads to the CPUHP_ONLINE step one by
one
3) Bring up the secondary SMT threads to the CPUHP_KICK_AP_ALIVE
step one by one
4) Bring up the secondary SMT threads to the CPUHP_ONLINE
step one by one
In case that SMT is not supported this is obviously reduced to step #1
and #2.
8) [P 33-37] Prepare X86 for parallel bringup and enable it
Caveats
-------
The non X86 changes have been all compile tested. Boot and runtime
testing has only be done on a few real hardware platforms and qemu as
available. That definitely needs some help from the people who have
these systems at their fingertips.
Results and analysis
--------------------
Here are numbers for a dual socket SKL 56 cores/ 112 threads machine. All
numbers in milliseconds. The time measured is the time which the cpu_up()
call takes for each CPU and phase. It's not exact as the system is already
scheduling, handling interrupts and soft interrupts, which is obviously
skewing the picture slightly.
Baseline tip tree x86/apic branch.
total avg/CPU min max
total : 912.081 8.217 3.720 113.271
The max of 100ms is due to the silly delay calibration for the second
socket which takes 100ms and was eliminated first. Also the other initial
cleanups and improvements take some time away.
So the real baseline becomes:
total avg/CPU min max
total : 785.960 7.081 3.752 36.098
The max here is on the first CPU of the second socket. 20ms of that is due
to TSC synchronization and an extra 2ms to react on the SIPI.
With parallel bootup enabled this becomes:
total avg/CPU min max
prepare: 39.108 0.352 0.238 0.883
online : 45.166 0.407 0.170 20.357
total : 84.274 0.759 0.408 21.240
That's a factor ~9.3 reduction on average.
Looking at the 27 primary threads of socket 0 then this becomes even more
interesting:
total avg/CPU min max
total : 325.764 12.065 11.981 14.125
versus:
total avg/CPU min max
prepare: 8.945 0.331 0.238 0.834
online : 4.830 0.179 0.170 0.212
total : 13.775 0.510 0.408 1.046
So the reduction factor is ~23.5 here. That's mostly because the 20ms TSC
sync is not skewing the picture.
For all 55 primaries, i.e with the 20ms TSC sync extra for socket 1 this
becomes:
total avg/CPU min max
total : 685.489 12.463 11.975 36.098
versus:
total avg/CPU min max
prepare: 19.080 0.353 0.238 0.883
online : 30.283 0.561 0.170 20.357
total : 49.363 0.914 0.408 21.240
The TSC sync reduces the win to a factor of ~13.8
With 'tsc=reliable' on the command line the socket sync is disabled which
brings it back to the socket 0 numbers:
total avg/CPU min max
prepare: 18.970 0.351 0.231 0.874
online : 10.328 0.191 0.169 0.358
total : 29.298 0.543 0.400 1.232
Now looking at the secondary threads only:
total avg/CPU min max
total : 100.471 1.794 0.375 4.745
versus:
total avg/CPU min max
prepare: 19.753 0.353 0.257 0.512
online : 14.671 0.262 0.179 3.461
total : 34.424 0.615 0.436 3.973
Still a factor of ~3.
The average on the secondaries for the serialized bringup is significantly
lower than for the primaries because the SIPI response time is shorter and
the microcode update takes no time.
This varies wildly with the system, whether microcode in BIOS is already up
to date, how big the microcode patch is and how long the INIT/SIPI response
time is. On an AMD Zen3 machine INIT/SIPI response time is amazingly fast
(350us), but then it lacks TSC_ADJUST and does a two millisecond TSC sync
test for _every_ AP. All of this sucks...
Possible further enhancements
-----------------------------
It's definitely worthwhile to look into reducing the cross socket TSC sync
test time. It's probably safe enough to use 5ms or even 2ms instead of 20ms
on systems with TSC_ADJUST and a few other 'TSC is sane' indicators. Moving
it out of the hotplug path is eventually possible, but that needs some deep
thoughts.
Let's take the TSC sync out of the picture by adding 'tsc=reliable" to the
kernel command line. So the bringup of 111 APs takes:
total avg/CPU min max
prepare: 38.936 0.351 0.231 0.874
online : 25.231 0.227 0.169 3.465
total : 64.167 0.578 0.400 4.339
Some of the outliers are not necessarily in the state callbacks as the
system is already scheduling and handles interrupts and soft
interrupts. Haven't analyzed that yet in detail.
In the prepare stage which runs on the control CPU the larger steps are:
smpcfd:prepare 16us avg/CPU
threads:prepare 98us avg/CPU
workqueue:prepare 43us avg/CPU
trace/RB:prepare 135us avg/CPU
The trace ringbuffer initialization allocates 354 pages and 354 control
structures one by one. That probably should allocate a large page and an
array of control structures and work from there. I'm sure that would reduce
this significantly. Steven?
smpcfd does just a percpu allocation. No idea why that takes that long.
Vs. threads and workqueues. David thought about spreading out the
preparation work and do it really in parallel. That's a nice idea, but the
threads and workqueue prepare steps are self serializing. The workqueue one
has a global mutex and aside of that both steps create kernel threads which
implicitely serialize on kthreadd. alloc_percpu(), which is used by
smpcfd:prepare is also globally serialized.
The rest of the prepare steps is pretty much in the single digit
microseconds range.
On the AP side it should be possible to move some of the initialization
steps before the alive synchronization point, but that really needs a lot
of analysis whether the functions are safe to invoke that early and outside
of the cpu_hotplug_lock held region for the case of two stage parallel
bringup; see below.
The largest part is:
identify_secondary_cpu() 99us avg/CPU
Inside of identify_secondary_cpu() the largest offender:
mcheck_init() 73us avg/CPU
This part is definitly worth to be looked at whether it can be at least
partially moved to the early startup code before the alive
synchronization point. There's a lot of deep analysis required and
ideally we just rewrite the whole CPUID evaluation trainwreck
completely.
The rest of the AP side is low single digit microseconds except of:
perf/x86:starting 14us avg/CPU
smpboot/threads:online 13us avg/CPU
workqueue:online 17us avg/CPU
mm/vmstat:online 17us avg/CPU
sched:active 30us avg/CPU
sched:active is special. Onlining the first secondary HT thread on the
second socket creates a 3.2ms outlier which skews the whole picture. That's
caused by enabling the static key sched_smt_present which patches the world
and some more. For all other APs this is really in the 1us range. This
definitely could be postponed during bootup like the scheduler domain
rebuild is done after the bringup. But that's still fully serialized and
single threaded and obviously could be done later in the context of async
parallel init. It's unclear why this is different with the fully serialized
bringup where it takes significantly less time, but that's something which
needs to be investigated.
Is truly parallel bringup feasible?
-----------------------------------
In theory yes, realistically no. Why?
1) The preparation phase
Allocating memory, creating threads for the to be brought up CPU must
obviously happen on an already online CPU.
While it would be possible to bring up a subset of CPUs first and let
them do the preparation steps for groups of still offline CPUs
concurrently, the actual benefit of doing so is dubious.
The prime example is kernel thread creation, which is implicitely
serialized on kthreadd.
A simple experiment shows that 4 concurrent workers on 4 different
CPUs where each is creating 14 * 5 = 70 kernel threads are 5% slower
than a single worker creating 4 * 14 * 5 = 280 threads.
So we'd need to have multiple kthreadd instances to handle that,
which would then serialize on tasklist lock and other things.
That aside the preparation phase is also affected by the problem
below.
2) Assumptions about hotplug serialization
a) There are quite some assumptions about CPU bringup being fully
serialized across state transitions. A lot of state callbacks rely
on that and would require local locking.
Adding that local locking is surely possible, but that has several
downsides:
- It adds complexity and makes it harder for developers to get
this correct. The subtle bugs resulting out of that are going
to be interesting
- Fine grained locking has a charm, but only if the time spent
for the actual work is larger than the time required for
serialization and synchronization.
Serializing a callback which takes less than a microsecond and
then having a large number of CPUs contending on the lock will
not make it any faster at all. That's a well known issue of
parallelizing and neither made up nor kernel specific.
b) Some operations definitely require to be protected by the
cpu_hotplug_lock, especially those which affect cpumasks as the
masks are guaranteed to be stable in a cpus_read_lock()'ed region.
As this lock cannot be taken in atomic contexts, it's required
that the control CPU holds the lock write locked across these
state transitions. And no, we are not making this a spinlock just
for that and we even can't.
Just slapping a lock into the x86 specific part of the cpumask
update function does not solve anything. The relevant patch in V17
is completely useless as it only serializes the actual cpumask/map
modifications, but all read side users are hosed if the update
would be moved before the alive synchronization point, i.e. into a
non hotplug lock protected region.
Even if the hotplug lock would be held accross the whole parallel
bringup operation then this would still expose all usage of these
masks and maps in the actual hotplug state callbacks to concurrent
modifications.
And no, we are not going to expose an architecture specific raw
spinlock to the hotplug state callbacks, especially not to those
in generic code.
c) Some cpu_read_lock()'ed regions also expect that there is no CPU
state transition happening which would modify their local
state. This would again require local serialization.
3) The amount of work and churn:
- Analyze the per architecture low level startup functions plus their
descendant functions and make them ready for concurrency if
necessary.
- Analyze ~300 hotplug state callbacks and their descendant functions
and make them ready for concurrency if necessary.
- Analyze all cpus_read_lock()'ed regions and address their
requirements.
- Rewrite the core code to handle the cpu_hotplug_lock requirements
only in distinct phases of the state machine.
- Rewrite the core code to handle state callback failure and the
related rollback in the context of the new rules.
- ...
Even if some people are dedicated enough to do that, it's very
questionable whether the resulting complexity is justified.
We've spent a serious amount of time to sanitize hotplug and bring it
into a state where it is correct. This also made it reasonably simple
for developers to implement hotplug state callbacks without having to
become hotplug experts.
Breaking this completely up will result in a flood of hard to diagnose
subtle issues for sure. Who is going to deal with them?
The experience with this series so far does not make me comfortable
about that thought in any way.
Summary
-------
The obvious and low hanging fruits have to be solved first:
- The CPUID evaluation and related setup mechanisms
- The trace/ringbuffer oddity
- The sched:active oddity for the first sibling on the second socket
- Some other expensive things which I'm not seeing in my test setup due
to lack of hardware or configuration.
Anything else is pretty much wishful thinking in my opinion.
To be clear. I'm not standing in the way if there is a proper solution,
but that requires to respect the basic engineering rules:
1) Correctness first
2) Keep it maintainable
3) Keep it simple
So far this stuff failed already at #1.
I completely understand why this is important for cloud people, but
the real question to ask here is what are the actual requirements.
As far as I understand the main goal is to make a (kexec) reboot
almost invisible to VM tenants.
Now lets look at how this works:
A) Freeze VMs and persist state
B) kexec into the new kernel
C) Restore VMs from persistant memory
D) Thaw VMs
So the key problem is how long it takes to get from #B to #C and finally
to #D.
As far as I understand #C takes a serious amount of time and cannot be
parallelized for whatever reasons.
At the same time the number of online CPUs required to restore the VMs
state is less than the number of online CPUs required to actually
operate them in #D.
That means it would be good enough to return to userspace with a
limited number of online CPUs as fast as possible. A certain amount of
CPUs are going to be busy with restoring the VMs state, i.e. one CPU
per VM. Some remaining non-busy CPU can bringup the rest of the system
and the APs in order to be functional for #D, i.e the restore of VM
operation.
Trying to optimize this purely in kernel space by adding complexity of
dubious value is simply bogus in my opinion.
It's already possible today to limit the number of CPUs which are
initially onlined and online the rest later from user space.
There are two issue there:
a) The death by MCE broadcast problem
Quite some (contemporary) x86 CPU generations are affected by
this:
- MCE can be broadcasted to all CPUs and not only issued locally
to the CPU which triggered it.
- Any CPU which has CR4.MCE == 0, even if it sits in a wait
for INIT/SIPI state, will cause an immediate shutdown of the
machine if a broadcasted MCE is delivered.
b) Do the parallel bringup via sysfs control knob
The per CPU target state interface allows to do that today one
by one, but it's akward and has quite some overhead.
A knob to online the rest of the not yet onlined present CPUs
with the benefit of the parallel bringup mechanism is
missing.
#a) That's a risk to take by the operator.
Even the regular serialized bringup does not protect against this
issue up to the point where all present CPUs have at least
initialized CR4.
Limiting the number of APs to online early via the kernel command
line widens that window and increases the risk further by
executing user space before all APs have CR4 initialized.
But the same applies to a deferred online mechanism implemented in
the kernel where some worker brings up the not yet online APs while
the early online CPUs are already executing user space code.
#b) Is a no brainer to implement on top of this.
Conclusion
----------
Adding the basic parallel bringup mechanism as provided by this series
makes a lot of sense. Improving particular issues as pointed out in the
analysis makes sense too.
But trying to solve an application specific problem fully in the kernel
with tons of complexity, without exploring straight forward and simple
approaches first, does not make any sense at all.
Thanks,
tglx
---
Documentation/admin-guide/kernel-parameters.txt | 20
Documentation/core-api/cpu_hotplug.rst | 13
arch/Kconfig | 23 +
arch/arm/Kconfig | 1
arch/arm/include/asm/smp.h | 2
arch/arm/kernel/smp.c | 18
arch/arm64/Kconfig | 1
arch/arm64/include/asm/smp.h | 2
arch/arm64/kernel/smp.c | 14
arch/csky/Kconfig | 1
arch/csky/include/asm/smp.h | 2
arch/csky/kernel/smp.c | 8
arch/mips/Kconfig | 1
arch/mips/cavium-octeon/smp.c | 1
arch/mips/include/asm/smp-ops.h | 1
arch/mips/kernel/smp-bmips.c | 1
arch/mips/kernel/smp-cps.c | 14
arch/mips/kernel/smp.c | 8
arch/mips/loongson64/smp.c | 1
arch/parisc/Kconfig | 1
arch/parisc/kernel/process.c | 4
arch/parisc/kernel/smp.c | 7
arch/riscv/Kconfig | 1
arch/riscv/include/asm/smp.h | 2
arch/riscv/kernel/cpu-hotplug.c | 14
arch/x86/Kconfig | 45 --
arch/x86/include/asm/apic.h | 5
arch/x86/include/asm/cpu.h | 5
arch/x86/include/asm/cpumask.h | 5
arch/x86/include/asm/processor.h | 1
arch/x86/include/asm/realmode.h | 3
arch/x86/include/asm/sev-common.h | 3
arch/x86/include/asm/smp.h | 26 -
arch/x86/include/asm/topology.h | 23 -
arch/x86/include/asm/tsc.h | 2
arch/x86/kernel/acpi/sleep.c | 9
arch/x86/kernel/apic/apic.c | 22 -
arch/x86/kernel/callthunks.c | 4
arch/x86/kernel/cpu/amd.c | 2
arch/x86/kernel/cpu/cacheinfo.c | 21
arch/x86/kernel/cpu/common.c | 50 --
arch/x86/kernel/cpu/topology.c | 3
arch/x86/kernel/head_32.S | 14
arch/x86/kernel/head_64.S | 121 +++++
arch/x86/kernel/sev.c | 2
arch/x86/kernel/smp.c | 3
arch/x86/kernel/smpboot.c | 508 ++++++++----------------
arch/x86/kernel/topology.c | 98 ----
arch/x86/kernel/tsc.c | 20
arch/x86/kernel/tsc_sync.c | 36 -
arch/x86/power/cpu.c | 37 -
arch/x86/realmode/init.c | 3
arch/x86/realmode/rm/trampoline_64.S | 27 +
arch/x86/xen/enlighten_hvm.c | 11
arch/x86/xen/smp_hvm.c | 16
arch/x86/xen/smp_pv.c | 56 +-
drivers/acpi/processor_idle.c | 4
include/linux/cpu.h | 4
include/linux/cpuhotplug.h | 17
kernel/cpu.c | 397 +++++++++++++++++-
kernel/smp.c | 2
kernel/smpboot.c | 163 -------
62 files changed, 953 insertions(+), 976 deletions(-)
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Hi!
This is a complete rework of the parallel bringup patch series (V17)
https://lore.kernel.org/lkml/20230328195758.1049469-1-usama.arif@bytedance.com
to address the issues which were discovered in review:
1) The X86 microcode loader serialization requirement
https://lore.kernel.org/lkml/87v8iirxun.ffs@tglx
Microcode loading on HT enabled X86 CPUs requires that the microcode is
loaded on the primary thread. The sibling thread(s) must be in
quiescent state; either looping in a place which is aware of potential
changes by the microcode update (see late loading) or in fully quiescent
state, i.e. waiting for INIT/SIPI.
This is required by hardware/firmware on Intel. Aside of that it's a
vendor independent software correctness issue. Assume the following
sequence:
CPU1.0 CPU1.1
CPUID($A)
Load microcode.
Changes CPUID($A, $B)
CPUID($B)
CPU1.1 makes a decision on $A and $B which might be inconsistent due
to the microcode update.
The solution for this is to bringup the primary threads first and after
that the siblings. Loading microcode on the siblings is a NOOP on Intel
and on AMD it is guaranteed to only modify thread local state.
This ensures that the APs can load microcode before reaching the alive
synchronization point w/o doing any further x86 specific
synchronization between the core siblings.
2) The general design issues discussed in V16
https://lore.kernel.org/lkml/87pm8y6yme.ffs@tglx
The previous parallel bringup patches just glued this mechanism into
the existing code without a deeper analysis of the synchronization
mechanisms and without generalizing it so that the control logic is
mostly in the core code and not made an architecture specific tinker
space.
Much of that had been pointed out 2 years ago in the discussions about
the early versions of parallel bringup already.
The series is based on:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip x86/apic
and also available from git:
git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git hotplug
Background
----------
The reason why people are interested in parallel bringup is to shorten
the (kexec) reboot time of cloud servers to reduce the downtime of the
VM tenants. There are obviously other interesting use cases for this
like VM startup time, embedded devices...
The current fully serialized bringup does the following per AP:
1) Prepare callbacks (allocate, intialize, create threads)
2) Kick the AP alive (e.g. INIT/SIPI on x86)
3) Wait for the AP to report alive state
4) Let the AP continue through the atomic bringup
5) Let the AP run the threaded bringup to full online state
There are two significant delays:
#3 The time for an AP to report alive state in start_secondary() on x86
has been measured in the range between 350us and 3.5ms depending on
vendor and CPU type, BIOS microcode size etc.
#4 The atomic bringup does the microcode update. This has been measured
to take up to ~8ms on the primary threads depending on the microcode
patch size to apply.
On a two socket SKL server with 56 cores (112 threads) the boot CPU spends
on current mainline about 800ms busy waiting for the APs to come up and
apply microcode. That's more than 80% of the actual onlining procedure.
By splitting the actual bringup mechanism into two parts this can be
reduced to waiting for the first AP to report alive or if the system is
large enough the first AP is already waiting when the boot CPU finished the
wake-up of the last AP.
The actual solution comes in several parts
------------------------------------------
1) [P 1-2] General cleanups (init annotations, kernel doc...)
2) [P 3] The obvious
Avoid pointless delay calibration when TSC is synchronized across
sockets. That removes a whopping 100ms delay for the first CPU of a
socket. This is an improvement independent of parallel bringup and had
been discussed two years ago already.
2) [P 3-6] Removal of the CPU0 hotplug hack.
This was added 11 years ago with the promise to make this a real
hardware mechanism, but that never materialized. As physical CPU
hotplug is not really supported and the physical unplugging of CPU0
never materialized there is no reason to keep this cruft around. It's
just maintenance ballast for no value and the removal makes
implementing the parallel bringup feature way simpler.
3) [P 7-16] Cleanup of the existing bringup mechanism:
a) Code reorganisation so that the general hotplug specific code is
in smpboot.c and not sprinkled all over the place
b) Decouple MTRR/PAT initialization from smp_callout_mask to prepare
for replacing that mask with a hotplug core code synchronization
mechanism.
c) Make TSC synchronization function call based so that the control CPU
does not have to busy wait for nothing if synchronization is not
required.
d) Remove the smp_callin_mask synchronization point as its not longer
required due to #3c.
e) Rework the sparse_irq_lock held region in the core code so that the
next polling synchronization point in the x86 code can be removed to.
f) Due to #3e it's not longer required to spin wait for the AP to set
it's online bit. Remove wait_cpu_online() and the XENPV
counterpart. So the control CPU can directly wait for the online
idle completion by the AP and free the control CPU up for other
work.
This reduces the synchronization points in the x86 code to one, which
is the AP alive one. This synchronization will be moved to core
infrastructure in the next section.
4) [P 17-27] Replace the disconnected CPU state tracking
The extra CPU state tracking which is used by a few architectures is
completely separate from the CPU hotplug core code.
Replacing it by a variant integrated in the core hotplug machinery
allows to reduce architecture specific code and provides a generic
synchronization mechanism for (parallel) CPU bringup/teardown.
- Convert x86 over and replace the AP alive synchronization on x86 with
the core variant which removes the remaining x86 hotplug
synchronization masks.
- Convert the other architectures usage and remove the old interface
and code.
5) [P 28-30] Split the bringup into two steps
First step invokes the wakeup function on the BP, e.g. SIPI/STARTUP on
x86. The second one waits on the BP for the AP to report alive and
releases it for the complete onlining.
As the hotplug state machine allows partial bringup this allows later
to kick all APs alive in a first iteration and then bring them up
completely one by one afterwards.
6) [P 31] Switch the primary thread detection to a cpumask
This makes the parallel bringup a simple cpumask based mechanism
without tons of conditionals and checks for primary threads.
7) [P 32] Implement the parallel bringup core code
The parallel bringup looks like this:
1) Bring up the primary SMT threads to the CPUHP_KICK_AP_ALIVE step
one by one
2) Bring up the primary SMT threads to the CPUHP_ONLINE step one by
one
3) Bring up the secondary SMT threads to the CPUHP_KICK_AP_ALIVE
step one by one
4) Bring up the secondary SMT threads to the CPUHP_ONLINE
step one by one
In case that SMT is not supported this is obviously reduced to step #1
and #2.
8) [P 33-37] Prepare X86 for parallel bringup and enable it
Caveats
-------
The non X86 changes have been all compile tested. Boot and runtime
testing has only be done on a few real hardware platforms and qemu as
available. That definitely needs some help from the people who have
these systems at their fingertips.
Results and analysis
--------------------
Here are numbers for a dual socket SKL 56 cores/ 112 threads machine. All
numbers in milliseconds. The time measured is the time which the cpu_up()
call takes for each CPU and phase. It's not exact as the system is already
scheduling, handling interrupts and soft interrupts, which is obviously
skewing the picture slightly.
Baseline tip tree x86/apic branch.
total avg/CPU min max
total : 912.081 8.217 3.720 113.271
The max of 100ms is due to the silly delay calibration for the second
socket which takes 100ms and was eliminated first. Also the other initial
cleanups and improvements take some time away.
So the real baseline becomes:
total avg/CPU min max
total : 785.960 7.081 3.752 36.098
The max here is on the first CPU of the second socket. 20ms of that is due
to TSC synchronization and an extra 2ms to react on the SIPI.
With parallel bootup enabled this becomes:
total avg/CPU min max
prepare: 39.108 0.352 0.238 0.883
online : 45.166 0.407 0.170 20.357
total : 84.274 0.759 0.408 21.240
That's a factor ~9.3 reduction on average.
Looking at the 27 primary threads of socket 0 then this becomes even more
interesting:
total avg/CPU min max
total : 325.764 12.065 11.981 14.125
versus:
total avg/CPU min max
prepare: 8.945 0.331 0.238 0.834
online : 4.830 0.179 0.170 0.212
total : 13.775 0.510 0.408 1.046
So the reduction factor is ~23.5 here. That's mostly because the 20ms TSC
sync is not skewing the picture.
For all 55 primaries, i.e with the 20ms TSC sync extra for socket 1 this
becomes:
total avg/CPU min max
total : 685.489 12.463 11.975 36.098
versus:
total avg/CPU min max
prepare: 19.080 0.353 0.238 0.883
online : 30.283 0.561 0.170 20.357
total : 49.363 0.914 0.408 21.240
The TSC sync reduces the win to a factor of ~13.8
With 'tsc=reliable' on the command line the socket sync is disabled which
brings it back to the socket 0 numbers:
total avg/CPU min max
prepare: 18.970 0.351 0.231 0.874
online : 10.328 0.191 0.169 0.358
total : 29.298 0.543 0.400 1.232
Now looking at the secondary threads only:
total avg/CPU min max
total : 100.471 1.794 0.375 4.745
versus:
total avg/CPU min max
prepare: 19.753 0.353 0.257 0.512
online : 14.671 0.262 0.179 3.461
total : 34.424 0.615 0.436 3.973
Still a factor of ~3.
The average on the secondaries for the serialized bringup is significantly
lower than for the primaries because the SIPI response time is shorter and
the microcode update takes no time.
This varies wildly with the system, whether microcode in BIOS is already up
to date, how big the microcode patch is and how long the INIT/SIPI response
time is. On an AMD Zen3 machine INIT/SIPI response time is amazingly fast
(350us), but then it lacks TSC_ADJUST and does a two millisecond TSC sync
test for _every_ AP. All of this sucks...
Possible further enhancements
-----------------------------
It's definitely worthwhile to look into reducing the cross socket TSC sync
test time. It's probably safe enough to use 5ms or even 2ms instead of 20ms
on systems with TSC_ADJUST and a few other 'TSC is sane' indicators. Moving
it out of the hotplug path is eventually possible, but that needs some deep
thoughts.
Let's take the TSC sync out of the picture by adding 'tsc=reliable" to the
kernel command line. So the bringup of 111 APs takes:
total avg/CPU min max
prepare: 38.936 0.351 0.231 0.874
online : 25.231 0.227 0.169 3.465
total : 64.167 0.578 0.400 4.339
Some of the outliers are not necessarily in the state callbacks as the
system is already scheduling and handles interrupts and soft
interrupts. Haven't analyzed that yet in detail.
In the prepare stage which runs on the control CPU the larger steps are:
smpcfd:prepare 16us avg/CPU
threads:prepare 98us avg/CPU
workqueue:prepare 43us avg/CPU
trace/RB:prepare 135us avg/CPU
The trace ringbuffer initialization allocates 354 pages and 354 control
structures one by one. That probably should allocate a large page and an
array of control structures and work from there. I'm sure that would reduce
this significantly. Steven?
smpcfd does just a percpu allocation. No idea why that takes that long.
Vs. threads and workqueues. David thought about spreading out the
preparation work and do it really in parallel. That's a nice idea, but the
threads and workqueue prepare steps are self serializing. The workqueue one
has a global mutex and aside of that both steps create kernel threads which
implicitely serialize on kthreadd. alloc_percpu(), which is used by
smpcfd:prepare is also globally serialized.
The rest of the prepare steps is pretty much in the single digit
microseconds range.
On the AP side it should be possible to move some of the initialization
steps before the alive synchronization point, but that really needs a lot
of analysis whether the functions are safe to invoke that early and outside
of the cpu_hotplug_lock held region for the case of two stage parallel
bringup; see below.
The largest part is:
identify_secondary_cpu() 99us avg/CPU
Inside of identify_secondary_cpu() the largest offender:
mcheck_init() 73us avg/CPU
This part is definitly worth to be looked at whether it can be at least
partially moved to the early startup code before the alive
synchronization point. There's a lot of deep analysis required and
ideally we just rewrite the whole CPUID evaluation trainwreck
completely.
The rest of the AP side is low single digit microseconds except of:
perf/x86:starting 14us avg/CPU
smpboot/threads:online 13us avg/CPU
workqueue:online 17us avg/CPU
mm/vmstat:online 17us avg/CPU
sched:active 30us avg/CPU
sched:active is special. Onlining the first secondary HT thread on the
second socket creates a 3.2ms outlier which skews the whole picture. That's
caused by enabling the static key sched_smt_present which patches the world
and some more. For all other APs this is really in the 1us range. This
definitely could be postponed during bootup like the scheduler domain
rebuild is done after the bringup. But that's still fully serialized and
single threaded and obviously could be done later in the context of async
parallel init. It's unclear why this is different with the fully serialized
bringup where it takes significantly less time, but that's something which
needs to be investigated.
Is truly parallel bringup feasible?
-----------------------------------
In theory yes, realistically no. Why?
1) The preparation phase
Allocating memory, creating threads for the to be brought up CPU must
obviously happen on an already online CPU.
While it would be possible to bring up a subset of CPUs first and let
them do the preparation steps for groups of still offline CPUs
concurrently, the actual benefit of doing so is dubious.
The prime example is kernel thread creation, which is implicitely
serialized on kthreadd.
A simple experiment shows that 4 concurrent workers on 4 different
CPUs where each is creating 14 * 5 = 70 kernel threads are 5% slower
than a single worker creating 4 * 14 * 5 = 280 threads.
So we'd need to have multiple kthreadd instances to handle that,
which would then serialize on tasklist lock and other things.
That aside the preparation phase is also affected by the problem
below.
2) Assumptions about hotplug serialization
a) There are quite some assumptions about CPU bringup being fully
serialized across state transitions. A lot of state callbacks rely
on that and would require local locking.
Adding that local locking is surely possible, but that has several
downsides:
- It adds complexity and makes it harder for developers to get
this correct. The subtle bugs resulting out of that are going
to be interesting
- Fine grained locking has a charm, but only if the time spent
for the actual work is larger than the time required for
serialization and synchronization.
Serializing a callback which takes less than a microsecond and
then having a large number of CPUs contending on the lock will
not make it any faster at all. That's a well known issue of
parallelizing and neither made up nor kernel specific.
b) Some operations definitely require to be protected by the
cpu_hotplug_lock, especially those which affect cpumasks as the
masks are guaranteed to be stable in a cpus_read_lock()'ed region.
As this lock cannot be taken in atomic contexts, it's required
that the control CPU holds the lock write locked across these
state transitions. And no, we are not making this a spinlock just
for that and we even can't.
Just slapping a lock into the x86 specific part of the cpumask
update function does not solve anything. The relevant patch in V17
is completely useless as it only serializes the actual cpumask/map
modifications, but all read side users are hosed if the update
would be moved before the alive synchronization point, i.e. into a
non hotplug lock protected region.
Even if the hotplug lock would be held accross the whole parallel
bringup operation then this would still expose all usage of these
masks and maps in the actual hotplug state callbacks to concurrent
modifications.
And no, we are not going to expose an architecture specific raw
spinlock to the hotplug state callbacks, especially not to those
in generic code.
c) Some cpu_read_lock()'ed regions also expect that there is no CPU
state transition happening which would modify their local
state. This would again require local serialization.
3) The amount of work and churn:
- Analyze the per architecture low level startup functions plus their
descendant functions and make them ready for concurrency if
necessary.
- Analyze ~300 hotplug state callbacks and their descendant functions
and make them ready for concurrency if necessary.
- Analyze all cpus_read_lock()'ed regions and address their
requirements.
- Rewrite the core code to handle the cpu_hotplug_lock requirements
only in distinct phases of the state machine.
- Rewrite the core code to handle state callback failure and the
related rollback in the context of the new rules.
- ...
Even if some people are dedicated enough to do that, it's very
questionable whether the resulting complexity is justified.
We've spent a serious amount of time to sanitize hotplug and bring it
into a state where it is correct. This also made it reasonably simple
for developers to implement hotplug state callbacks without having to
become hotplug experts.
Breaking this completely up will result in a flood of hard to diagnose
subtle issues for sure. Who is going to deal with them?
The experience with this series so far does not make me comfortable
about that thought in any way.
Summary
-------
The obvious and low hanging fruits have to be solved first:
- The CPUID evaluation and related setup mechanisms
- The trace/ringbuffer oddity
- The sched:active oddity for the first sibling on the second socket
- Some other expensive things which I'm not seeing in my test setup due
to lack of hardware or configuration.
Anything else is pretty much wishful thinking in my opinion.
To be clear. I'm not standing in the way if there is a proper solution,
but that requires to respect the basic engineering rules:
1) Correctness first
2) Keep it maintainable
3) Keep it simple
So far this stuff failed already at #1.
I completely understand why this is important for cloud people, but
the real question to ask here is what are the actual requirements.
As far as I understand the main goal is to make a (kexec) reboot
almost invisible to VM tenants.
Now lets look at how this works:
A) Freeze VMs and persist state
B) kexec into the new kernel
C) Restore VMs from persistant memory
D) Thaw VMs
So the key problem is how long it takes to get from #B to #C and finally
to #D.
As far as I understand #C takes a serious amount of time and cannot be
parallelized for whatever reasons.
At the same time the number of online CPUs required to restore the VMs
state is less than the number of online CPUs required to actually
operate them in #D.
That means it would be good enough to return to userspace with a
limited number of online CPUs as fast as possible. A certain amount of
CPUs are going to be busy with restoring the VMs state, i.e. one CPU
per VM. Some remaining non-busy CPU can bringup the rest of the system
and the APs in order to be functional for #D, i.e the restore of VM
operation.
Trying to optimize this purely in kernel space by adding complexity of
dubious value is simply bogus in my opinion.
It's already possible today to limit the number of CPUs which are
initially onlined and online the rest later from user space.
There are two issue there:
a) The death by MCE broadcast problem
Quite some (contemporary) x86 CPU generations are affected by
this:
- MCE can be broadcasted to all CPUs and not only issued locally
to the CPU which triggered it.
- Any CPU which has CR4.MCE == 0, even if it sits in a wait
for INIT/SIPI state, will cause an immediate shutdown of the
machine if a broadcasted MCE is delivered.
b) Do the parallel bringup via sysfs control knob
The per CPU target state interface allows to do that today one
by one, but it's akward and has quite some overhead.
A knob to online the rest of the not yet onlined present CPUs
with the benefit of the parallel bringup mechanism is
missing.
#a) That's a risk to take by the operator.
Even the regular serialized bringup does not protect against this
issue up to the point where all present CPUs have at least
initialized CR4.
Limiting the number of APs to online early via the kernel command
line widens that window and increases the risk further by
executing user space before all APs have CR4 initialized.
But the same applies to a deferred online mechanism implemented in
the kernel where some worker brings up the not yet online APs while
the early online CPUs are already executing user space code.
#b) Is a no brainer to implement on top of this.
Conclusion
----------
Adding the basic parallel bringup mechanism as provided by this series
makes a lot of sense. Improving particular issues as pointed out in the
analysis makes sense too.
But trying to solve an application specific problem fully in the kernel
with tons of complexity, without exploring straight forward and simple
approaches first, does not make any sense at all.
Thanks,
tglx
---
Documentation/admin-guide/kernel-parameters.txt | 20
Documentation/core-api/cpu_hotplug.rst | 13
arch/Kconfig | 23 +
arch/arm/Kconfig | 1
arch/arm/include/asm/smp.h | 2
arch/arm/kernel/smp.c | 18
arch/arm64/Kconfig | 1
arch/arm64/include/asm/smp.h | 2
arch/arm64/kernel/smp.c | 14
arch/csky/Kconfig | 1
arch/csky/include/asm/smp.h | 2
arch/csky/kernel/smp.c | 8
arch/mips/Kconfig | 1
arch/mips/cavium-octeon/smp.c | 1
arch/mips/include/asm/smp-ops.h | 1
arch/mips/kernel/smp-bmips.c | 1
arch/mips/kernel/smp-cps.c | 14
arch/mips/kernel/smp.c | 8
arch/mips/loongson64/smp.c | 1
arch/parisc/Kconfig | 1
arch/parisc/kernel/process.c | 4
arch/parisc/kernel/smp.c | 7
arch/riscv/Kconfig | 1
arch/riscv/include/asm/smp.h | 2
arch/riscv/kernel/cpu-hotplug.c | 14
arch/x86/Kconfig | 45 --
arch/x86/include/asm/apic.h | 5
arch/x86/include/asm/cpu.h | 5
arch/x86/include/asm/cpumask.h | 5
arch/x86/include/asm/processor.h | 1
arch/x86/include/asm/realmode.h | 3
arch/x86/include/asm/sev-common.h | 3
arch/x86/include/asm/smp.h | 26 -
arch/x86/include/asm/topology.h | 23 -
arch/x86/include/asm/tsc.h | 2
arch/x86/kernel/acpi/sleep.c | 9
arch/x86/kernel/apic/apic.c | 22 -
arch/x86/kernel/callthunks.c | 4
arch/x86/kernel/cpu/amd.c | 2
arch/x86/kernel/cpu/cacheinfo.c | 21
arch/x86/kernel/cpu/common.c | 50 --
arch/x86/kernel/cpu/topology.c | 3
arch/x86/kernel/head_32.S | 14
arch/x86/kernel/head_64.S | 121 +++++
arch/x86/kernel/sev.c | 2
arch/x86/kernel/smp.c | 3
arch/x86/kernel/smpboot.c | 508 ++++++++----------------
arch/x86/kernel/topology.c | 98 ----
arch/x86/kernel/tsc.c | 20
arch/x86/kernel/tsc_sync.c | 36 -
arch/x86/power/cpu.c | 37 -
arch/x86/realmode/init.c | 3
arch/x86/realmode/rm/trampoline_64.S | 27 +
arch/x86/xen/enlighten_hvm.c | 11
arch/x86/xen/smp_hvm.c | 16
arch/x86/xen/smp_pv.c | 56 +-
drivers/acpi/processor_idle.c | 4
include/linux/cpu.h | 4
include/linux/cpuhotplug.h | 17
kernel/cpu.c | 397 +++++++++++++++++-
kernel/smp.c | 2
kernel/smpboot.c | 163 -------
62 files changed, 953 insertions(+), 976 deletions(-)
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Hi!
This is a complete rework of the parallel bringup patch series (V17)
https://lore.kernel.org/lkml/20230328195758.1049469-1-usama.arif@bytedance.com
to address the issues which were discovered in review:
1) The X86 microcode loader serialization requirement
https://lore.kernel.org/lkml/87v8iirxun.ffs@tglx
Microcode loading on HT enabled X86 CPUs requires that the microcode is
loaded on the primary thread. The sibling thread(s) must be in
quiescent state; either looping in a place which is aware of potential
changes by the microcode update (see late loading) or in fully quiescent
state, i.e. waiting for INIT/SIPI.
This is required by hardware/firmware on Intel. Aside of that it's a
vendor independent software correctness issue. Assume the following
sequence:
CPU1.0 CPU1.1
CPUID($A)
Load microcode.
Changes CPUID($A, $B)
CPUID($B)
CPU1.1 makes a decision on $A and $B which might be inconsistent due
to the microcode update.
The solution for this is to bringup the primary threads first and after
that the siblings. Loading microcode on the siblings is a NOOP on Intel
and on AMD it is guaranteed to only modify thread local state.
This ensures that the APs can load microcode before reaching the alive
synchronization point w/o doing any further x86 specific
synchronization between the core siblings.
2) The general design issues discussed in V16
https://lore.kernel.org/lkml/87pm8y6yme.ffs@tglx
The previous parallel bringup patches just glued this mechanism into
the existing code without a deeper analysis of the synchronization
mechanisms and without generalizing it so that the control logic is
mostly in the core code and not made an architecture specific tinker
space.
Much of that had been pointed out 2 years ago in the discussions about
the early versions of parallel bringup already.
The series is based on:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip x86/apic
and also available from git:
git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git hotplug
Background
----------
The reason why people are interested in parallel bringup is to shorten
the (kexec) reboot time of cloud servers to reduce the downtime of the
VM tenants. There are obviously other interesting use cases for this
like VM startup time, embedded devices...
The current fully serialized bringup does the following per AP:
1) Prepare callbacks (allocate, intialize, create threads)
2) Kick the AP alive (e.g. INIT/SIPI on x86)
3) Wait for the AP to report alive state
4) Let the AP continue through the atomic bringup
5) Let the AP run the threaded bringup to full online state
There are two significant delays:
#3 The time for an AP to report alive state in start_secondary() on x86
has been measured in the range between 350us and 3.5ms depending on
vendor and CPU type, BIOS microcode size etc.
#4 The atomic bringup does the microcode update. This has been measured
to take up to ~8ms on the primary threads depending on the microcode
patch size to apply.
On a two socket SKL server with 56 cores (112 threads) the boot CPU spends
on current mainline about 800ms busy waiting for the APs to come up and
apply microcode. That's more than 80% of the actual onlining procedure.
By splitting the actual bringup mechanism into two parts this can be
reduced to waiting for the first AP to report alive or if the system is
large enough the first AP is already waiting when the boot CPU finished the
wake-up of the last AP.
The actual solution comes in several parts
------------------------------------------
1) [P 1-2] General cleanups (init annotations, kernel doc...)
2) [P 3] The obvious
Avoid pointless delay calibration when TSC is synchronized across
sockets. That removes a whopping 100ms delay for the first CPU of a
socket. This is an improvement independent of parallel bringup and had
been discussed two years ago already.
2) [P 3-6] Removal of the CPU0 hotplug hack.
This was added 11 years ago with the promise to make this a real
hardware mechanism, but that never materialized. As physical CPU
hotplug is not really supported and the physical unplugging of CPU0
never materialized there is no reason to keep this cruft around. It's
just maintenance ballast for no value and the removal makes
implementing the parallel bringup feature way simpler.
3) [P 7-16] Cleanup of the existing bringup mechanism:
a) Code reorganisation so that the general hotplug specific code is
in smpboot.c and not sprinkled all over the place
b) Decouple MTRR/PAT initialization from smp_callout_mask to prepare
for replacing that mask with a hotplug core code synchronization
mechanism.
c) Make TSC synchronization function call based so that the control CPU
does not have to busy wait for nothing if synchronization is not
required.
d) Remove the smp_callin_mask synchronization point as its not longer
required due to #3c.
e) Rework the sparse_irq_lock held region in the core code so that the
next polling synchronization point in the x86 code can be removed to.
f) Due to #3e it's not longer required to spin wait for the AP to set
it's online bit. Remove wait_cpu_online() and the XENPV
counterpart. So the control CPU can directly wait for the online
idle completion by the AP and free the control CPU up for other
work.
This reduces the synchronization points in the x86 code to one, which
is the AP alive one. This synchronization will be moved to core
infrastructure in the next section.
4) [P 17-27] Replace the disconnected CPU state tracking
The extra CPU state tracking which is used by a few architectures is
completely separate from the CPU hotplug core code.
Replacing it by a variant integrated in the core hotplug machinery
allows to reduce architecture specific code and provides a generic
synchronization mechanism for (parallel) CPU bringup/teardown.
- Convert x86 over and replace the AP alive synchronization on x86 with
the core variant which removes the remaining x86 hotplug
synchronization masks.
- Convert the other architectures usage and remove the old interface
and code.
5) [P 28-30] Split the bringup into two steps
First step invokes the wakeup function on the BP, e.g. SIPI/STARTUP on
x86. The second one waits on the BP for the AP to report alive and
releases it for the complete onlining.
As the hotplug state machine allows partial bringup this allows later
to kick all APs alive in a first iteration and then bring them up
completely one by one afterwards.
6) [P 31] Switch the primary thread detection to a cpumask
This makes the parallel bringup a simple cpumask based mechanism
without tons of conditionals and checks for primary threads.
7) [P 32] Implement the parallel bringup core code
The parallel bringup looks like this:
1) Bring up the primary SMT threads to the CPUHP_KICK_AP_ALIVE step
one by one
2) Bring up the primary SMT threads to the CPUHP_ONLINE step one by
one
3) Bring up the secondary SMT threads to the CPUHP_KICK_AP_ALIVE
step one by one
4) Bring up the secondary SMT threads to the CPUHP_ONLINE
step one by one
In case that SMT is not supported this is obviously reduced to step #1
and #2.
8) [P 33-37] Prepare X86 for parallel bringup and enable it
Caveats
-------
The non X86 changes have been all compile tested. Boot and runtime
testing has only be done on a few real hardware platforms and qemu as
available. That definitely needs some help from the people who have
these systems at their fingertips.
Results and analysis
--------------------
Here are numbers for a dual socket SKL 56 cores/ 112 threads machine. All
numbers in milliseconds. The time measured is the time which the cpu_up()
call takes for each CPU and phase. It's not exact as the system is already
scheduling, handling interrupts and soft interrupts, which is obviously
skewing the picture slightly.
Baseline tip tree x86/apic branch.
total avg/CPU min max
total : 912.081 8.217 3.720 113.271
The max of 100ms is due to the silly delay calibration for the second
socket which takes 100ms and was eliminated first. Also the other initial
cleanups and improvements take some time away.
So the real baseline becomes:
total avg/CPU min max
total : 785.960 7.081 3.752 36.098
The max here is on the first CPU of the second socket. 20ms of that is due
to TSC synchronization and an extra 2ms to react on the SIPI.
With parallel bootup enabled this becomes:
total avg/CPU min max
prepare: 39.108 0.352 0.238 0.883
online : 45.166 0.407 0.170 20.357
total : 84.274 0.759 0.408 21.240
That's a factor ~9.3 reduction on average.
Looking at the 27 primary threads of socket 0 then this becomes even more
interesting:
total avg/CPU min max
total : 325.764 12.065 11.981 14.125
versus:
total avg/CPU min max
prepare: 8.945 0.331 0.238 0.834
online : 4.830 0.179 0.170 0.212
total : 13.775 0.510 0.408 1.046
So the reduction factor is ~23.5 here. That's mostly because the 20ms TSC
sync is not skewing the picture.
For all 55 primaries, i.e with the 20ms TSC sync extra for socket 1 this
becomes:
total avg/CPU min max
total : 685.489 12.463 11.975 36.098
versus:
total avg/CPU min max
prepare: 19.080 0.353 0.238 0.883
online : 30.283 0.561 0.170 20.357
total : 49.363 0.914 0.408 21.240
The TSC sync reduces the win to a factor of ~13.8
With 'tsc=reliable' on the command line the socket sync is disabled which
brings it back to the socket 0 numbers:
total avg/CPU min max
prepare: 18.970 0.351 0.231 0.874
online : 10.328 0.191 0.169 0.358
total : 29.298 0.543 0.400 1.232
Now looking at the secondary threads only:
total avg/CPU min max
total : 100.471 1.794 0.375 4.745
versus:
total avg/CPU min max
prepare: 19.753 0.353 0.257 0.512
online : 14.671 0.262 0.179 3.461
total : 34.424 0.615 0.436 3.973
Still a factor of ~3.
The average on the secondaries for the serialized bringup is significantly
lower than for the primaries because the SIPI response time is shorter and
the microcode update takes no time.
This varies wildly with the system, whether microcode in BIOS is already up
to date, how big the microcode patch is and how long the INIT/SIPI response
time is. On an AMD Zen3 machine INIT/SIPI response time is amazingly fast
(350us), but then it lacks TSC_ADJUST and does a two millisecond TSC sync
test for _every_ AP. All of this sucks...
Possible further enhancements
-----------------------------
It's definitely worthwhile to look into reducing the cross socket TSC sync
test time. It's probably safe enough to use 5ms or even 2ms instead of 20ms
on systems with TSC_ADJUST and a few other 'TSC is sane' indicators. Moving
it out of the hotplug path is eventually possible, but that needs some deep
thoughts.
Let's take the TSC sync out of the picture by adding 'tsc=reliable" to the
kernel command line. So the bringup of 111 APs takes:
total avg/CPU min max
prepare: 38.936 0.351 0.231 0.874
online : 25.231 0.227 0.169 3.465
total : 64.167 0.578 0.400 4.339
Some of the outliers are not necessarily in the state callbacks as the
system is already scheduling and handles interrupts and soft
interrupts. Haven't analyzed that yet in detail.
In the prepare stage which runs on the control CPU the larger steps are:
smpcfd:prepare 16us avg/CPU
threads:prepare 98us avg/CPU
workqueue:prepare 43us avg/CPU
trace/RB:prepare 135us avg/CPU
The trace ringbuffer initialization allocates 354 pages and 354 control
structures one by one. That probably should allocate a large page and an
array of control structures and work from there. I'm sure that would reduce
this significantly. Steven?
smpcfd does just a percpu allocation. No idea why that takes that long.
Vs. threads and workqueues. David thought about spreading out the
preparation work and do it really in parallel. That's a nice idea, but the
threads and workqueue prepare steps are self serializing. The workqueue one
has a global mutex and aside of that both steps create kernel threads which
implicitely serialize on kthreadd. alloc_percpu(), which is used by
smpcfd:prepare is also globally serialized.
The rest of the prepare steps is pretty much in the single digit
microseconds range.
On the AP side it should be possible to move some of the initialization
steps before the alive synchronization point, but that really needs a lot
of analysis whether the functions are safe to invoke that early and outside
of the cpu_hotplug_lock held region for the case of two stage parallel
bringup; see below.
The largest part is:
identify_secondary_cpu() 99us avg/CPU
Inside of identify_secondary_cpu() the largest offender:
mcheck_init() 73us avg/CPU
This part is definitly worth to be looked at whether it can be at least
partially moved to the early startup code before the alive
synchronization point. There's a lot of deep analysis required and
ideally we just rewrite the whole CPUID evaluation trainwreck
completely.
The rest of the AP side is low single digit microseconds except of:
perf/x86:starting 14us avg/CPU
smpboot/threads:online 13us avg/CPU
workqueue:online 17us avg/CPU
mm/vmstat:online 17us avg/CPU
sched:active 30us avg/CPU
sched:active is special. Onlining the first secondary HT thread on the
second socket creates a 3.2ms outlier which skews the whole picture. That's
caused by enabling the static key sched_smt_present which patches the world
and some more. For all other APs this is really in the 1us range. This
definitely could be postponed during bootup like the scheduler domain
rebuild is done after the bringup. But that's still fully serialized and
single threaded and obviously could be done later in the context of async
parallel init. It's unclear why this is different with the fully serialized
bringup where it takes significantly less time, but that's something which
needs to be investigated.
Is truly parallel bringup feasible?
-----------------------------------
In theory yes, realistically no. Why?
1) The preparation phase
Allocating memory, creating threads for the to be brought up CPU must
obviously happen on an already online CPU.
While it would be possible to bring up a subset of CPUs first and let
them do the preparation steps for groups of still offline CPUs
concurrently, the actual benefit of doing so is dubious.
The prime example is kernel thread creation, which is implicitely
serialized on kthreadd.
A simple experiment shows that 4 concurrent workers on 4 different
CPUs where each is creating 14 * 5 = 70 kernel threads are 5% slower
than a single worker creating 4 * 14 * 5 = 280 threads.
So we'd need to have multiple kthreadd instances to handle that,
which would then serialize on tasklist lock and other things.
That aside the preparation phase is also affected by the problem
below.
2) Assumptions about hotplug serialization
a) There are quite some assumptions about CPU bringup being fully
serialized across state transitions. A lot of state callbacks rely
on that and would require local locking.
Adding that local locking is surely possible, but that has several
downsides:
- It adds complexity and makes it harder for developers to get
this correct. The subtle bugs resulting out of that are going
to be interesting
- Fine grained locking has a charm, but only if the time spent
for the actual work is larger than the time required for
serialization and synchronization.
Serializing a callback which takes less than a microsecond and
then having a large number of CPUs contending on the lock will
not make it any faster at all. That's a well known issue of
parallelizing and neither made up nor kernel specific.
b) Some operations definitely require to be protected by the
cpu_hotplug_lock, especially those which affect cpumasks as the
masks are guaranteed to be stable in a cpus_read_lock()'ed region.
As this lock cannot be taken in atomic contexts, it's required
that the control CPU holds the lock write locked across these
state transitions. And no, we are not making this a spinlock just
for that and we even can't.
Just slapping a lock into the x86 specific part of the cpumask
update function does not solve anything. The relevant patch in V17
is completely useless as it only serializes the actual cpumask/map
modifications, but all read side users are hosed if the update
would be moved before the alive synchronization point, i.e. into a
non hotplug lock protected region.
Even if the hotplug lock would be held accross the whole parallel
bringup operation then this would still expose all usage of these
masks and maps in the actual hotplug state callbacks to concurrent
modifications.
And no, we are not going to expose an architecture specific raw
spinlock to the hotplug state callbacks, especially not to those
in generic code.
c) Some cpu_read_lock()'ed regions also expect that there is no CPU
state transition happening which would modify their local
state. This would again require local serialization.
3) The amount of work and churn:
- Analyze the per architecture low level startup functions plus their
descendant functions and make them ready for concurrency if
necessary.
- Analyze ~300 hotplug state callbacks and their descendant functions
and make them ready for concurrency if necessary.
- Analyze all cpus_read_lock()'ed regions and address their
requirements.
- Rewrite the core code to handle the cpu_hotplug_lock requirements
only in distinct phases of the state machine.
- Rewrite the core code to handle state callback failure and the
related rollback in the context of the new rules.
- ...
Even if some people are dedicated enough to do that, it's very
questionable whether the resulting complexity is justified.
We've spent a serious amount of time to sanitize hotplug and bring it
into a state where it is correct. This also made it reasonably simple
for developers to implement hotplug state callbacks without having to
become hotplug experts.
Breaking this completely up will result in a flood of hard to diagnose
subtle issues for sure. Who is going to deal with them?
The experience with this series so far does not make me comfortable
about that thought in any way.
Summary
-------
The obvious and low hanging fruits have to be solved first:
- The CPUID evaluation and related setup mechanisms
- The trace/ringbuffer oddity
- The sched:active oddity for the first sibling on the second socket
- Some other expensive things which I'm not seeing in my test setup due
to lack of hardware or configuration.
Anything else is pretty much wishful thinking in my opinion.
To be clear. I'm not standing in the way if there is a proper solution,
but that requires to respect the basic engineering rules:
1) Correctness first
2) Keep it maintainable
3) Keep it simple
So far this stuff failed already at #1.
I completely understand why this is important for cloud people, but
the real question to ask here is what are the actual requirements.
As far as I understand the main goal is to make a (kexec) reboot
almost invisible to VM tenants.
Now lets look at how this works:
A) Freeze VMs and persist state
B) kexec into the new kernel
C) Restore VMs from persistant memory
D) Thaw VMs
So the key problem is how long it takes to get from #B to #C and finally
to #D.
As far as I understand #C takes a serious amount of time and cannot be
parallelized for whatever reasons.
At the same time the number of online CPUs required to restore the VMs
state is less than the number of online CPUs required to actually
operate them in #D.
That means it would be good enough to return to userspace with a
limited number of online CPUs as fast as possible. A certain amount of
CPUs are going to be busy with restoring the VMs state, i.e. one CPU
per VM. Some remaining non-busy CPU can bringup the rest of the system
and the APs in order to be functional for #D, i.e the restore of VM
operation.
Trying to optimize this purely in kernel space by adding complexity of
dubious value is simply bogus in my opinion.
It's already possible today to limit the number of CPUs which are
initially onlined and online the rest later from user space.
There are two issue there:
a) The death by MCE broadcast problem
Quite some (contemporary) x86 CPU generations are affected by
this:
- MCE can be broadcasted to all CPUs and not only issued locally
to the CPU which triggered it.
- Any CPU which has CR4.MCE == 0, even if it sits in a wait
for INIT/SIPI state, will cause an immediate shutdown of the
machine if a broadcasted MCE is delivered.
b) Do the parallel bringup via sysfs control knob
The per CPU target state interface allows to do that today one
by one, but it's akward and has quite some overhead.
A knob to online the rest of the not yet onlined present CPUs
with the benefit of the parallel bringup mechanism is
missing.
#a) That's a risk to take by the operator.
Even the regular serialized bringup does not protect against this
issue up to the point where all present CPUs have at least
initialized CR4.
Limiting the number of APs to online early via the kernel command
line widens that window and increases the risk further by
executing user space before all APs have CR4 initialized.
But the same applies to a deferred online mechanism implemented in
the kernel where some worker brings up the not yet online APs while
the early online CPUs are already executing user space code.
#b) Is a no brainer to implement on top of this.
Conclusion
----------
Adding the basic parallel bringup mechanism as provided by this series
makes a lot of sense. Improving particular issues as pointed out in the
analysis makes sense too.
But trying to solve an application specific problem fully in the kernel
with tons of complexity, without exploring straight forward and simple
approaches first, does not make any sense at all.
Thanks,
tglx
---
Documentation/admin-guide/kernel-parameters.txt | 20
Documentation/core-api/cpu_hotplug.rst | 13
arch/Kconfig | 23 +
arch/arm/Kconfig | 1
arch/arm/include/asm/smp.h | 2
arch/arm/kernel/smp.c | 18
arch/arm64/Kconfig | 1
arch/arm64/include/asm/smp.h | 2
arch/arm64/kernel/smp.c | 14
arch/csky/Kconfig | 1
arch/csky/include/asm/smp.h | 2
arch/csky/kernel/smp.c | 8
arch/mips/Kconfig | 1
arch/mips/cavium-octeon/smp.c | 1
arch/mips/include/asm/smp-ops.h | 1
arch/mips/kernel/smp-bmips.c | 1
arch/mips/kernel/smp-cps.c | 14
arch/mips/kernel/smp.c | 8
arch/mips/loongson64/smp.c | 1
arch/parisc/Kconfig | 1
arch/parisc/kernel/process.c | 4
arch/parisc/kernel/smp.c | 7
arch/riscv/Kconfig | 1
arch/riscv/include/asm/smp.h | 2
arch/riscv/kernel/cpu-hotplug.c | 14
arch/x86/Kconfig | 45 --
arch/x86/include/asm/apic.h | 5
arch/x86/include/asm/cpu.h | 5
arch/x86/include/asm/cpumask.h | 5
arch/x86/include/asm/processor.h | 1
arch/x86/include/asm/realmode.h | 3
arch/x86/include/asm/sev-common.h | 3
arch/x86/include/asm/smp.h | 26 -
arch/x86/include/asm/topology.h | 23 -
arch/x86/include/asm/tsc.h | 2
arch/x86/kernel/acpi/sleep.c | 9
arch/x86/kernel/apic/apic.c | 22 -
arch/x86/kernel/callthunks.c | 4
arch/x86/kernel/cpu/amd.c | 2
arch/x86/kernel/cpu/cacheinfo.c | 21
arch/x86/kernel/cpu/common.c | 50 --
arch/x86/kernel/cpu/topology.c | 3
arch/x86/kernel/head_32.S | 14
arch/x86/kernel/head_64.S | 121 +++++
arch/x86/kernel/sev.c | 2
arch/x86/kernel/smp.c | 3
arch/x86/kernel/smpboot.c | 508 ++++++++----------------
arch/x86/kernel/topology.c | 98 ----
arch/x86/kernel/tsc.c | 20
arch/x86/kernel/tsc_sync.c | 36 -
arch/x86/power/cpu.c | 37 -
arch/x86/realmode/init.c | 3
arch/x86/realmode/rm/trampoline_64.S | 27 +
arch/x86/xen/enlighten_hvm.c | 11
arch/x86/xen/smp_hvm.c | 16
arch/x86/xen/smp_pv.c | 56 +-
drivers/acpi/processor_idle.c | 4
include/linux/cpu.h | 4
include/linux/cpuhotplug.h | 17
kernel/cpu.c | 397 +++++++++++++++++-
kernel/smp.c | 2
kernel/smpboot.c | 163 -------
62 files changed, 953 insertions(+), 976 deletions(-)
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 01/37] x86/smpboot: Cleanup topology_phys_to_logical_pkg()/die()
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-14 23:44 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Make topology_phys_to_logical_pkg_die() static as it's only used in
smpboot.c and fixup the kernel-doc warnings for both functions.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/include/asm/topology.h | 3 ---
arch/x86/kernel/smpboot.c | 10 ++++++----
2 files changed, 6 insertions(+), 7 deletions(-)
--- a/arch/x86/include/asm/topology.h
+++ b/arch/x86/include/asm/topology.h
@@ -139,7 +139,6 @@ static inline int topology_max_smt_threa
int topology_update_package_map(unsigned int apicid, unsigned int cpu);
int topology_update_die_map(unsigned int dieid, unsigned int cpu);
int topology_phys_to_logical_pkg(unsigned int pkg);
-int topology_phys_to_logical_die(unsigned int die, unsigned int cpu);
bool topology_is_primary_thread(unsigned int cpu);
bool topology_smt_supported(void);
#else
@@ -149,8 +148,6 @@ topology_update_package_map(unsigned int
static inline int
topology_update_die_map(unsigned int dieid, unsigned int cpu) { return 0; }
static inline int topology_phys_to_logical_pkg(unsigned int pkg) { return 0; }
-static inline int topology_phys_to_logical_die(unsigned int die,
- unsigned int cpu) { return 0; }
static inline int topology_max_die_per_package(void) { return 1; }
static inline int topology_max_smt_threads(void) { return 1; }
static inline bool topology_is_primary_thread(unsigned int cpu) { return true; }
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -288,6 +288,7 @@ bool topology_smt_supported(void)
/**
* topology_phys_to_logical_pkg - Map a physical package id to a logical
+ * @phys_pkg: The physical package id to map
*
* Returns logical package id or -1 if not found
*/
@@ -304,15 +305,17 @@ int topology_phys_to_logical_pkg(unsigne
return -1;
}
EXPORT_SYMBOL(topology_phys_to_logical_pkg);
+
/**
* topology_phys_to_logical_die - Map a physical die id to logical
+ * @die_id: The physical die id to map
+ * @cur_cpu: The CPU for which the mapping is done
*
* Returns logical die id or -1 if not found
*/
-int topology_phys_to_logical_die(unsigned int die_id, unsigned int cur_cpu)
+static int topology_phys_to_logical_die(unsigned int die_id, unsigned int cur_cpu)
{
- int cpu;
- int proc_id = cpu_data(cur_cpu).phys_proc_id;
+ int cpu, proc_id = cpu_data(cur_cpu).phys_proc_id;
for_each_possible_cpu(cpu) {
struct cpuinfo_x86 *c = &cpu_data(cpu);
@@ -323,7 +326,6 @@ int topology_phys_to_logical_die(unsigne
}
return -1;
}
-EXPORT_SYMBOL(topology_phys_to_logical_die);
/**
* topology_update_package_map - Update the physical to logical package map
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 01/37] x86/smpboot: Cleanup topology_phys_to_logical_pkg()/die()
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Make topology_phys_to_logical_pkg_die() static as it's only used in
smpboot.c and fixup the kernel-doc warnings for both functions.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/include/asm/topology.h | 3 ---
arch/x86/kernel/smpboot.c | 10 ++++++----
2 files changed, 6 insertions(+), 7 deletions(-)
--- a/arch/x86/include/asm/topology.h
+++ b/arch/x86/include/asm/topology.h
@@ -139,7 +139,6 @@ static inline int topology_max_smt_threa
int topology_update_package_map(unsigned int apicid, unsigned int cpu);
int topology_update_die_map(unsigned int dieid, unsigned int cpu);
int topology_phys_to_logical_pkg(unsigned int pkg);
-int topology_phys_to_logical_die(unsigned int die, unsigned int cpu);
bool topology_is_primary_thread(unsigned int cpu);
bool topology_smt_supported(void);
#else
@@ -149,8 +148,6 @@ topology_update_package_map(unsigned int
static inline int
topology_update_die_map(unsigned int dieid, unsigned int cpu) { return 0; }
static inline int topology_phys_to_logical_pkg(unsigned int pkg) { return 0; }
-static inline int topology_phys_to_logical_die(unsigned int die,
- unsigned int cpu) { return 0; }
static inline int topology_max_die_per_package(void) { return 1; }
static inline int topology_max_smt_threads(void) { return 1; }
static inline bool topology_is_primary_thread(unsigned int cpu) { return true; }
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -288,6 +288,7 @@ bool topology_smt_supported(void)
/**
* topology_phys_to_logical_pkg - Map a physical package id to a logical
+ * @phys_pkg: The physical package id to map
*
* Returns logical package id or -1 if not found
*/
@@ -304,15 +305,17 @@ int topology_phys_to_logical_pkg(unsigne
return -1;
}
EXPORT_SYMBOL(topology_phys_to_logical_pkg);
+
/**
* topology_phys_to_logical_die - Map a physical die id to logical
+ * @die_id: The physical die id to map
+ * @cur_cpu: The CPU for which the mapping is done
*
* Returns logical die id or -1 if not found
*/
-int topology_phys_to_logical_die(unsigned int die_id, unsigned int cur_cpu)
+static int topology_phys_to_logical_die(unsigned int die_id, unsigned int cur_cpu)
{
- int cpu;
- int proc_id = cpu_data(cur_cpu).phys_proc_id;
+ int cpu, proc_id = cpu_data(cur_cpu).phys_proc_id;
for_each_possible_cpu(cpu) {
struct cpuinfo_x86 *c = &cpu_data(cpu);
@@ -323,7 +326,6 @@ int topology_phys_to_logical_die(unsigne
}
return -1;
}
-EXPORT_SYMBOL(topology_phys_to_logical_die);
/**
* topology_update_package_map - Update the physical to logical package map
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 01/37] x86/smpboot: Cleanup topology_phys_to_logical_pkg()/die()
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Make topology_phys_to_logical_pkg_die() static as it's only used in
smpboot.c and fixup the kernel-doc warnings for both functions.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/include/asm/topology.h | 3 ---
arch/x86/kernel/smpboot.c | 10 ++++++----
2 files changed, 6 insertions(+), 7 deletions(-)
--- a/arch/x86/include/asm/topology.h
+++ b/arch/x86/include/asm/topology.h
@@ -139,7 +139,6 @@ static inline int topology_max_smt_threa
int topology_update_package_map(unsigned int apicid, unsigned int cpu);
int topology_update_die_map(unsigned int dieid, unsigned int cpu);
int topology_phys_to_logical_pkg(unsigned int pkg);
-int topology_phys_to_logical_die(unsigned int die, unsigned int cpu);
bool topology_is_primary_thread(unsigned int cpu);
bool topology_smt_supported(void);
#else
@@ -149,8 +148,6 @@ topology_update_package_map(unsigned int
static inline int
topology_update_die_map(unsigned int dieid, unsigned int cpu) { return 0; }
static inline int topology_phys_to_logical_pkg(unsigned int pkg) { return 0; }
-static inline int topology_phys_to_logical_die(unsigned int die,
- unsigned int cpu) { return 0; }
static inline int topology_max_die_per_package(void) { return 1; }
static inline int topology_max_smt_threads(void) { return 1; }
static inline bool topology_is_primary_thread(unsigned int cpu) { return true; }
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -288,6 +288,7 @@ bool topology_smt_supported(void)
/**
* topology_phys_to_logical_pkg - Map a physical package id to a logical
+ * @phys_pkg: The physical package id to map
*
* Returns logical package id or -1 if not found
*/
@@ -304,15 +305,17 @@ int topology_phys_to_logical_pkg(unsigne
return -1;
}
EXPORT_SYMBOL(topology_phys_to_logical_pkg);
+
/**
* topology_phys_to_logical_die - Map a physical die id to logical
+ * @die_id: The physical die id to map
+ * @cur_cpu: The CPU for which the mapping is done
*
* Returns logical die id or -1 if not found
*/
-int topology_phys_to_logical_die(unsigned int die_id, unsigned int cur_cpu)
+static int topology_phys_to_logical_die(unsigned int die_id, unsigned int cur_cpu)
{
- int cpu;
- int proc_id = cpu_data(cur_cpu).phys_proc_id;
+ int cpu, proc_id = cpu_data(cur_cpu).phys_proc_id;
for_each_possible_cpu(cpu) {
struct cpuinfo_x86 *c = &cpu_data(cpu);
@@ -323,7 +326,6 @@ int topology_phys_to_logical_die(unsigne
}
return -1;
}
-EXPORT_SYMBOL(topology_phys_to_logical_die);
/**
* topology_update_package_map - Update the physical to logical package map
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 02/37] cpu/hotplug: Mark arch_disable_smp_support() and bringup_nonboot_cpus() __init
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-14 23:44 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
No point in keeping them around.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/kernel/smpboot.c | 4 ++--
kernel/cpu.c | 2 +-
kernel/smp.c | 2 +-
3 files changed, 4 insertions(+), 4 deletions(-)
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -1269,9 +1269,9 @@ int native_cpu_up(unsigned int cpu, stru
}
/**
- * arch_disable_smp_support() - disables SMP support for x86 at runtime
+ * arch_disable_smp_support() - Disables SMP support for x86 at boottime
*/
-void arch_disable_smp_support(void)
+void __init arch_disable_smp_support(void)
{
disable_ioapic_support();
}
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -1502,7 +1502,7 @@ int bringup_hibernate_cpu(unsigned int s
return 0;
}
-void bringup_nonboot_cpus(unsigned int setup_max_cpus)
+void __init bringup_nonboot_cpus(unsigned int setup_max_cpus)
{
unsigned int cpu;
--- a/kernel/smp.c
+++ b/kernel/smp.c
@@ -1051,7 +1051,7 @@ EXPORT_SYMBOL(setup_max_cpus);
* SMP mode to <NUM>.
*/
-void __weak arch_disable_smp_support(void) { }
+void __weak __init arch_disable_smp_support(void) { }
static int __init nosmp(char *str)
{
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 02/37] cpu/hotplug: Mark arch_disable_smp_support() and bringup_nonboot_cpus() __init
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
No point in keeping them around.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/kernel/smpboot.c | 4 ++--
kernel/cpu.c | 2 +-
kernel/smp.c | 2 +-
3 files changed, 4 insertions(+), 4 deletions(-)
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -1269,9 +1269,9 @@ int native_cpu_up(unsigned int cpu, stru
}
/**
- * arch_disable_smp_support() - disables SMP support for x86 at runtime
+ * arch_disable_smp_support() - Disables SMP support for x86 at boottime
*/
-void arch_disable_smp_support(void)
+void __init arch_disable_smp_support(void)
{
disable_ioapic_support();
}
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -1502,7 +1502,7 @@ int bringup_hibernate_cpu(unsigned int s
return 0;
}
-void bringup_nonboot_cpus(unsigned int setup_max_cpus)
+void __init bringup_nonboot_cpus(unsigned int setup_max_cpus)
{
unsigned int cpu;
--- a/kernel/smp.c
+++ b/kernel/smp.c
@@ -1051,7 +1051,7 @@ EXPORT_SYMBOL(setup_max_cpus);
* SMP mode to <NUM>.
*/
-void __weak arch_disable_smp_support(void) { }
+void __weak __init arch_disable_smp_support(void) { }
static int __init nosmp(char *str)
{
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 02/37] cpu/hotplug: Mark arch_disable_smp_support() and bringup_nonboot_cpus() __init
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
No point in keeping them around.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/kernel/smpboot.c | 4 ++--
kernel/cpu.c | 2 +-
kernel/smp.c | 2 +-
3 files changed, 4 insertions(+), 4 deletions(-)
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -1269,9 +1269,9 @@ int native_cpu_up(unsigned int cpu, stru
}
/**
- * arch_disable_smp_support() - disables SMP support for x86 at runtime
+ * arch_disable_smp_support() - Disables SMP support for x86 at boottime
*/
-void arch_disable_smp_support(void)
+void __init arch_disable_smp_support(void)
{
disable_ioapic_support();
}
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -1502,7 +1502,7 @@ int bringup_hibernate_cpu(unsigned int s
return 0;
}
-void bringup_nonboot_cpus(unsigned int setup_max_cpus)
+void __init bringup_nonboot_cpus(unsigned int setup_max_cpus)
{
unsigned int cpu;
--- a/kernel/smp.c
+++ b/kernel/smp.c
@@ -1051,7 +1051,7 @@ EXPORT_SYMBOL(setup_max_cpus);
* SMP mode to <NUM>.
*/
-void __weak arch_disable_smp_support(void) { }
+void __weak __init arch_disable_smp_support(void) { }
static int __init nosmp(char *str)
{
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 03/37] x86/smpboot: Avoid pointless delay calibration is TSC is synchronized
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-14 23:44 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
When TSC is synchronized across sockets then there is no reason in
calibrating the delay for the first CPU which comes up on a socket.
Just reuse the existing calibration value.
This removes 100ms pointlessly wasted time from CPU hotplug.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/kernel/smpboot.c | 38 ++++++++++++++++++++++++--------------
arch/x86/kernel/tsc.c | 20 ++++++++++++++++----
2 files changed, 40 insertions(+), 18 deletions(-)
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -178,10 +178,7 @@ static void smp_callin(void)
*/
apic_ap_setup();
- /*
- * Save our processor parameters. Note: this information
- * is needed for clock calibration.
- */
+ /* Save our processor parameters. */
smp_store_cpu_info(cpuid);
/*
@@ -192,14 +189,6 @@ static void smp_callin(void)
ap_init_aperfmperf();
- /*
- * Get our bogomips.
- * Update loops_per_jiffy in cpu_data. Previous call to
- * smp_store_cpu_info() stored a value that is close but not as
- * accurate as the value just calculated.
- */
- calibrate_delay();
- cpu_data(cpuid).loops_per_jiffy = loops_per_jiffy;
pr_debug("Stack at about %p\n", &cpuid);
wmb();
@@ -212,8 +201,24 @@ static void smp_callin(void)
cpumask_set_cpu(cpuid, cpu_callin_mask);
}
+static void ap_calibrate_delay(void)
+{
+ /*
+ * Calibrate the delay loop and update loops_per_jiffy in cpu_data.
+ * smp_store_cpu_info() stored a value that is close but not as
+ * accurate as the value just calculated.
+ *
+ * As this is invoked after the TSC synchronization check,
+ * calibrate_delay_is_known() will skip the calibration routine
+ * when TSC is synchronized across sockets.
+ */
+ calibrate_delay();
+ cpu_data(smp_processor_id()).loops_per_jiffy = loops_per_jiffy;
+}
+
static int cpu0_logical_apicid;
static int enable_start_cpu0;
+
/*
* Activate a secondary processor.
*/
@@ -240,10 +245,15 @@ static void notrace start_secondary(void
/* otherwise gcc will move up smp_processor_id before the cpu_init */
barrier();
+ /* Check TSC synchronization with the control CPU: */
+ check_tsc_sync_target();
+
/*
- * Check TSC synchronization with the boot CPU:
+ * Calibrate the delay loop after the TSC synchronization check.
+ * This allows to skip the calibration when TSC is synchronized
+ * across sockets.
*/
- check_tsc_sync_target();
+ ap_calibrate_delay();
speculative_store_bypass_ht_init();
--- a/arch/x86/kernel/tsc.c
+++ b/arch/x86/kernel/tsc.c
@@ -1598,10 +1598,7 @@ void __init tsc_init(void)
#ifdef CONFIG_SMP
/*
- * If we have a constant TSC and are using the TSC for the delay loop,
- * we can skip clock calibration if another cpu in the same socket has already
- * been calibrated. This assumes that CONSTANT_TSC applies to all
- * cpus in the socket - this should be a safe assumption.
+ * Check whether existing calibration data can be reused.
*/
unsigned long calibrate_delay_is_known(void)
{
@@ -1609,6 +1606,21 @@ unsigned long calibrate_delay_is_known(v
int constant_tsc = cpu_has(&cpu_data(cpu), X86_FEATURE_CONSTANT_TSC);
const struct cpumask *mask = topology_core_cpumask(cpu);
+ /*
+ * If TSC has constant frequency and TSC is synchronized across
+ * sockets then reuse CPU0 calibration.
+ */
+ if (constant_tsc && !tsc_unstable)
+ return cpu_data(0).loops_per_jiffy;
+
+ /*
+ * If TSC has constant frequency and TSC is not synchronized across
+ * sockets and this is not the first CPU in the socket, then reuse
+ * the calibration value of an already online CPU on that socket.
+ *
+ * This assumes that CONSTANT_TSC is consistent for all CPUs in a
+ * socket.
+ */
if (!constant_tsc || !mask)
return 0;
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 03/37] x86/smpboot: Avoid pointless delay calibration is TSC is synchronized
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
When TSC is synchronized across sockets then there is no reason in
calibrating the delay for the first CPU which comes up on a socket.
Just reuse the existing calibration value.
This removes 100ms pointlessly wasted time from CPU hotplug.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/kernel/smpboot.c | 38 ++++++++++++++++++++++++--------------
arch/x86/kernel/tsc.c | 20 ++++++++++++++++----
2 files changed, 40 insertions(+), 18 deletions(-)
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -178,10 +178,7 @@ static void smp_callin(void)
*/
apic_ap_setup();
- /*
- * Save our processor parameters. Note: this information
- * is needed for clock calibration.
- */
+ /* Save our processor parameters. */
smp_store_cpu_info(cpuid);
/*
@@ -192,14 +189,6 @@ static void smp_callin(void)
ap_init_aperfmperf();
- /*
- * Get our bogomips.
- * Update loops_per_jiffy in cpu_data. Previous call to
- * smp_store_cpu_info() stored a value that is close but not as
- * accurate as the value just calculated.
- */
- calibrate_delay();
- cpu_data(cpuid).loops_per_jiffy = loops_per_jiffy;
pr_debug("Stack at about %p\n", &cpuid);
wmb();
@@ -212,8 +201,24 @@ static void smp_callin(void)
cpumask_set_cpu(cpuid, cpu_callin_mask);
}
+static void ap_calibrate_delay(void)
+{
+ /*
+ * Calibrate the delay loop and update loops_per_jiffy in cpu_data.
+ * smp_store_cpu_info() stored a value that is close but not as
+ * accurate as the value just calculated.
+ *
+ * As this is invoked after the TSC synchronization check,
+ * calibrate_delay_is_known() will skip the calibration routine
+ * when TSC is synchronized across sockets.
+ */
+ calibrate_delay();
+ cpu_data(smp_processor_id()).loops_per_jiffy = loops_per_jiffy;
+}
+
static int cpu0_logical_apicid;
static int enable_start_cpu0;
+
/*
* Activate a secondary processor.
*/
@@ -240,10 +245,15 @@ static void notrace start_secondary(void
/* otherwise gcc will move up smp_processor_id before the cpu_init */
barrier();
+ /* Check TSC synchronization with the control CPU: */
+ check_tsc_sync_target();
+
/*
- * Check TSC synchronization with the boot CPU:
+ * Calibrate the delay loop after the TSC synchronization check.
+ * This allows to skip the calibration when TSC is synchronized
+ * across sockets.
*/
- check_tsc_sync_target();
+ ap_calibrate_delay();
speculative_store_bypass_ht_init();
--- a/arch/x86/kernel/tsc.c
+++ b/arch/x86/kernel/tsc.c
@@ -1598,10 +1598,7 @@ void __init tsc_init(void)
#ifdef CONFIG_SMP
/*
- * If we have a constant TSC and are using the TSC for the delay loop,
- * we can skip clock calibration if another cpu in the same socket has already
- * been calibrated. This assumes that CONSTANT_TSC applies to all
- * cpus in the socket - this should be a safe assumption.
+ * Check whether existing calibration data can be reused.
*/
unsigned long calibrate_delay_is_known(void)
{
@@ -1609,6 +1606,21 @@ unsigned long calibrate_delay_is_known(v
int constant_tsc = cpu_has(&cpu_data(cpu), X86_FEATURE_CONSTANT_TSC);
const struct cpumask *mask = topology_core_cpumask(cpu);
+ /*
+ * If TSC has constant frequency and TSC is synchronized across
+ * sockets then reuse CPU0 calibration.
+ */
+ if (constant_tsc && !tsc_unstable)
+ return cpu_data(0).loops_per_jiffy;
+
+ /*
+ * If TSC has constant frequency and TSC is not synchronized across
+ * sockets and this is not the first CPU in the socket, then reuse
+ * the calibration value of an already online CPU on that socket.
+ *
+ * This assumes that CONSTANT_TSC is consistent for all CPUs in a
+ * socket.
+ */
if (!constant_tsc || !mask)
return 0;
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 03/37] x86/smpboot: Avoid pointless delay calibration is TSC is synchronized
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
When TSC is synchronized across sockets then there is no reason in
calibrating the delay for the first CPU which comes up on a socket.
Just reuse the existing calibration value.
This removes 100ms pointlessly wasted time from CPU hotplug.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/kernel/smpboot.c | 38 ++++++++++++++++++++++++--------------
arch/x86/kernel/tsc.c | 20 ++++++++++++++++----
2 files changed, 40 insertions(+), 18 deletions(-)
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -178,10 +178,7 @@ static void smp_callin(void)
*/
apic_ap_setup();
- /*
- * Save our processor parameters. Note: this information
- * is needed for clock calibration.
- */
+ /* Save our processor parameters. */
smp_store_cpu_info(cpuid);
/*
@@ -192,14 +189,6 @@ static void smp_callin(void)
ap_init_aperfmperf();
- /*
- * Get our bogomips.
- * Update loops_per_jiffy in cpu_data. Previous call to
- * smp_store_cpu_info() stored a value that is close but not as
- * accurate as the value just calculated.
- */
- calibrate_delay();
- cpu_data(cpuid).loops_per_jiffy = loops_per_jiffy;
pr_debug("Stack at about %p\n", &cpuid);
wmb();
@@ -212,8 +201,24 @@ static void smp_callin(void)
cpumask_set_cpu(cpuid, cpu_callin_mask);
}
+static void ap_calibrate_delay(void)
+{
+ /*
+ * Calibrate the delay loop and update loops_per_jiffy in cpu_data.
+ * smp_store_cpu_info() stored a value that is close but not as
+ * accurate as the value just calculated.
+ *
+ * As this is invoked after the TSC synchronization check,
+ * calibrate_delay_is_known() will skip the calibration routine
+ * when TSC is synchronized across sockets.
+ */
+ calibrate_delay();
+ cpu_data(smp_processor_id()).loops_per_jiffy = loops_per_jiffy;
+}
+
static int cpu0_logical_apicid;
static int enable_start_cpu0;
+
/*
* Activate a secondary processor.
*/
@@ -240,10 +245,15 @@ static void notrace start_secondary(void
/* otherwise gcc will move up smp_processor_id before the cpu_init */
barrier();
+ /* Check TSC synchronization with the control CPU: */
+ check_tsc_sync_target();
+
/*
- * Check TSC synchronization with the boot CPU:
+ * Calibrate the delay loop after the TSC synchronization check.
+ * This allows to skip the calibration when TSC is synchronized
+ * across sockets.
*/
- check_tsc_sync_target();
+ ap_calibrate_delay();
speculative_store_bypass_ht_init();
--- a/arch/x86/kernel/tsc.c
+++ b/arch/x86/kernel/tsc.c
@@ -1598,10 +1598,7 @@ void __init tsc_init(void)
#ifdef CONFIG_SMP
/*
- * If we have a constant TSC and are using the TSC for the delay loop,
- * we can skip clock calibration if another cpu in the same socket has already
- * been calibrated. This assumes that CONSTANT_TSC applies to all
- * cpus in the socket - this should be a safe assumption.
+ * Check whether existing calibration data can be reused.
*/
unsigned long calibrate_delay_is_known(void)
{
@@ -1609,6 +1606,21 @@ unsigned long calibrate_delay_is_known(v
int constant_tsc = cpu_has(&cpu_data(cpu), X86_FEATURE_CONSTANT_TSC);
const struct cpumask *mask = topology_core_cpumask(cpu);
+ /*
+ * If TSC has constant frequency and TSC is synchronized across
+ * sockets then reuse CPU0 calibration.
+ */
+ if (constant_tsc && !tsc_unstable)
+ return cpu_data(0).loops_per_jiffy;
+
+ /*
+ * If TSC has constant frequency and TSC is not synchronized across
+ * sockets and this is not the first CPU in the socket, then reuse
+ * the calibration value of an already online CPU on that socket.
+ *
+ * This assumes that CONSTANT_TSC is consistent for all CPUs in a
+ * socket.
+ */
if (!constant_tsc || !mask)
return 0;
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 04/37] x86/smpboot: Rename start_cpu0() to soft_restart_cpu()
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-14 23:44 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
This is used in the SEV play_dead() implementation to re-online CPUs. But
that has nothing to do with CPU0.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
---
arch/x86/include/asm/cpu.h | 2 +-
arch/x86/kernel/callthunks.c | 2 +-
arch/x86/kernel/head_32.S | 10 +++++-----
arch/x86/kernel/head_64.S | 10 +++++-----
arch/x86/kernel/sev.c | 2 +-
5 files changed, 13 insertions(+), 13 deletions(-)
--- a/arch/x86/include/asm/cpu.h
+++ b/arch/x86/include/asm/cpu.h
@@ -30,7 +30,7 @@ struct x86_cpu {
#ifdef CONFIG_HOTPLUG_CPU
extern int arch_register_cpu(int num);
extern void arch_unregister_cpu(int);
-extern void start_cpu0(void);
+extern void soft_restart_cpu(void);
#ifdef CONFIG_DEBUG_HOTPLUG_CPU0
extern int _debug_hotplug_cpu(int cpu, int action);
#endif
--- a/arch/x86/kernel/callthunks.c
+++ b/arch/x86/kernel/callthunks.c
@@ -134,7 +134,7 @@ static bool skip_addr(void *dest)
if (dest == ret_from_fork)
return true;
#ifdef CONFIG_HOTPLUG_CPU
- if (dest == start_cpu0)
+ if (dest == soft_restart_cpu)
return true;
#endif
#ifdef CONFIG_FUNCTION_TRACER
--- a/arch/x86/kernel/head_32.S
+++ b/arch/x86/kernel/head_32.S
@@ -140,16 +140,16 @@ SYM_CODE_END(startup_32)
#ifdef CONFIG_HOTPLUG_CPU
/*
- * Boot CPU0 entry point. It's called from play_dead(). Everything has been set
- * up already except stack. We just set up stack here. Then call
- * start_secondary().
+ * Entry point for soft restart of a CPU. Invoked from xxx_play_dead() for
+ * restarting the boot CPU or for restarting SEV guest CPUs after CPU hot
+ * unplug. Everything is set up already except the stack.
*/
-SYM_FUNC_START(start_cpu0)
+SYM_FUNC_START(soft_restart_cpu)
movl initial_stack, %ecx
movl %ecx, %esp
call *(initial_code)
1: jmp 1b
-SYM_FUNC_END(start_cpu0)
+SYM_FUNC_END(soft_restart_cpu)
#endif
/*
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -377,11 +377,11 @@ SYM_CODE_END(secondary_startup_64)
#ifdef CONFIG_HOTPLUG_CPU
/*
- * Boot CPU0 entry point. It's called from play_dead(). Everything has been set
- * up already except stack. We just set up stack here. Then call
- * start_secondary() via .Ljump_to_C_code.
+ * Entry point for soft restart of a CPU. Invoked from xxx_play_dead() for
+ * restarting the boot CPU or for restarting SEV guest CPUs after CPU hot
+ * unplug. Everything is set up already except the stack.
*/
-SYM_CODE_START(start_cpu0)
+SYM_CODE_START(soft_restart_cpu)
ANNOTATE_NOENDBR
UNWIND_HINT_EMPTY
@@ -390,7 +390,7 @@ SYM_CODE_START(start_cpu0)
movq TASK_threadsp(%rcx), %rsp
jmp .Ljump_to_C_code
-SYM_CODE_END(start_cpu0)
+SYM_CODE_END(soft_restart_cpu)
#endif
#ifdef CONFIG_AMD_MEM_ENCRYPT
--- a/arch/x86/kernel/sev.c
+++ b/arch/x86/kernel/sev.c
@@ -1326,7 +1326,7 @@ static void sev_es_play_dead(void)
* If we get here, the VCPU was woken up again. Jump to CPU
* startup code to get it back online.
*/
- start_cpu0();
+ soft_restart_cpu();
}
#else /* CONFIG_HOTPLUG_CPU */
#define sev_es_play_dead native_play_dead
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 04/37] x86/smpboot: Rename start_cpu0() to soft_restart_cpu()
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
This is used in the SEV play_dead() implementation to re-online CPUs. But
that has nothing to do with CPU0.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
---
arch/x86/include/asm/cpu.h | 2 +-
arch/x86/kernel/callthunks.c | 2 +-
arch/x86/kernel/head_32.S | 10 +++++-----
arch/x86/kernel/head_64.S | 10 +++++-----
arch/x86/kernel/sev.c | 2 +-
5 files changed, 13 insertions(+), 13 deletions(-)
--- a/arch/x86/include/asm/cpu.h
+++ b/arch/x86/include/asm/cpu.h
@@ -30,7 +30,7 @@ struct x86_cpu {
#ifdef CONFIG_HOTPLUG_CPU
extern int arch_register_cpu(int num);
extern void arch_unregister_cpu(int);
-extern void start_cpu0(void);
+extern void soft_restart_cpu(void);
#ifdef CONFIG_DEBUG_HOTPLUG_CPU0
extern int _debug_hotplug_cpu(int cpu, int action);
#endif
--- a/arch/x86/kernel/callthunks.c
+++ b/arch/x86/kernel/callthunks.c
@@ -134,7 +134,7 @@ static bool skip_addr(void *dest)
if (dest == ret_from_fork)
return true;
#ifdef CONFIG_HOTPLUG_CPU
- if (dest == start_cpu0)
+ if (dest == soft_restart_cpu)
return true;
#endif
#ifdef CONFIG_FUNCTION_TRACER
--- a/arch/x86/kernel/head_32.S
+++ b/arch/x86/kernel/head_32.S
@@ -140,16 +140,16 @@ SYM_CODE_END(startup_32)
#ifdef CONFIG_HOTPLUG_CPU
/*
- * Boot CPU0 entry point. It's called from play_dead(). Everything has been set
- * up already except stack. We just set up stack here. Then call
- * start_secondary().
+ * Entry point for soft restart of a CPU. Invoked from xxx_play_dead() for
+ * restarting the boot CPU or for restarting SEV guest CPUs after CPU hot
+ * unplug. Everything is set up already except the stack.
*/
-SYM_FUNC_START(start_cpu0)
+SYM_FUNC_START(soft_restart_cpu)
movl initial_stack, %ecx
movl %ecx, %esp
call *(initial_code)
1: jmp 1b
-SYM_FUNC_END(start_cpu0)
+SYM_FUNC_END(soft_restart_cpu)
#endif
/*
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -377,11 +377,11 @@ SYM_CODE_END(secondary_startup_64)
#ifdef CONFIG_HOTPLUG_CPU
/*
- * Boot CPU0 entry point. It's called from play_dead(). Everything has been set
- * up already except stack. We just set up stack here. Then call
- * start_secondary() via .Ljump_to_C_code.
+ * Entry point for soft restart of a CPU. Invoked from xxx_play_dead() for
+ * restarting the boot CPU or for restarting SEV guest CPUs after CPU hot
+ * unplug. Everything is set up already except the stack.
*/
-SYM_CODE_START(start_cpu0)
+SYM_CODE_START(soft_restart_cpu)
ANNOTATE_NOENDBR
UNWIND_HINT_EMPTY
@@ -390,7 +390,7 @@ SYM_CODE_START(start_cpu0)
movq TASK_threadsp(%rcx), %rsp
jmp .Ljump_to_C_code
-SYM_CODE_END(start_cpu0)
+SYM_CODE_END(soft_restart_cpu)
#endif
#ifdef CONFIG_AMD_MEM_ENCRYPT
--- a/arch/x86/kernel/sev.c
+++ b/arch/x86/kernel/sev.c
@@ -1326,7 +1326,7 @@ static void sev_es_play_dead(void)
* If we get here, the VCPU was woken up again. Jump to CPU
* startup code to get it back online.
*/
- start_cpu0();
+ soft_restart_cpu();
}
#else /* CONFIG_HOTPLUG_CPU */
#define sev_es_play_dead native_play_dead
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 04/37] x86/smpboot: Rename start_cpu0() to soft_restart_cpu()
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
This is used in the SEV play_dead() implementation to re-online CPUs. But
that has nothing to do with CPU0.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
---
arch/x86/include/asm/cpu.h | 2 +-
arch/x86/kernel/callthunks.c | 2 +-
arch/x86/kernel/head_32.S | 10 +++++-----
arch/x86/kernel/head_64.S | 10 +++++-----
arch/x86/kernel/sev.c | 2 +-
5 files changed, 13 insertions(+), 13 deletions(-)
--- a/arch/x86/include/asm/cpu.h
+++ b/arch/x86/include/asm/cpu.h
@@ -30,7 +30,7 @@ struct x86_cpu {
#ifdef CONFIG_HOTPLUG_CPU
extern int arch_register_cpu(int num);
extern void arch_unregister_cpu(int);
-extern void start_cpu0(void);
+extern void soft_restart_cpu(void);
#ifdef CONFIG_DEBUG_HOTPLUG_CPU0
extern int _debug_hotplug_cpu(int cpu, int action);
#endif
--- a/arch/x86/kernel/callthunks.c
+++ b/arch/x86/kernel/callthunks.c
@@ -134,7 +134,7 @@ static bool skip_addr(void *dest)
if (dest == ret_from_fork)
return true;
#ifdef CONFIG_HOTPLUG_CPU
- if (dest == start_cpu0)
+ if (dest == soft_restart_cpu)
return true;
#endif
#ifdef CONFIG_FUNCTION_TRACER
--- a/arch/x86/kernel/head_32.S
+++ b/arch/x86/kernel/head_32.S
@@ -140,16 +140,16 @@ SYM_CODE_END(startup_32)
#ifdef CONFIG_HOTPLUG_CPU
/*
- * Boot CPU0 entry point. It's called from play_dead(). Everything has been set
- * up already except stack. We just set up stack here. Then call
- * start_secondary().
+ * Entry point for soft restart of a CPU. Invoked from xxx_play_dead() for
+ * restarting the boot CPU or for restarting SEV guest CPUs after CPU hot
+ * unplug. Everything is set up already except the stack.
*/
-SYM_FUNC_START(start_cpu0)
+SYM_FUNC_START(soft_restart_cpu)
movl initial_stack, %ecx
movl %ecx, %esp
call *(initial_code)
1: jmp 1b
-SYM_FUNC_END(start_cpu0)
+SYM_FUNC_END(soft_restart_cpu)
#endif
/*
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -377,11 +377,11 @@ SYM_CODE_END(secondary_startup_64)
#ifdef CONFIG_HOTPLUG_CPU
/*
- * Boot CPU0 entry point. It's called from play_dead(). Everything has been set
- * up already except stack. We just set up stack here. Then call
- * start_secondary() via .Ljump_to_C_code.
+ * Entry point for soft restart of a CPU. Invoked from xxx_play_dead() for
+ * restarting the boot CPU or for restarting SEV guest CPUs after CPU hot
+ * unplug. Everything is set up already except the stack.
*/
-SYM_CODE_START(start_cpu0)
+SYM_CODE_START(soft_restart_cpu)
ANNOTATE_NOENDBR
UNWIND_HINT_EMPTY
@@ -390,7 +390,7 @@ SYM_CODE_START(start_cpu0)
movq TASK_threadsp(%rcx), %rsp
jmp .Ljump_to_C_code
-SYM_CODE_END(start_cpu0)
+SYM_CODE_END(soft_restart_cpu)
#endif
#ifdef CONFIG_AMD_MEM_ENCRYPT
--- a/arch/x86/kernel/sev.c
+++ b/arch/x86/kernel/sev.c
@@ -1326,7 +1326,7 @@ static void sev_es_play_dead(void)
* If we get here, the VCPU was woken up again. Jump to CPU
* startup code to get it back online.
*/
- start_cpu0();
+ soft_restart_cpu();
}
#else /* CONFIG_HOTPLUG_CPU */
#define sev_es_play_dead native_play_dead
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 05/37] x86/topology: Remove CPU0 hotplug option
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-14 23:44 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
This was introduced together with commit e1c467e69040 ("x86, hotplug: Wake
up CPU0 via NMI instead of INIT, SIPI, SIPI") to eventually support
physical hotplug of CPU0:
"We'll change this code in the future to wake up hard offlined CPU0 if
real platform and request are available."
11 years later this has not happened and physical hotplug is not officially
supported. Remove the cruft.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
Documentation/admin-guide/kernel-parameters.txt | 14 ---
Documentation/core-api/cpu_hotplug.rst | 13 ---
arch/x86/Kconfig | 43 ----------
arch/x86/include/asm/cpu.h | 3
arch/x86/kernel/topology.c | 98 ------------------------
arch/x86/power/cpu.c | 37 ---------
6 files changed, 6 insertions(+), 202 deletions(-)
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -795,20 +795,6 @@
Format:
<first_slot>,<last_slot>,<port>,<enum_bit>[,<debug>]
- cpu0_hotplug [X86] Turn on CPU0 hotplug feature when
- CONFIG_BOOTPARAM_HOTPLUG_CPU0 is off.
- Some features depend on CPU0. Known dependencies are:
- 1. Resume from suspend/hibernate depends on CPU0.
- Suspend/hibernate will fail if CPU0 is offline and you
- need to online CPU0 before suspend/hibernate.
- 2. PIC interrupts also depend on CPU0. CPU0 can't be
- removed if a PIC interrupt is detected.
- It's said poweroff/reboot may depend on CPU0 on some
- machines although I haven't seen such issues so far
- after CPU0 is offline on a few tested machines.
- If the dependencies are under your control, you can
- turn on cpu0_hotplug.
-
cpuidle.off=1 [CPU_IDLE]
disable the cpuidle sub-system
--- a/Documentation/core-api/cpu_hotplug.rst
+++ b/Documentation/core-api/cpu_hotplug.rst
@@ -127,17 +127,8 @@ Once the CPU is shutdown, it will be rem
$ echo 1 > /sys/devices/system/cpu/cpu4/online
smpboot: Booting Node 0 Processor 4 APIC 0x1
-The CPU is usable again. This should work on all CPUs. CPU0 is often special
-and excluded from CPU hotplug. On X86 the kernel option
-*CONFIG_BOOTPARAM_HOTPLUG_CPU0* has to be enabled in order to be able to
-shutdown CPU0. Alternatively the kernel command option *cpu0_hotplug* can be
-used. Some known dependencies of CPU0:
-
-* Resume from hibernate/suspend. Hibernate/suspend will fail if CPU0 is offline.
-* PIC interrupts. CPU0 can't be removed if a PIC interrupt is detected.
-
-Please let Fenghua Yu <fenghua.yu@intel.com> know if you find any dependencies
-on CPU0.
+The CPU is usable again. This should work on all CPUs, but CPU0 is often special
+and excluded from CPU hotplug.
The CPU hotplug coordination
============================
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -2294,49 +2294,6 @@ config HOTPLUG_CPU
def_bool y
depends on SMP
-config BOOTPARAM_HOTPLUG_CPU0
- bool "Set default setting of cpu0_hotpluggable"
- depends on HOTPLUG_CPU
- help
- Set whether default state of cpu0_hotpluggable is on or off.
-
- Say Y here to enable CPU0 hotplug by default. If this switch
- is turned on, there is no need to give cpu0_hotplug kernel
- parameter and the CPU0 hotplug feature is enabled by default.
-
- Please note: there are two known CPU0 dependencies if you want
- to enable the CPU0 hotplug feature either by this switch or by
- cpu0_hotplug kernel parameter.
-
- First, resume from hibernate or suspend always starts from CPU0.
- So hibernate and suspend are prevented if CPU0 is offline.
-
- Second dependency is PIC interrupts always go to CPU0. CPU0 can not
- offline if any interrupt can not migrate out of CPU0. There may
- be other CPU0 dependencies.
-
- Please make sure the dependencies are under your control before
- you enable this feature.
-
- Say N if you don't want to enable CPU0 hotplug feature by default.
- You still can enable the CPU0 hotplug feature at boot by kernel
- parameter cpu0_hotplug.
-
-config DEBUG_HOTPLUG_CPU0
- def_bool n
- prompt "Debug CPU0 hotplug"
- depends on HOTPLUG_CPU
- help
- Enabling this option offlines CPU0 (if CPU0 can be offlined) as
- soon as possible and boots up userspace with CPU0 offlined. User
- can online CPU0 back after boot time.
-
- To debug CPU0 hotplug, you need to enable CPU0 offline/online
- feature by either turning on CONFIG_BOOTPARAM_HOTPLUG_CPU0 during
- compilation or giving cpu0_hotplug kernel parameter at boot.
-
- If unsure, say N.
-
config COMPAT_VDSO
def_bool n
prompt "Disable the 32-bit vDSO (needed for glibc 2.3.3)"
--- a/arch/x86/include/asm/cpu.h
+++ b/arch/x86/include/asm/cpu.h
@@ -31,9 +31,6 @@ struct x86_cpu {
extern int arch_register_cpu(int num);
extern void arch_unregister_cpu(int);
extern void soft_restart_cpu(void);
-#ifdef CONFIG_DEBUG_HOTPLUG_CPU0
-extern int _debug_hotplug_cpu(int cpu, int action);
-#endif
#endif
extern void ap_init_aperfmperf(void);
--- a/arch/x86/kernel/topology.c
+++ b/arch/x86/kernel/topology.c
@@ -38,102 +38,12 @@
static DEFINE_PER_CPU(struct x86_cpu, cpu_devices);
#ifdef CONFIG_HOTPLUG_CPU
-
-#ifdef CONFIG_BOOTPARAM_HOTPLUG_CPU0
-static int cpu0_hotpluggable = 1;
-#else
-static int cpu0_hotpluggable;
-static int __init enable_cpu0_hotplug(char *str)
-{
- cpu0_hotpluggable = 1;
- return 1;
-}
-
-__setup("cpu0_hotplug", enable_cpu0_hotplug);
-#endif
-
-#ifdef CONFIG_DEBUG_HOTPLUG_CPU0
-/*
- * This function offlines a CPU as early as possible and allows userspace to
- * boot up without the CPU. The CPU can be onlined back by user after boot.
- *
- * This is only called for debugging CPU offline/online feature.
- */
-int _debug_hotplug_cpu(int cpu, int action)
-{
- int ret;
-
- if (!cpu_is_hotpluggable(cpu))
- return -EINVAL;
-
- switch (action) {
- case 0:
- ret = remove_cpu(cpu);
- if (!ret)
- pr_info("DEBUG_HOTPLUG_CPU0: CPU %u is now offline\n", cpu);
- else
- pr_debug("Can't offline CPU%d.\n", cpu);
- break;
- case 1:
- ret = add_cpu(cpu);
- if (ret)
- pr_debug("Can't online CPU%d.\n", cpu);
-
- break;
- default:
- ret = -EINVAL;
- }
-
- return ret;
-}
-
-static int __init debug_hotplug_cpu(void)
+int arch_register_cpu(int cpu)
{
- _debug_hotplug_cpu(0, 0);
- return 0;
-}
-
-late_initcall_sync(debug_hotplug_cpu);
-#endif /* CONFIG_DEBUG_HOTPLUG_CPU0 */
-
-int arch_register_cpu(int num)
-{
- struct cpuinfo_x86 *c = &cpu_data(num);
-
- /*
- * Currently CPU0 is only hotpluggable on Intel platforms. Other
- * vendors can add hotplug support later.
- * Xen PV guests don't support CPU0 hotplug at all.
- */
- if (c->x86_vendor != X86_VENDOR_INTEL ||
- cpu_feature_enabled(X86_FEATURE_XENPV))
- cpu0_hotpluggable = 0;
-
- /*
- * Two known BSP/CPU0 dependencies: Resume from suspend/hibernate
- * depends on BSP. PIC interrupts depend on BSP.
- *
- * If the BSP dependencies are under control, one can tell kernel to
- * enable BSP hotplug. This basically adds a control file and
- * one can attempt to offline BSP.
- */
- if (num == 0 && cpu0_hotpluggable) {
- unsigned int irq;
- /*
- * We won't take down the boot processor on i386 if some
- * interrupts only are able to be serviced by the BSP in PIC.
- */
- for_each_active_irq(irq) {
- if (!IO_APIC_IRQ(irq) && irq_has_action(irq)) {
- cpu0_hotpluggable = 0;
- break;
- }
- }
- }
- if (num || cpu0_hotpluggable)
- per_cpu(cpu_devices, num).cpu.hotpluggable = 1;
+ struct x86_cpu *xc = per_cpu_ptr(&cpu_devices, cpu);
- return register_cpu(&per_cpu(cpu_devices, num).cpu, num);
+ xc->cpu.hotpluggable = cpu > 0;
+ return register_cpu(&xc->cpu, cpu);
}
EXPORT_SYMBOL(arch_register_cpu);
--- a/arch/x86/power/cpu.c
+++ b/arch/x86/power/cpu.c
@@ -351,43 +351,6 @@ static int bsp_pm_callback(struct notifi
case PM_HIBERNATION_PREPARE:
ret = bsp_check();
break;
-#ifdef CONFIG_DEBUG_HOTPLUG_CPU0
- case PM_RESTORE_PREPARE:
- /*
- * When system resumes from hibernation, online CPU0 because
- * 1. it's required for resume and
- * 2. the CPU was online before hibernation
- */
- if (!cpu_online(0))
- _debug_hotplug_cpu(0, 1);
- break;
- case PM_POST_RESTORE:
- /*
- * When a resume really happens, this code won't be called.
- *
- * This code is called only when user space hibernation software
- * prepares for snapshot device during boot time. So we just
- * call _debug_hotplug_cpu() to restore to CPU0's state prior to
- * preparing the snapshot device.
- *
- * This works for normal boot case in our CPU0 hotplug debug
- * mode, i.e. CPU0 is offline and user mode hibernation
- * software initializes during boot time.
- *
- * If CPU0 is online and user application accesses snapshot
- * device after boot time, this will offline CPU0 and user may
- * see different CPU0 state before and after accessing
- * the snapshot device. But hopefully this is not a case when
- * user debugging CPU0 hotplug. Even if users hit this case,
- * they can easily online CPU0 back.
- *
- * To simplify this debug code, we only consider normal boot
- * case. Otherwise we need to remember CPU0's state and restore
- * to that state and resolve racy conditions etc.
- */
- _debug_hotplug_cpu(0, 0);
- break;
-#endif
default:
break;
}
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 05/37] x86/topology: Remove CPU0 hotplug option
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
This was introduced together with commit e1c467e69040 ("x86, hotplug: Wake
up CPU0 via NMI instead of INIT, SIPI, SIPI") to eventually support
physical hotplug of CPU0:
"We'll change this code in the future to wake up hard offlined CPU0 if
real platform and request are available."
11 years later this has not happened and physical hotplug is not officially
supported. Remove the cruft.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
Documentation/admin-guide/kernel-parameters.txt | 14 ---
Documentation/core-api/cpu_hotplug.rst | 13 ---
arch/x86/Kconfig | 43 ----------
arch/x86/include/asm/cpu.h | 3
arch/x86/kernel/topology.c | 98 ------------------------
arch/x86/power/cpu.c | 37 ---------
6 files changed, 6 insertions(+), 202 deletions(-)
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -795,20 +795,6 @@
Format:
<first_slot>,<last_slot>,<port>,<enum_bit>[,<debug>]
- cpu0_hotplug [X86] Turn on CPU0 hotplug feature when
- CONFIG_BOOTPARAM_HOTPLUG_CPU0 is off.
- Some features depend on CPU0. Known dependencies are:
- 1. Resume from suspend/hibernate depends on CPU0.
- Suspend/hibernate will fail if CPU0 is offline and you
- need to online CPU0 before suspend/hibernate.
- 2. PIC interrupts also depend on CPU0. CPU0 can't be
- removed if a PIC interrupt is detected.
- It's said poweroff/reboot may depend on CPU0 on some
- machines although I haven't seen such issues so far
- after CPU0 is offline on a few tested machines.
- If the dependencies are under your control, you can
- turn on cpu0_hotplug.
-
cpuidle.off=1 [CPU_IDLE]
disable the cpuidle sub-system
--- a/Documentation/core-api/cpu_hotplug.rst
+++ b/Documentation/core-api/cpu_hotplug.rst
@@ -127,17 +127,8 @@ Once the CPU is shutdown, it will be rem
$ echo 1 > /sys/devices/system/cpu/cpu4/online
smpboot: Booting Node 0 Processor 4 APIC 0x1
-The CPU is usable again. This should work on all CPUs. CPU0 is often special
-and excluded from CPU hotplug. On X86 the kernel option
-*CONFIG_BOOTPARAM_HOTPLUG_CPU0* has to be enabled in order to be able to
-shutdown CPU0. Alternatively the kernel command option *cpu0_hotplug* can be
-used. Some known dependencies of CPU0:
-
-* Resume from hibernate/suspend. Hibernate/suspend will fail if CPU0 is offline.
-* PIC interrupts. CPU0 can't be removed if a PIC interrupt is detected.
-
-Please let Fenghua Yu <fenghua.yu@intel.com> know if you find any dependencies
-on CPU0.
+The CPU is usable again. This should work on all CPUs, but CPU0 is often special
+and excluded from CPU hotplug.
The CPU hotplug coordination
============================
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -2294,49 +2294,6 @@ config HOTPLUG_CPU
def_bool y
depends on SMP
-config BOOTPARAM_HOTPLUG_CPU0
- bool "Set default setting of cpu0_hotpluggable"
- depends on HOTPLUG_CPU
- help
- Set whether default state of cpu0_hotpluggable is on or off.
-
- Say Y here to enable CPU0 hotplug by default. If this switch
- is turned on, there is no need to give cpu0_hotplug kernel
- parameter and the CPU0 hotplug feature is enabled by default.
-
- Please note: there are two known CPU0 dependencies if you want
- to enable the CPU0 hotplug feature either by this switch or by
- cpu0_hotplug kernel parameter.
-
- First, resume from hibernate or suspend always starts from CPU0.
- So hibernate and suspend are prevented if CPU0 is offline.
-
- Second dependency is PIC interrupts always go to CPU0. CPU0 can not
- offline if any interrupt can not migrate out of CPU0. There may
- be other CPU0 dependencies.
-
- Please make sure the dependencies are under your control before
- you enable this feature.
-
- Say N if you don't want to enable CPU0 hotplug feature by default.
- You still can enable the CPU0 hotplug feature at boot by kernel
- parameter cpu0_hotplug.
-
-config DEBUG_HOTPLUG_CPU0
- def_bool n
- prompt "Debug CPU0 hotplug"
- depends on HOTPLUG_CPU
- help
- Enabling this option offlines CPU0 (if CPU0 can be offlined) as
- soon as possible and boots up userspace with CPU0 offlined. User
- can online CPU0 back after boot time.
-
- To debug CPU0 hotplug, you need to enable CPU0 offline/online
- feature by either turning on CONFIG_BOOTPARAM_HOTPLUG_CPU0 during
- compilation or giving cpu0_hotplug kernel parameter at boot.
-
- If unsure, say N.
-
config COMPAT_VDSO
def_bool n
prompt "Disable the 32-bit vDSO (needed for glibc 2.3.3)"
--- a/arch/x86/include/asm/cpu.h
+++ b/arch/x86/include/asm/cpu.h
@@ -31,9 +31,6 @@ struct x86_cpu {
extern int arch_register_cpu(int num);
extern void arch_unregister_cpu(int);
extern void soft_restart_cpu(void);
-#ifdef CONFIG_DEBUG_HOTPLUG_CPU0
-extern int _debug_hotplug_cpu(int cpu, int action);
-#endif
#endif
extern void ap_init_aperfmperf(void);
--- a/arch/x86/kernel/topology.c
+++ b/arch/x86/kernel/topology.c
@@ -38,102 +38,12 @@
static DEFINE_PER_CPU(struct x86_cpu, cpu_devices);
#ifdef CONFIG_HOTPLUG_CPU
-
-#ifdef CONFIG_BOOTPARAM_HOTPLUG_CPU0
-static int cpu0_hotpluggable = 1;
-#else
-static int cpu0_hotpluggable;
-static int __init enable_cpu0_hotplug(char *str)
-{
- cpu0_hotpluggable = 1;
- return 1;
-}
-
-__setup("cpu0_hotplug", enable_cpu0_hotplug);
-#endif
-
-#ifdef CONFIG_DEBUG_HOTPLUG_CPU0
-/*
- * This function offlines a CPU as early as possible and allows userspace to
- * boot up without the CPU. The CPU can be onlined back by user after boot.
- *
- * This is only called for debugging CPU offline/online feature.
- */
-int _debug_hotplug_cpu(int cpu, int action)
-{
- int ret;
-
- if (!cpu_is_hotpluggable(cpu))
- return -EINVAL;
-
- switch (action) {
- case 0:
- ret = remove_cpu(cpu);
- if (!ret)
- pr_info("DEBUG_HOTPLUG_CPU0: CPU %u is now offline\n", cpu);
- else
- pr_debug("Can't offline CPU%d.\n", cpu);
- break;
- case 1:
- ret = add_cpu(cpu);
- if (ret)
- pr_debug("Can't online CPU%d.\n", cpu);
-
- break;
- default:
- ret = -EINVAL;
- }
-
- return ret;
-}
-
-static int __init debug_hotplug_cpu(void)
+int arch_register_cpu(int cpu)
{
- _debug_hotplug_cpu(0, 0);
- return 0;
-}
-
-late_initcall_sync(debug_hotplug_cpu);
-#endif /* CONFIG_DEBUG_HOTPLUG_CPU0 */
-
-int arch_register_cpu(int num)
-{
- struct cpuinfo_x86 *c = &cpu_data(num);
-
- /*
- * Currently CPU0 is only hotpluggable on Intel platforms. Other
- * vendors can add hotplug support later.
- * Xen PV guests don't support CPU0 hotplug at all.
- */
- if (c->x86_vendor != X86_VENDOR_INTEL ||
- cpu_feature_enabled(X86_FEATURE_XENPV))
- cpu0_hotpluggable = 0;
-
- /*
- * Two known BSP/CPU0 dependencies: Resume from suspend/hibernate
- * depends on BSP. PIC interrupts depend on BSP.
- *
- * If the BSP dependencies are under control, one can tell kernel to
- * enable BSP hotplug. This basically adds a control file and
- * one can attempt to offline BSP.
- */
- if (num == 0 && cpu0_hotpluggable) {
- unsigned int irq;
- /*
- * We won't take down the boot processor on i386 if some
- * interrupts only are able to be serviced by the BSP in PIC.
- */
- for_each_active_irq(irq) {
- if (!IO_APIC_IRQ(irq) && irq_has_action(irq)) {
- cpu0_hotpluggable = 0;
- break;
- }
- }
- }
- if (num || cpu0_hotpluggable)
- per_cpu(cpu_devices, num).cpu.hotpluggable = 1;
+ struct x86_cpu *xc = per_cpu_ptr(&cpu_devices, cpu);
- return register_cpu(&per_cpu(cpu_devices, num).cpu, num);
+ xc->cpu.hotpluggable = cpu > 0;
+ return register_cpu(&xc->cpu, cpu);
}
EXPORT_SYMBOL(arch_register_cpu);
--- a/arch/x86/power/cpu.c
+++ b/arch/x86/power/cpu.c
@@ -351,43 +351,6 @@ static int bsp_pm_callback(struct notifi
case PM_HIBERNATION_PREPARE:
ret = bsp_check();
break;
-#ifdef CONFIG_DEBUG_HOTPLUG_CPU0
- case PM_RESTORE_PREPARE:
- /*
- * When system resumes from hibernation, online CPU0 because
- * 1. it's required for resume and
- * 2. the CPU was online before hibernation
- */
- if (!cpu_online(0))
- _debug_hotplug_cpu(0, 1);
- break;
- case PM_POST_RESTORE:
- /*
- * When a resume really happens, this code won't be called.
- *
- * This code is called only when user space hibernation software
- * prepares for snapshot device during boot time. So we just
- * call _debug_hotplug_cpu() to restore to CPU0's state prior to
- * preparing the snapshot device.
- *
- * This works for normal boot case in our CPU0 hotplug debug
- * mode, i.e. CPU0 is offline and user mode hibernation
- * software initializes during boot time.
- *
- * If CPU0 is online and user application accesses snapshot
- * device after boot time, this will offline CPU0 and user may
- * see different CPU0 state before and after accessing
- * the snapshot device. But hopefully this is not a case when
- * user debugging CPU0 hotplug. Even if users hit this case,
- * they can easily online CPU0 back.
- *
- * To simplify this debug code, we only consider normal boot
- * case. Otherwise we need to remember CPU0's state and restore
- * to that state and resolve racy conditions etc.
- */
- _debug_hotplug_cpu(0, 0);
- break;
-#endif
default:
break;
}
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 05/37] x86/topology: Remove CPU0 hotplug option
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
This was introduced together with commit e1c467e69040 ("x86, hotplug: Wake
up CPU0 via NMI instead of INIT, SIPI, SIPI") to eventually support
physical hotplug of CPU0:
"We'll change this code in the future to wake up hard offlined CPU0 if
real platform and request are available."
11 years later this has not happened and physical hotplug is not officially
supported. Remove the cruft.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
Documentation/admin-guide/kernel-parameters.txt | 14 ---
Documentation/core-api/cpu_hotplug.rst | 13 ---
arch/x86/Kconfig | 43 ----------
arch/x86/include/asm/cpu.h | 3
arch/x86/kernel/topology.c | 98 ------------------------
arch/x86/power/cpu.c | 37 ---------
6 files changed, 6 insertions(+), 202 deletions(-)
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -795,20 +795,6 @@
Format:
<first_slot>,<last_slot>,<port>,<enum_bit>[,<debug>]
- cpu0_hotplug [X86] Turn on CPU0 hotplug feature when
- CONFIG_BOOTPARAM_HOTPLUG_CPU0 is off.
- Some features depend on CPU0. Known dependencies are:
- 1. Resume from suspend/hibernate depends on CPU0.
- Suspend/hibernate will fail if CPU0 is offline and you
- need to online CPU0 before suspend/hibernate.
- 2. PIC interrupts also depend on CPU0. CPU0 can't be
- removed if a PIC interrupt is detected.
- It's said poweroff/reboot may depend on CPU0 on some
- machines although I haven't seen such issues so far
- after CPU0 is offline on a few tested machines.
- If the dependencies are under your control, you can
- turn on cpu0_hotplug.
-
cpuidle.off=1 [CPU_IDLE]
disable the cpuidle sub-system
--- a/Documentation/core-api/cpu_hotplug.rst
+++ b/Documentation/core-api/cpu_hotplug.rst
@@ -127,17 +127,8 @@ Once the CPU is shutdown, it will be rem
$ echo 1 > /sys/devices/system/cpu/cpu4/online
smpboot: Booting Node 0 Processor 4 APIC 0x1
-The CPU is usable again. This should work on all CPUs. CPU0 is often special
-and excluded from CPU hotplug. On X86 the kernel option
-*CONFIG_BOOTPARAM_HOTPLUG_CPU0* has to be enabled in order to be able to
-shutdown CPU0. Alternatively the kernel command option *cpu0_hotplug* can be
-used. Some known dependencies of CPU0:
-
-* Resume from hibernate/suspend. Hibernate/suspend will fail if CPU0 is offline.
-* PIC interrupts. CPU0 can't be removed if a PIC interrupt is detected.
-
-Please let Fenghua Yu <fenghua.yu@intel.com> know if you find any dependencies
-on CPU0.
+The CPU is usable again. This should work on all CPUs, but CPU0 is often special
+and excluded from CPU hotplug.
The CPU hotplug coordination
============================
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -2294,49 +2294,6 @@ config HOTPLUG_CPU
def_bool y
depends on SMP
-config BOOTPARAM_HOTPLUG_CPU0
- bool "Set default setting of cpu0_hotpluggable"
- depends on HOTPLUG_CPU
- help
- Set whether default state of cpu0_hotpluggable is on or off.
-
- Say Y here to enable CPU0 hotplug by default. If this switch
- is turned on, there is no need to give cpu0_hotplug kernel
- parameter and the CPU0 hotplug feature is enabled by default.
-
- Please note: there are two known CPU0 dependencies if you want
- to enable the CPU0 hotplug feature either by this switch or by
- cpu0_hotplug kernel parameter.
-
- First, resume from hibernate or suspend always starts from CPU0.
- So hibernate and suspend are prevented if CPU0 is offline.
-
- Second dependency is PIC interrupts always go to CPU0. CPU0 can not
- offline if any interrupt can not migrate out of CPU0. There may
- be other CPU0 dependencies.
-
- Please make sure the dependencies are under your control before
- you enable this feature.
-
- Say N if you don't want to enable CPU0 hotplug feature by default.
- You still can enable the CPU0 hotplug feature at boot by kernel
- parameter cpu0_hotplug.
-
-config DEBUG_HOTPLUG_CPU0
- def_bool n
- prompt "Debug CPU0 hotplug"
- depends on HOTPLUG_CPU
- help
- Enabling this option offlines CPU0 (if CPU0 can be offlined) as
- soon as possible and boots up userspace with CPU0 offlined. User
- can online CPU0 back after boot time.
-
- To debug CPU0 hotplug, you need to enable CPU0 offline/online
- feature by either turning on CONFIG_BOOTPARAM_HOTPLUG_CPU0 during
- compilation or giving cpu0_hotplug kernel parameter at boot.
-
- If unsure, say N.
-
config COMPAT_VDSO
def_bool n
prompt "Disable the 32-bit vDSO (needed for glibc 2.3.3)"
--- a/arch/x86/include/asm/cpu.h
+++ b/arch/x86/include/asm/cpu.h
@@ -31,9 +31,6 @@ struct x86_cpu {
extern int arch_register_cpu(int num);
extern void arch_unregister_cpu(int);
extern void soft_restart_cpu(void);
-#ifdef CONFIG_DEBUG_HOTPLUG_CPU0
-extern int _debug_hotplug_cpu(int cpu, int action);
-#endif
#endif
extern void ap_init_aperfmperf(void);
--- a/arch/x86/kernel/topology.c
+++ b/arch/x86/kernel/topology.c
@@ -38,102 +38,12 @@
static DEFINE_PER_CPU(struct x86_cpu, cpu_devices);
#ifdef CONFIG_HOTPLUG_CPU
-
-#ifdef CONFIG_BOOTPARAM_HOTPLUG_CPU0
-static int cpu0_hotpluggable = 1;
-#else
-static int cpu0_hotpluggable;
-static int __init enable_cpu0_hotplug(char *str)
-{
- cpu0_hotpluggable = 1;
- return 1;
-}
-
-__setup("cpu0_hotplug", enable_cpu0_hotplug);
-#endif
-
-#ifdef CONFIG_DEBUG_HOTPLUG_CPU0
-/*
- * This function offlines a CPU as early as possible and allows userspace to
- * boot up without the CPU. The CPU can be onlined back by user after boot.
- *
- * This is only called for debugging CPU offline/online feature.
- */
-int _debug_hotplug_cpu(int cpu, int action)
-{
- int ret;
-
- if (!cpu_is_hotpluggable(cpu))
- return -EINVAL;
-
- switch (action) {
- case 0:
- ret = remove_cpu(cpu);
- if (!ret)
- pr_info("DEBUG_HOTPLUG_CPU0: CPU %u is now offline\n", cpu);
- else
- pr_debug("Can't offline CPU%d.\n", cpu);
- break;
- case 1:
- ret = add_cpu(cpu);
- if (ret)
- pr_debug("Can't online CPU%d.\n", cpu);
-
- break;
- default:
- ret = -EINVAL;
- }
-
- return ret;
-}
-
-static int __init debug_hotplug_cpu(void)
+int arch_register_cpu(int cpu)
{
- _debug_hotplug_cpu(0, 0);
- return 0;
-}
-
-late_initcall_sync(debug_hotplug_cpu);
-#endif /* CONFIG_DEBUG_HOTPLUG_CPU0 */
-
-int arch_register_cpu(int num)
-{
- struct cpuinfo_x86 *c = &cpu_data(num);
-
- /*
- * Currently CPU0 is only hotpluggable on Intel platforms. Other
- * vendors can add hotplug support later.
- * Xen PV guests don't support CPU0 hotplug at all.
- */
- if (c->x86_vendor != X86_VENDOR_INTEL ||
- cpu_feature_enabled(X86_FEATURE_XENPV))
- cpu0_hotpluggable = 0;
-
- /*
- * Two known BSP/CPU0 dependencies: Resume from suspend/hibernate
- * depends on BSP. PIC interrupts depend on BSP.
- *
- * If the BSP dependencies are under control, one can tell kernel to
- * enable BSP hotplug. This basically adds a control file and
- * one can attempt to offline BSP.
- */
- if (num == 0 && cpu0_hotpluggable) {
- unsigned int irq;
- /*
- * We won't take down the boot processor on i386 if some
- * interrupts only are able to be serviced by the BSP in PIC.
- */
- for_each_active_irq(irq) {
- if (!IO_APIC_IRQ(irq) && irq_has_action(irq)) {
- cpu0_hotpluggable = 0;
- break;
- }
- }
- }
- if (num || cpu0_hotpluggable)
- per_cpu(cpu_devices, num).cpu.hotpluggable = 1;
+ struct x86_cpu *xc = per_cpu_ptr(&cpu_devices, cpu);
- return register_cpu(&per_cpu(cpu_devices, num).cpu, num);
+ xc->cpu.hotpluggable = cpu > 0;
+ return register_cpu(&xc->cpu, cpu);
}
EXPORT_SYMBOL(arch_register_cpu);
--- a/arch/x86/power/cpu.c
+++ b/arch/x86/power/cpu.c
@@ -351,43 +351,6 @@ static int bsp_pm_callback(struct notifi
case PM_HIBERNATION_PREPARE:
ret = bsp_check();
break;
-#ifdef CONFIG_DEBUG_HOTPLUG_CPU0
- case PM_RESTORE_PREPARE:
- /*
- * When system resumes from hibernation, online CPU0 because
- * 1. it's required for resume and
- * 2. the CPU was online before hibernation
- */
- if (!cpu_online(0))
- _debug_hotplug_cpu(0, 1);
- break;
- case PM_POST_RESTORE:
- /*
- * When a resume really happens, this code won't be called.
- *
- * This code is called only when user space hibernation software
- * prepares for snapshot device during boot time. So we just
- * call _debug_hotplug_cpu() to restore to CPU0's state prior to
- * preparing the snapshot device.
- *
- * This works for normal boot case in our CPU0 hotplug debug
- * mode, i.e. CPU0 is offline and user mode hibernation
- * software initializes during boot time.
- *
- * If CPU0 is online and user application accesses snapshot
- * device after boot time, this will offline CPU0 and user may
- * see different CPU0 state before and after accessing
- * the snapshot device. But hopefully this is not a case when
- * user debugging CPU0 hotplug. Even if users hit this case,
- * they can easily online CPU0 back.
- *
- * To simplify this debug code, we only consider normal boot
- * case. Otherwise we need to remember CPU0's state and restore
- * to that state and resolve racy conditions etc.
- */
- _debug_hotplug_cpu(0, 0);
- break;
-#endif
default:
break;
}
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 06/37] x86/smpboot: Remove the CPU0 hotplug kludge
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-14 23:44 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
This was introduced with commit e1c467e69040 ("x86, hotplug: Wake up CPU0
via NMI instead of INIT, SIPI, SIPI") to eventually support physical
hotplug of CPU0:
"We'll change this code in the future to wake up hard offlined CPU0 if
real platform and request are available."
11 years later this has not happened and physical hotplug is not officially
supported. Remove the cruft.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/include/asm/apic.h | 1
arch/x86/include/asm/smp.h | 1
arch/x86/kernel/smpboot.c | 170 +++---------------------------------------
drivers/acpi/processor_idle.c | 4
4 files changed, 14 insertions(+), 162 deletions(-)
--- a/arch/x86/include/asm/apic.h
+++ b/arch/x86/include/asm/apic.h
@@ -377,7 +377,6 @@ extern struct apic *__apicdrivers[], *__
* APIC functionality to boot other CPUs - only used on SMP:
*/
#ifdef CONFIG_SMP
-extern int wakeup_secondary_cpu_via_nmi(int apicid, unsigned long start_eip);
extern int lapic_can_unplug_cpu(void);
#endif
--- a/arch/x86/include/asm/smp.h
+++ b/arch/x86/include/asm/smp.h
@@ -129,7 +129,6 @@ void native_play_dead(void);
void play_dead_common(void);
void wbinvd_on_cpu(int cpu);
int wbinvd_on_all_cpus(void);
-void cond_wakeup_cpu0(void);
void native_smp_send_reschedule(int cpu);
void native_send_call_func_ipi(const struct cpumask *mask);
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -216,9 +216,6 @@ static void ap_calibrate_delay(void)
cpu_data(smp_processor_id()).loops_per_jiffy = loops_per_jiffy;
}
-static int cpu0_logical_apicid;
-static int enable_start_cpu0;
-
/*
* Activate a secondary processor.
*/
@@ -241,8 +238,6 @@ static void notrace start_secondary(void
x86_cpuinit.early_percpu_clock_init();
smp_callin();
- enable_start_cpu0 = 0;
-
/* otherwise gcc will move up smp_processor_id before the cpu_init */
barrier();
/* Check TSC synchronization with the control CPU: */
@@ -410,7 +405,7 @@ void smp_store_cpu_info(int id)
c->cpu_index = id;
/*
* During boot time, CPU0 has this setup already. Save the info when
- * bringing up AP or offlined CPU0.
+ * bringing up an AP.
*/
identify_secondary_cpu(c);
c->initialized = true;
@@ -807,51 +802,14 @@ static void __init smp_quirk_init_udelay
}
/*
- * Poke the other CPU in the eye via NMI to wake it up. Remember that the normal
- * INIT, INIT, STARTUP sequence will reset the chip hard for us, and this
- * won't ... remember to clear down the APIC, etc later.
- */
-int
-wakeup_secondary_cpu_via_nmi(int apicid, unsigned long start_eip)
-{
- u32 dm = apic->dest_mode_logical ? APIC_DEST_LOGICAL : APIC_DEST_PHYSICAL;
- unsigned long send_status, accept_status = 0;
- int maxlvt;
-
- /* Target chip */
- /* Boot on the stack */
- /* Kick the second */
- apic_icr_write(APIC_DM_NMI | dm, apicid);
-
- pr_debug("Waiting for send to finish...\n");
- send_status = safe_apic_wait_icr_idle();
-
- /*
- * Give the other CPU some time to accept the IPI.
- */
- udelay(200);
- if (APIC_INTEGRATED(boot_cpu_apic_version)) {
- maxlvt = lapic_get_maxlvt();
- if (maxlvt > 3) /* Due to the Pentium erratum 3AP. */
- apic_write(APIC_ESR, 0);
- accept_status = (apic_read(APIC_ESR) & 0xEF);
- }
- pr_debug("NMI sent\n");
-
- if (send_status)
- pr_err("APIC never delivered???\n");
- if (accept_status)
- pr_err("APIC delivery error (%lx)\n", accept_status);
-
- return (send_status | accept_status);
-}
-
-static int
-wakeup_secondary_cpu_via_init(int phys_apicid, unsigned long start_eip)
+ * Wake up AP by INIT, INIT, STARTUP sequence.
+ */
+static int wakeup_secondary_cpu_via_init(int phys_apicid, unsigned long start_eip)
{
unsigned long send_status = 0, accept_status = 0;
int maxlvt, num_starts, j;
+ preempt_disable();
maxlvt = lapic_get_maxlvt();
/*
@@ -957,6 +915,7 @@ wakeup_secondary_cpu_via_init(int phys_a
if (accept_status)
pr_err("APIC delivery error (%lx)\n", accept_status);
+ preempt_enable();
return (send_status | accept_status);
}
@@ -997,67 +956,6 @@ static void announce_cpu(int cpu, int ap
node, cpu, apicid);
}
-static int wakeup_cpu0_nmi(unsigned int cmd, struct pt_regs *regs)
-{
- int cpu;
-
- cpu = smp_processor_id();
- if (cpu == 0 && !cpu_online(cpu) && enable_start_cpu0)
- return NMI_HANDLED;
-
- return NMI_DONE;
-}
-
-/*
- * Wake up AP by INIT, INIT, STARTUP sequence.
- *
- * Instead of waiting for STARTUP after INITs, BSP will execute the BIOS
- * boot-strap code which is not a desired behavior for waking up BSP. To
- * void the boot-strap code, wake up CPU0 by NMI instead.
- *
- * This works to wake up soft offlined CPU0 only. If CPU0 is hard offlined
- * (i.e. physically hot removed and then hot added), NMI won't wake it up.
- * We'll change this code in the future to wake up hard offlined CPU0 if
- * real platform and request are available.
- */
-static int
-wakeup_cpu_via_init_nmi(int cpu, unsigned long start_ip, int apicid,
- int *cpu0_nmi_registered)
-{
- int id;
- int boot_error;
-
- preempt_disable();
-
- /*
- * Wake up AP by INIT, INIT, STARTUP sequence.
- */
- if (cpu) {
- boot_error = wakeup_secondary_cpu_via_init(apicid, start_ip);
- goto out;
- }
-
- /*
- * Wake up BSP by nmi.
- *
- * Register a NMI handler to help wake up CPU0.
- */
- boot_error = register_nmi_handler(NMI_LOCAL,
- wakeup_cpu0_nmi, 0, "wake_cpu0");
-
- if (!boot_error) {
- enable_start_cpu0 = 1;
- *cpu0_nmi_registered = 1;
- id = apic->dest_mode_logical ? cpu0_logical_apicid : apicid;
- boot_error = wakeup_secondary_cpu_via_nmi(id, start_ip);
- }
-
-out:
- preempt_enable();
-
- return boot_error;
-}
-
int common_cpu_up(unsigned int cpu, struct task_struct *idle)
{
int ret;
@@ -1086,8 +984,7 @@ int common_cpu_up(unsigned int cpu, stru
* Returns zero if CPU booted OK, else error code from
* ->wakeup_secondary_cpu.
*/
-static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle,
- int *cpu0_nmi_registered)
+static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle)
{
/* start_ip had better be page-aligned! */
unsigned long start_ip = real_mode_header->trampoline_start;
@@ -1120,7 +1017,6 @@ static int do_boot_cpu(int apicid, int c
* This grunge runs the startup process for
* the targeted processor.
*/
-
if (x86_platform.legacy.warm_reset) {
pr_debug("Setting warm reset code and vector.\n");
@@ -1149,15 +1045,14 @@ static int do_boot_cpu(int apicid, int c
* - Use a method from the APIC driver if one defined, with wakeup
* straight to 64-bit mode preferred over wakeup to RM.
* Otherwise,
- * - Use an INIT boot APIC message for APs or NMI for BSP.
+ * - Use an INIT boot APIC message
*/
if (apic->wakeup_secondary_cpu_64)
boot_error = apic->wakeup_secondary_cpu_64(apicid, start_ip);
else if (apic->wakeup_secondary_cpu)
boot_error = apic->wakeup_secondary_cpu(apicid, start_ip);
else
- boot_error = wakeup_cpu_via_init_nmi(cpu, start_ip, apicid,
- cpu0_nmi_registered);
+ boot_error = wakeup_secondary_cpu_via_init(apicid, start_ip);
if (!boot_error) {
/*
@@ -1206,9 +1101,8 @@ static int do_boot_cpu(int apicid, int c
int native_cpu_up(unsigned int cpu, struct task_struct *tidle)
{
int apicid = apic->cpu_present_to_apicid(cpu);
- int cpu0_nmi_registered = 0;
unsigned long flags;
- int err, ret = 0;
+ int err;
lockdep_assert_irqs_enabled();
@@ -1247,11 +1141,10 @@ int native_cpu_up(unsigned int cpu, stru
if (err)
return err;
- err = do_boot_cpu(apicid, cpu, tidle, &cpu0_nmi_registered);
+ err = do_boot_cpu(apicid, cpu, tidle);
if (err) {
pr_err("do_boot_cpu failed(%d) to wakeup CPU#%u\n", err, cpu);
- ret = -EIO;
- goto unreg_nmi;
+ return err;
}
/*
@@ -1267,15 +1160,7 @@ int native_cpu_up(unsigned int cpu, stru
touch_nmi_watchdog();
}
-unreg_nmi:
- /*
- * Clean up the nmi handler. Do this after the callin and callout sync
- * to avoid impact of possible long unregister time.
- */
- if (cpu0_nmi_registered)
- unregister_nmi_handler(NMI_LOCAL, "wake_cpu0");
-
- return ret;
+ return 0;
}
/**
@@ -1373,14 +1258,6 @@ static void __init smp_cpu_index_default
}
}
-static void __init smp_get_logical_apicid(void)
-{
- if (x2apic_mode)
- cpu0_logical_apicid = apic_read(APIC_LDR);
- else
- cpu0_logical_apicid = GET_APIC_LOGICAL_ID(apic_read(APIC_LDR));
-}
-
void __init smp_prepare_cpus_common(void)
{
unsigned int i;
@@ -1443,8 +1320,6 @@ void __init native_smp_prepare_cpus(unsi
/* Setup local timer */
x86_init.timers.setup_percpu_clockev();
- smp_get_logical_apicid();
-
pr_info("CPU0: ");
print_cpu_info(&cpu_data(0));
@@ -1752,18 +1627,6 @@ void play_dead_common(void)
local_irq_disable();
}
-/**
- * cond_wakeup_cpu0 - Wake up CPU0 if needed.
- *
- * If NMI wants to wake up CPU0, start CPU0.
- */
-void cond_wakeup_cpu0(void)
-{
- if (smp_processor_id() == 0 && enable_start_cpu0)
- start_cpu0();
-}
-EXPORT_SYMBOL_GPL(cond_wakeup_cpu0);
-
/*
* We need to flush the caches before going to sleep, lest we have
* dirty data in our caches when we come back up.
@@ -1831,8 +1694,6 @@ static inline void mwait_play_dead(void)
__monitor(mwait_ptr, 0, 0);
mb();
__mwait(eax, 0);
-
- cond_wakeup_cpu0();
}
}
@@ -1841,11 +1702,8 @@ void hlt_play_dead(void)
if (__this_cpu_read(cpu_info.x86) >= 4)
wbinvd();
- while (1) {
+ while (1)
native_halt();
-
- cond_wakeup_cpu0();
- }
}
void native_play_dead(void)
--- a/drivers/acpi/processor_idle.c
+++ b/drivers/acpi/processor_idle.c
@@ -597,10 +597,6 @@ static int acpi_idle_play_dead(struct cp
io_idle(cx->address);
} else
return -ENODEV;
-
-#if defined(CONFIG_X86) && defined(CONFIG_HOTPLUG_CPU)
- cond_wakeup_cpu0();
-#endif
}
/* Never reached */
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 06/37] x86/smpboot: Remove the CPU0 hotplug kludge
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
This was introduced with commit e1c467e69040 ("x86, hotplug: Wake up CPU0
via NMI instead of INIT, SIPI, SIPI") to eventually support physical
hotplug of CPU0:
"We'll change this code in the future to wake up hard offlined CPU0 if
real platform and request are available."
11 years later this has not happened and physical hotplug is not officially
supported. Remove the cruft.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/include/asm/apic.h | 1
arch/x86/include/asm/smp.h | 1
arch/x86/kernel/smpboot.c | 170 +++---------------------------------------
drivers/acpi/processor_idle.c | 4
4 files changed, 14 insertions(+), 162 deletions(-)
--- a/arch/x86/include/asm/apic.h
+++ b/arch/x86/include/asm/apic.h
@@ -377,7 +377,6 @@ extern struct apic *__apicdrivers[], *__
* APIC functionality to boot other CPUs - only used on SMP:
*/
#ifdef CONFIG_SMP
-extern int wakeup_secondary_cpu_via_nmi(int apicid, unsigned long start_eip);
extern int lapic_can_unplug_cpu(void);
#endif
--- a/arch/x86/include/asm/smp.h
+++ b/arch/x86/include/asm/smp.h
@@ -129,7 +129,6 @@ void native_play_dead(void);
void play_dead_common(void);
void wbinvd_on_cpu(int cpu);
int wbinvd_on_all_cpus(void);
-void cond_wakeup_cpu0(void);
void native_smp_send_reschedule(int cpu);
void native_send_call_func_ipi(const struct cpumask *mask);
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -216,9 +216,6 @@ static void ap_calibrate_delay(void)
cpu_data(smp_processor_id()).loops_per_jiffy = loops_per_jiffy;
}
-static int cpu0_logical_apicid;
-static int enable_start_cpu0;
-
/*
* Activate a secondary processor.
*/
@@ -241,8 +238,6 @@ static void notrace start_secondary(void
x86_cpuinit.early_percpu_clock_init();
smp_callin();
- enable_start_cpu0 = 0;
-
/* otherwise gcc will move up smp_processor_id before the cpu_init */
barrier();
/* Check TSC synchronization with the control CPU: */
@@ -410,7 +405,7 @@ void smp_store_cpu_info(int id)
c->cpu_index = id;
/*
* During boot time, CPU0 has this setup already. Save the info when
- * bringing up AP or offlined CPU0.
+ * bringing up an AP.
*/
identify_secondary_cpu(c);
c->initialized = true;
@@ -807,51 +802,14 @@ static void __init smp_quirk_init_udelay
}
/*
- * Poke the other CPU in the eye via NMI to wake it up. Remember that the normal
- * INIT, INIT, STARTUP sequence will reset the chip hard for us, and this
- * won't ... remember to clear down the APIC, etc later.
- */
-int
-wakeup_secondary_cpu_via_nmi(int apicid, unsigned long start_eip)
-{
- u32 dm = apic->dest_mode_logical ? APIC_DEST_LOGICAL : APIC_DEST_PHYSICAL;
- unsigned long send_status, accept_status = 0;
- int maxlvt;
-
- /* Target chip */
- /* Boot on the stack */
- /* Kick the second */
- apic_icr_write(APIC_DM_NMI | dm, apicid);
-
- pr_debug("Waiting for send to finish...\n");
- send_status = safe_apic_wait_icr_idle();
-
- /*
- * Give the other CPU some time to accept the IPI.
- */
- udelay(200);
- if (APIC_INTEGRATED(boot_cpu_apic_version)) {
- maxlvt = lapic_get_maxlvt();
- if (maxlvt > 3) /* Due to the Pentium erratum 3AP. */
- apic_write(APIC_ESR, 0);
- accept_status = (apic_read(APIC_ESR) & 0xEF);
- }
- pr_debug("NMI sent\n");
-
- if (send_status)
- pr_err("APIC never delivered???\n");
- if (accept_status)
- pr_err("APIC delivery error (%lx)\n", accept_status);
-
- return (send_status | accept_status);
-}
-
-static int
-wakeup_secondary_cpu_via_init(int phys_apicid, unsigned long start_eip)
+ * Wake up AP by INIT, INIT, STARTUP sequence.
+ */
+static int wakeup_secondary_cpu_via_init(int phys_apicid, unsigned long start_eip)
{
unsigned long send_status = 0, accept_status = 0;
int maxlvt, num_starts, j;
+ preempt_disable();
maxlvt = lapic_get_maxlvt();
/*
@@ -957,6 +915,7 @@ wakeup_secondary_cpu_via_init(int phys_a
if (accept_status)
pr_err("APIC delivery error (%lx)\n", accept_status);
+ preempt_enable();
return (send_status | accept_status);
}
@@ -997,67 +956,6 @@ static void announce_cpu(int cpu, int ap
node, cpu, apicid);
}
-static int wakeup_cpu0_nmi(unsigned int cmd, struct pt_regs *regs)
-{
- int cpu;
-
- cpu = smp_processor_id();
- if (cpu == 0 && !cpu_online(cpu) && enable_start_cpu0)
- return NMI_HANDLED;
-
- return NMI_DONE;
-}
-
-/*
- * Wake up AP by INIT, INIT, STARTUP sequence.
- *
- * Instead of waiting for STARTUP after INITs, BSP will execute the BIOS
- * boot-strap code which is not a desired behavior for waking up BSP. To
- * void the boot-strap code, wake up CPU0 by NMI instead.
- *
- * This works to wake up soft offlined CPU0 only. If CPU0 is hard offlined
- * (i.e. physically hot removed and then hot added), NMI won't wake it up.
- * We'll change this code in the future to wake up hard offlined CPU0 if
- * real platform and request are available.
- */
-static int
-wakeup_cpu_via_init_nmi(int cpu, unsigned long start_ip, int apicid,
- int *cpu0_nmi_registered)
-{
- int id;
- int boot_error;
-
- preempt_disable();
-
- /*
- * Wake up AP by INIT, INIT, STARTUP sequence.
- */
- if (cpu) {
- boot_error = wakeup_secondary_cpu_via_init(apicid, start_ip);
- goto out;
- }
-
- /*
- * Wake up BSP by nmi.
- *
- * Register a NMI handler to help wake up CPU0.
- */
- boot_error = register_nmi_handler(NMI_LOCAL,
- wakeup_cpu0_nmi, 0, "wake_cpu0");
-
- if (!boot_error) {
- enable_start_cpu0 = 1;
- *cpu0_nmi_registered = 1;
- id = apic->dest_mode_logical ? cpu0_logical_apicid : apicid;
- boot_error = wakeup_secondary_cpu_via_nmi(id, start_ip);
- }
-
-out:
- preempt_enable();
-
- return boot_error;
-}
-
int common_cpu_up(unsigned int cpu, struct task_struct *idle)
{
int ret;
@@ -1086,8 +984,7 @@ int common_cpu_up(unsigned int cpu, stru
* Returns zero if CPU booted OK, else error code from
* ->wakeup_secondary_cpu.
*/
-static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle,
- int *cpu0_nmi_registered)
+static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle)
{
/* start_ip had better be page-aligned! */
unsigned long start_ip = real_mode_header->trampoline_start;
@@ -1120,7 +1017,6 @@ static int do_boot_cpu(int apicid, int c
* This grunge runs the startup process for
* the targeted processor.
*/
-
if (x86_platform.legacy.warm_reset) {
pr_debug("Setting warm reset code and vector.\n");
@@ -1149,15 +1045,14 @@ static int do_boot_cpu(int apicid, int c
* - Use a method from the APIC driver if one defined, with wakeup
* straight to 64-bit mode preferred over wakeup to RM.
* Otherwise,
- * - Use an INIT boot APIC message for APs or NMI for BSP.
+ * - Use an INIT boot APIC message
*/
if (apic->wakeup_secondary_cpu_64)
boot_error = apic->wakeup_secondary_cpu_64(apicid, start_ip);
else if (apic->wakeup_secondary_cpu)
boot_error = apic->wakeup_secondary_cpu(apicid, start_ip);
else
- boot_error = wakeup_cpu_via_init_nmi(cpu, start_ip, apicid,
- cpu0_nmi_registered);
+ boot_error = wakeup_secondary_cpu_via_init(apicid, start_ip);
if (!boot_error) {
/*
@@ -1206,9 +1101,8 @@ static int do_boot_cpu(int apicid, int c
int native_cpu_up(unsigned int cpu, struct task_struct *tidle)
{
int apicid = apic->cpu_present_to_apicid(cpu);
- int cpu0_nmi_registered = 0;
unsigned long flags;
- int err, ret = 0;
+ int err;
lockdep_assert_irqs_enabled();
@@ -1247,11 +1141,10 @@ int native_cpu_up(unsigned int cpu, stru
if (err)
return err;
- err = do_boot_cpu(apicid, cpu, tidle, &cpu0_nmi_registered);
+ err = do_boot_cpu(apicid, cpu, tidle);
if (err) {
pr_err("do_boot_cpu failed(%d) to wakeup CPU#%u\n", err, cpu);
- ret = -EIO;
- goto unreg_nmi;
+ return err;
}
/*
@@ -1267,15 +1160,7 @@ int native_cpu_up(unsigned int cpu, stru
touch_nmi_watchdog();
}
-unreg_nmi:
- /*
- * Clean up the nmi handler. Do this after the callin and callout sync
- * to avoid impact of possible long unregister time.
- */
- if (cpu0_nmi_registered)
- unregister_nmi_handler(NMI_LOCAL, "wake_cpu0");
-
- return ret;
+ return 0;
}
/**
@@ -1373,14 +1258,6 @@ static void __init smp_cpu_index_default
}
}
-static void __init smp_get_logical_apicid(void)
-{
- if (x2apic_mode)
- cpu0_logical_apicid = apic_read(APIC_LDR);
- else
- cpu0_logical_apicid = GET_APIC_LOGICAL_ID(apic_read(APIC_LDR));
-}
-
void __init smp_prepare_cpus_common(void)
{
unsigned int i;
@@ -1443,8 +1320,6 @@ void __init native_smp_prepare_cpus(unsi
/* Setup local timer */
x86_init.timers.setup_percpu_clockev();
- smp_get_logical_apicid();
-
pr_info("CPU0: ");
print_cpu_info(&cpu_data(0));
@@ -1752,18 +1627,6 @@ void play_dead_common(void)
local_irq_disable();
}
-/**
- * cond_wakeup_cpu0 - Wake up CPU0 if needed.
- *
- * If NMI wants to wake up CPU0, start CPU0.
- */
-void cond_wakeup_cpu0(void)
-{
- if (smp_processor_id() == 0 && enable_start_cpu0)
- start_cpu0();
-}
-EXPORT_SYMBOL_GPL(cond_wakeup_cpu0);
-
/*
* We need to flush the caches before going to sleep, lest we have
* dirty data in our caches when we come back up.
@@ -1831,8 +1694,6 @@ static inline void mwait_play_dead(void)
__monitor(mwait_ptr, 0, 0);
mb();
__mwait(eax, 0);
-
- cond_wakeup_cpu0();
}
}
@@ -1841,11 +1702,8 @@ void hlt_play_dead(void)
if (__this_cpu_read(cpu_info.x86) >= 4)
wbinvd();
- while (1) {
+ while (1)
native_halt();
-
- cond_wakeup_cpu0();
- }
}
void native_play_dead(void)
--- a/drivers/acpi/processor_idle.c
+++ b/drivers/acpi/processor_idle.c
@@ -597,10 +597,6 @@ static int acpi_idle_play_dead(struct cp
io_idle(cx->address);
} else
return -ENODEV;
-
-#if defined(CONFIG_X86) && defined(CONFIG_HOTPLUG_CPU)
- cond_wakeup_cpu0();
-#endif
}
/* Never reached */
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 06/37] x86/smpboot: Remove the CPU0 hotplug kludge
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
This was introduced with commit e1c467e69040 ("x86, hotplug: Wake up CPU0
via NMI instead of INIT, SIPI, SIPI") to eventually support physical
hotplug of CPU0:
"We'll change this code in the future to wake up hard offlined CPU0 if
real platform and request are available."
11 years later this has not happened and physical hotplug is not officially
supported. Remove the cruft.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/include/asm/apic.h | 1
arch/x86/include/asm/smp.h | 1
arch/x86/kernel/smpboot.c | 170 +++---------------------------------------
drivers/acpi/processor_idle.c | 4
4 files changed, 14 insertions(+), 162 deletions(-)
--- a/arch/x86/include/asm/apic.h
+++ b/arch/x86/include/asm/apic.h
@@ -377,7 +377,6 @@ extern struct apic *__apicdrivers[], *__
* APIC functionality to boot other CPUs - only used on SMP:
*/
#ifdef CONFIG_SMP
-extern int wakeup_secondary_cpu_via_nmi(int apicid, unsigned long start_eip);
extern int lapic_can_unplug_cpu(void);
#endif
--- a/arch/x86/include/asm/smp.h
+++ b/arch/x86/include/asm/smp.h
@@ -129,7 +129,6 @@ void native_play_dead(void);
void play_dead_common(void);
void wbinvd_on_cpu(int cpu);
int wbinvd_on_all_cpus(void);
-void cond_wakeup_cpu0(void);
void native_smp_send_reschedule(int cpu);
void native_send_call_func_ipi(const struct cpumask *mask);
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -216,9 +216,6 @@ static void ap_calibrate_delay(void)
cpu_data(smp_processor_id()).loops_per_jiffy = loops_per_jiffy;
}
-static int cpu0_logical_apicid;
-static int enable_start_cpu0;
-
/*
* Activate a secondary processor.
*/
@@ -241,8 +238,6 @@ static void notrace start_secondary(void
x86_cpuinit.early_percpu_clock_init();
smp_callin();
- enable_start_cpu0 = 0;
-
/* otherwise gcc will move up smp_processor_id before the cpu_init */
barrier();
/* Check TSC synchronization with the control CPU: */
@@ -410,7 +405,7 @@ void smp_store_cpu_info(int id)
c->cpu_index = id;
/*
* During boot time, CPU0 has this setup already. Save the info when
- * bringing up AP or offlined CPU0.
+ * bringing up an AP.
*/
identify_secondary_cpu(c);
c->initialized = true;
@@ -807,51 +802,14 @@ static void __init smp_quirk_init_udelay
}
/*
- * Poke the other CPU in the eye via NMI to wake it up. Remember that the normal
- * INIT, INIT, STARTUP sequence will reset the chip hard for us, and this
- * won't ... remember to clear down the APIC, etc later.
- */
-int
-wakeup_secondary_cpu_via_nmi(int apicid, unsigned long start_eip)
-{
- u32 dm = apic->dest_mode_logical ? APIC_DEST_LOGICAL : APIC_DEST_PHYSICAL;
- unsigned long send_status, accept_status = 0;
- int maxlvt;
-
- /* Target chip */
- /* Boot on the stack */
- /* Kick the second */
- apic_icr_write(APIC_DM_NMI | dm, apicid);
-
- pr_debug("Waiting for send to finish...\n");
- send_status = safe_apic_wait_icr_idle();
-
- /*
- * Give the other CPU some time to accept the IPI.
- */
- udelay(200);
- if (APIC_INTEGRATED(boot_cpu_apic_version)) {
- maxlvt = lapic_get_maxlvt();
- if (maxlvt > 3) /* Due to the Pentium erratum 3AP. */
- apic_write(APIC_ESR, 0);
- accept_status = (apic_read(APIC_ESR) & 0xEF);
- }
- pr_debug("NMI sent\n");
-
- if (send_status)
- pr_err("APIC never delivered???\n");
- if (accept_status)
- pr_err("APIC delivery error (%lx)\n", accept_status);
-
- return (send_status | accept_status);
-}
-
-static int
-wakeup_secondary_cpu_via_init(int phys_apicid, unsigned long start_eip)
+ * Wake up AP by INIT, INIT, STARTUP sequence.
+ */
+static int wakeup_secondary_cpu_via_init(int phys_apicid, unsigned long start_eip)
{
unsigned long send_status = 0, accept_status = 0;
int maxlvt, num_starts, j;
+ preempt_disable();
maxlvt = lapic_get_maxlvt();
/*
@@ -957,6 +915,7 @@ wakeup_secondary_cpu_via_init(int phys_a
if (accept_status)
pr_err("APIC delivery error (%lx)\n", accept_status);
+ preempt_enable();
return (send_status | accept_status);
}
@@ -997,67 +956,6 @@ static void announce_cpu(int cpu, int ap
node, cpu, apicid);
}
-static int wakeup_cpu0_nmi(unsigned int cmd, struct pt_regs *regs)
-{
- int cpu;
-
- cpu = smp_processor_id();
- if (cpu == 0 && !cpu_online(cpu) && enable_start_cpu0)
- return NMI_HANDLED;
-
- return NMI_DONE;
-}
-
-/*
- * Wake up AP by INIT, INIT, STARTUP sequence.
- *
- * Instead of waiting for STARTUP after INITs, BSP will execute the BIOS
- * boot-strap code which is not a desired behavior for waking up BSP. To
- * void the boot-strap code, wake up CPU0 by NMI instead.
- *
- * This works to wake up soft offlined CPU0 only. If CPU0 is hard offlined
- * (i.e. physically hot removed and then hot added), NMI won't wake it up.
- * We'll change this code in the future to wake up hard offlined CPU0 if
- * real platform and request are available.
- */
-static int
-wakeup_cpu_via_init_nmi(int cpu, unsigned long start_ip, int apicid,
- int *cpu0_nmi_registered)
-{
- int id;
- int boot_error;
-
- preempt_disable();
-
- /*
- * Wake up AP by INIT, INIT, STARTUP sequence.
- */
- if (cpu) {
- boot_error = wakeup_secondary_cpu_via_init(apicid, start_ip);
- goto out;
- }
-
- /*
- * Wake up BSP by nmi.
- *
- * Register a NMI handler to help wake up CPU0.
- */
- boot_error = register_nmi_handler(NMI_LOCAL,
- wakeup_cpu0_nmi, 0, "wake_cpu0");
-
- if (!boot_error) {
- enable_start_cpu0 = 1;
- *cpu0_nmi_registered = 1;
- id = apic->dest_mode_logical ? cpu0_logical_apicid : apicid;
- boot_error = wakeup_secondary_cpu_via_nmi(id, start_ip);
- }
-
-out:
- preempt_enable();
-
- return boot_error;
-}
-
int common_cpu_up(unsigned int cpu, struct task_struct *idle)
{
int ret;
@@ -1086,8 +984,7 @@ int common_cpu_up(unsigned int cpu, stru
* Returns zero if CPU booted OK, else error code from
* ->wakeup_secondary_cpu.
*/
-static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle,
- int *cpu0_nmi_registered)
+static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle)
{
/* start_ip had better be page-aligned! */
unsigned long start_ip = real_mode_header->trampoline_start;
@@ -1120,7 +1017,6 @@ static int do_boot_cpu(int apicid, int c
* This grunge runs the startup process for
* the targeted processor.
*/
-
if (x86_platform.legacy.warm_reset) {
pr_debug("Setting warm reset code and vector.\n");
@@ -1149,15 +1045,14 @@ static int do_boot_cpu(int apicid, int c
* - Use a method from the APIC driver if one defined, with wakeup
* straight to 64-bit mode preferred over wakeup to RM.
* Otherwise,
- * - Use an INIT boot APIC message for APs or NMI for BSP.
+ * - Use an INIT boot APIC message
*/
if (apic->wakeup_secondary_cpu_64)
boot_error = apic->wakeup_secondary_cpu_64(apicid, start_ip);
else if (apic->wakeup_secondary_cpu)
boot_error = apic->wakeup_secondary_cpu(apicid, start_ip);
else
- boot_error = wakeup_cpu_via_init_nmi(cpu, start_ip, apicid,
- cpu0_nmi_registered);
+ boot_error = wakeup_secondary_cpu_via_init(apicid, start_ip);
if (!boot_error) {
/*
@@ -1206,9 +1101,8 @@ static int do_boot_cpu(int apicid, int c
int native_cpu_up(unsigned int cpu, struct task_struct *tidle)
{
int apicid = apic->cpu_present_to_apicid(cpu);
- int cpu0_nmi_registered = 0;
unsigned long flags;
- int err, ret = 0;
+ int err;
lockdep_assert_irqs_enabled();
@@ -1247,11 +1141,10 @@ int native_cpu_up(unsigned int cpu, stru
if (err)
return err;
- err = do_boot_cpu(apicid, cpu, tidle, &cpu0_nmi_registered);
+ err = do_boot_cpu(apicid, cpu, tidle);
if (err) {
pr_err("do_boot_cpu failed(%d) to wakeup CPU#%u\n", err, cpu);
- ret = -EIO;
- goto unreg_nmi;
+ return err;
}
/*
@@ -1267,15 +1160,7 @@ int native_cpu_up(unsigned int cpu, stru
touch_nmi_watchdog();
}
-unreg_nmi:
- /*
- * Clean up the nmi handler. Do this after the callin and callout sync
- * to avoid impact of possible long unregister time.
- */
- if (cpu0_nmi_registered)
- unregister_nmi_handler(NMI_LOCAL, "wake_cpu0");
-
- return ret;
+ return 0;
}
/**
@@ -1373,14 +1258,6 @@ static void __init smp_cpu_index_default
}
}
-static void __init smp_get_logical_apicid(void)
-{
- if (x2apic_mode)
- cpu0_logical_apicid = apic_read(APIC_LDR);
- else
- cpu0_logical_apicid = GET_APIC_LOGICAL_ID(apic_read(APIC_LDR));
-}
-
void __init smp_prepare_cpus_common(void)
{
unsigned int i;
@@ -1443,8 +1320,6 @@ void __init native_smp_prepare_cpus(unsi
/* Setup local timer */
x86_init.timers.setup_percpu_clockev();
- smp_get_logical_apicid();
-
pr_info("CPU0: ");
print_cpu_info(&cpu_data(0));
@@ -1752,18 +1627,6 @@ void play_dead_common(void)
local_irq_disable();
}
-/**
- * cond_wakeup_cpu0 - Wake up CPU0 if needed.
- *
- * If NMI wants to wake up CPU0, start CPU0.
- */
-void cond_wakeup_cpu0(void)
-{
- if (smp_processor_id() == 0 && enable_start_cpu0)
- start_cpu0();
-}
-EXPORT_SYMBOL_GPL(cond_wakeup_cpu0);
-
/*
* We need to flush the caches before going to sleep, lest we have
* dirty data in our caches when we come back up.
@@ -1831,8 +1694,6 @@ static inline void mwait_play_dead(void)
__monitor(mwait_ptr, 0, 0);
mb();
__mwait(eax, 0);
-
- cond_wakeup_cpu0();
}
}
@@ -1841,11 +1702,8 @@ void hlt_play_dead(void)
if (__this_cpu_read(cpu_info.x86) >= 4)
wbinvd();
- while (1) {
+ while (1)
native_halt();
-
- cond_wakeup_cpu0();
- }
}
void native_play_dead(void)
--- a/drivers/acpi/processor_idle.c
+++ b/drivers/acpi/processor_idle.c
@@ -597,10 +597,6 @@ static int acpi_idle_play_dead(struct cp
io_idle(cx->address);
} else
return -ENODEV;
-
-#if defined(CONFIG_X86) && defined(CONFIG_HOTPLUG_CPU)
- cond_wakeup_cpu0();
-#endif
}
/* Never reached */
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 07/37] x86/smpboot: Restrict soft_restart_cpu() to SEV
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-14 23:44 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Now that the CPU0 hotplug cruft is gone, the only user is AMD SEV.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
---
arch/x86/kernel/callthunks.c | 2 +-
arch/x86/kernel/head_32.S | 14 --------------
arch/x86/kernel/head_64.S | 2 +-
3 files changed, 2 insertions(+), 16 deletions(-)
--- a/arch/x86/kernel/callthunks.c
+++ b/arch/x86/kernel/callthunks.c
@@ -133,7 +133,7 @@ static bool skip_addr(void *dest)
/* Accounts directly */
if (dest == ret_from_fork)
return true;
-#ifdef CONFIG_HOTPLUG_CPU
+#if defined(CONFIG_HOTPLUG_CPU) && defined(CONFIG_AMD_MEM_ENCRYPT)
if (dest == soft_restart_cpu)
return true;
#endif
--- a/arch/x86/kernel/head_32.S
+++ b/arch/x86/kernel/head_32.S
@@ -138,20 +138,6 @@ SYM_CODE_START(startup_32)
jmp .Ldefault_entry
SYM_CODE_END(startup_32)
-#ifdef CONFIG_HOTPLUG_CPU
-/*
- * Entry point for soft restart of a CPU. Invoked from xxx_play_dead() for
- * restarting the boot CPU or for restarting SEV guest CPUs after CPU hot
- * unplug. Everything is set up already except the stack.
- */
-SYM_FUNC_START(soft_restart_cpu)
- movl initial_stack, %ecx
- movl %ecx, %esp
- call *(initial_code)
-1: jmp 1b
-SYM_FUNC_END(soft_restart_cpu)
-#endif
-
/*
* Non-boot CPU entry point; entered from trampoline.S
* We can't lgdt here, because lgdt itself uses a data segment, but
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -375,7 +375,7 @@ SYM_CODE_END(secondary_startup_64)
#include "verify_cpu.S"
#include "sev_verify_cbit.S"
-#ifdef CONFIG_HOTPLUG_CPU
+#if defined(CONFIG_HOTPLUG_CPU) && defined(CONFIG_AMD_MEM_ENCRYPT)
/*
* Entry point for soft restart of a CPU. Invoked from xxx_play_dead() for
* restarting the boot CPU or for restarting SEV guest CPUs after CPU hot
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 07/37] x86/smpboot: Restrict soft_restart_cpu() to SEV
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Now that the CPU0 hotplug cruft is gone, the only user is AMD SEV.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
---
arch/x86/kernel/callthunks.c | 2 +-
arch/x86/kernel/head_32.S | 14 --------------
arch/x86/kernel/head_64.S | 2 +-
3 files changed, 2 insertions(+), 16 deletions(-)
--- a/arch/x86/kernel/callthunks.c
+++ b/arch/x86/kernel/callthunks.c
@@ -133,7 +133,7 @@ static bool skip_addr(void *dest)
/* Accounts directly */
if (dest == ret_from_fork)
return true;
-#ifdef CONFIG_HOTPLUG_CPU
+#if defined(CONFIG_HOTPLUG_CPU) && defined(CONFIG_AMD_MEM_ENCRYPT)
if (dest == soft_restart_cpu)
return true;
#endif
--- a/arch/x86/kernel/head_32.S
+++ b/arch/x86/kernel/head_32.S
@@ -138,20 +138,6 @@ SYM_CODE_START(startup_32)
jmp .Ldefault_entry
SYM_CODE_END(startup_32)
-#ifdef CONFIG_HOTPLUG_CPU
-/*
- * Entry point for soft restart of a CPU. Invoked from xxx_play_dead() for
- * restarting the boot CPU or for restarting SEV guest CPUs after CPU hot
- * unplug. Everything is set up already except the stack.
- */
-SYM_FUNC_START(soft_restart_cpu)
- movl initial_stack, %ecx
- movl %ecx, %esp
- call *(initial_code)
-1: jmp 1b
-SYM_FUNC_END(soft_restart_cpu)
-#endif
-
/*
* Non-boot CPU entry point; entered from trampoline.S
* We can't lgdt here, because lgdt itself uses a data segment, but
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -375,7 +375,7 @@ SYM_CODE_END(secondary_startup_64)
#include "verify_cpu.S"
#include "sev_verify_cbit.S"
-#ifdef CONFIG_HOTPLUG_CPU
+#if defined(CONFIG_HOTPLUG_CPU) && defined(CONFIG_AMD_MEM_ENCRYPT)
/*
* Entry point for soft restart of a CPU. Invoked from xxx_play_dead() for
* restarting the boot CPU or for restarting SEV guest CPUs after CPU hot
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 07/37] x86/smpboot: Restrict soft_restart_cpu() to SEV
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Now that the CPU0 hotplug cruft is gone, the only user is AMD SEV.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
---
arch/x86/kernel/callthunks.c | 2 +-
arch/x86/kernel/head_32.S | 14 --------------
arch/x86/kernel/head_64.S | 2 +-
3 files changed, 2 insertions(+), 16 deletions(-)
--- a/arch/x86/kernel/callthunks.c
+++ b/arch/x86/kernel/callthunks.c
@@ -133,7 +133,7 @@ static bool skip_addr(void *dest)
/* Accounts directly */
if (dest == ret_from_fork)
return true;
-#ifdef CONFIG_HOTPLUG_CPU
+#if defined(CONFIG_HOTPLUG_CPU) && defined(CONFIG_AMD_MEM_ENCRYPT)
if (dest == soft_restart_cpu)
return true;
#endif
--- a/arch/x86/kernel/head_32.S
+++ b/arch/x86/kernel/head_32.S
@@ -138,20 +138,6 @@ SYM_CODE_START(startup_32)
jmp .Ldefault_entry
SYM_CODE_END(startup_32)
-#ifdef CONFIG_HOTPLUG_CPU
-/*
- * Entry point for soft restart of a CPU. Invoked from xxx_play_dead() for
- * restarting the boot CPU or for restarting SEV guest CPUs after CPU hot
- * unplug. Everything is set up already except the stack.
- */
-SYM_FUNC_START(soft_restart_cpu)
- movl initial_stack, %ecx
- movl %ecx, %esp
- call *(initial_code)
-1: jmp 1b
-SYM_FUNC_END(soft_restart_cpu)
-#endif
-
/*
* Non-boot CPU entry point; entered from trampoline.S
* We can't lgdt here, because lgdt itself uses a data segment, but
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -375,7 +375,7 @@ SYM_CODE_END(secondary_startup_64)
#include "verify_cpu.S"
#include "sev_verify_cbit.S"
-#ifdef CONFIG_HOTPLUG_CPU
+#if defined(CONFIG_HOTPLUG_CPU) && defined(CONFIG_AMD_MEM_ENCRYPT)
/*
* Entry point for soft restart of a CPU. Invoked from xxx_play_dead() for
* restarting the boot CPU or for restarting SEV guest CPUs after CPU hot
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 08/37] x86/smpboot: Split up native_cpu_up() into separate phases and document them
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-14 23:44 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
From: David Woodhouse <dwmw@amazon.co.uk>
There are four logical parts to what native_cpu_up() does on the BSP (or
on the controlling CPU for a later hotplug):
1) Wake the AP by sending the INIT/SIPI/SIPI sequence.
2) Wait for the AP to make it as far as wait_for_master_cpu() which
sets that CPU's bit in cpu_initialized_mask, then sets the bit in
cpu_callout_mask to let the AP proceed through cpu_init().
3) Wait for the AP to finish cpu_init() and get as far as the
smp_callin() call, which sets that CPU's bit in cpu_callin_mask.
4) Perform the TSC synchronization and wait for the AP to actually
mark itself online in cpu_online_mask.
In preparation to allow these phases to operate in parallel on multiple
APs, split them out into separate functions and document the interactions
a little more clearly in both the BP and AP code paths.
No functional change intended.
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: Usama Arif <usama.arif@bytedance.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/kernel/smpboot.c | 187 +++++++++++++++++++++++++++++-----------------
1 file changed, 121 insertions(+), 66 deletions(-)
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -193,6 +193,10 @@ static void smp_callin(void)
wmb();
+ /*
+ * This runs the AP through all the cpuhp states to its target
+ * state (CPUHP_ONLINE in the case of serial bringup).
+ */
notify_cpu_starting(cpuid);
/*
@@ -233,14 +237,31 @@ static void notrace start_secondary(void
load_cr3(swapper_pg_dir);
__flush_tlb_all();
#endif
+ /*
+ * Sync point with wait_cpu_initialized(). Before proceeding through
+ * cpu_init(), the AP will call wait_for_master_cpu() which sets its
+ * own bit in cpu_initialized_mask and then waits for the BSP to set
+ * its bit in cpu_callout_mask to release it.
+ */
cpu_init_secondary();
rcu_cpu_starting(raw_smp_processor_id());
x86_cpuinit.early_percpu_clock_init();
+
+ /*
+ * Sync point with wait_cpu_callin(). The AP doesn't wait here
+ * but just sets the bit to let the controlling CPU (BSP) know that
+ * it's got this far.
+ */
smp_callin();
- /* otherwise gcc will move up smp_processor_id before the cpu_init */
+ /* Otherwise gcc will move up smp_processor_id() before cpu_init() */
barrier();
- /* Check TSC synchronization with the control CPU: */
+
+ /*
+ * Check TSC synchronization with the control CPU, which will do
+ * its part of this from wait_cpu_online(), making it an implicit
+ * synchronization point.
+ */
check_tsc_sync_target();
/*
@@ -259,6 +280,7 @@ static void notrace start_secondary(void
* half valid vector space.
*/
lock_vector_lock();
+ /* Sync point with do_wait_cpu_online() */
set_cpu_online(smp_processor_id(), true);
lapic_online();
unlock_vector_lock();
@@ -981,17 +1003,13 @@ int common_cpu_up(unsigned int cpu, stru
/*
* NOTE - on most systems this is a PHYSICAL apic ID, but on multiquad
* (ie clustered apic addressing mode), this is a LOGICAL apic ID.
- * Returns zero if CPU booted OK, else error code from
+ * Returns zero if startup was successfully sent, else error code from
* ->wakeup_secondary_cpu.
*/
static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle)
{
- /* start_ip had better be page-aligned! */
unsigned long start_ip = real_mode_header->trampoline_start;
- unsigned long boot_error = 0;
- unsigned long timeout;
-
#ifdef CONFIG_X86_64
/* If 64-bit wakeup method exists, use the 64-bit mode trampoline IP */
if (apic->wakeup_secondary_cpu_64)
@@ -1048,60 +1066,89 @@ static int do_boot_cpu(int apicid, int c
* - Use an INIT boot APIC message
*/
if (apic->wakeup_secondary_cpu_64)
- boot_error = apic->wakeup_secondary_cpu_64(apicid, start_ip);
+ return apic->wakeup_secondary_cpu_64(apicid, start_ip);
else if (apic->wakeup_secondary_cpu)
- boot_error = apic->wakeup_secondary_cpu(apicid, start_ip);
- else
- boot_error = wakeup_secondary_cpu_via_init(apicid, start_ip);
+ return apic->wakeup_secondary_cpu(apicid, start_ip);
- if (!boot_error) {
- /*
- * Wait 10s total for first sign of life from AP
- */
- boot_error = -1;
- timeout = jiffies + 10*HZ;
- while (time_before(jiffies, timeout)) {
- if (cpumask_test_cpu(cpu, cpu_initialized_mask)) {
- /*
- * Tell AP to proceed with initialization
- */
- cpumask_set_cpu(cpu, cpu_callout_mask);
- boot_error = 0;
- break;
- }
- schedule();
- }
- }
+ return wakeup_secondary_cpu_via_init(apicid, start_ip);
+}
- if (!boot_error) {
- /*
- * Wait till AP completes initial initialization
- */
- while (!cpumask_test_cpu(cpu, cpu_callin_mask)) {
- /*
- * Allow other tasks to run while we wait for the
- * AP to come online. This also gives a chance
- * for the MTRR work(triggered by the AP coming online)
- * to be completed in the stop machine context.
- */
- schedule();
- }
- }
+static int wait_cpu_cpumask(unsigned int cpu, const struct cpumask *mask)
+{
+ unsigned long timeout;
- if (x86_platform.legacy.warm_reset) {
- /*
- * Cleanup possible dangling ends...
- */
- smpboot_restore_warm_reset_vector();
+ /*
+ * Wait up to 10s for the CPU to report in.
+ */
+ timeout = jiffies + 10*HZ;
+ while (time_before(jiffies, timeout)) {
+ if (cpumask_test_cpu(cpu, mask))
+ return 0;
+
+ schedule();
}
+ return -1;
+}
- return boot_error;
+/*
+ * Bringup step two: Wait for the target AP to reach cpu_init_secondary()
+ * and thus wait_for_master_cpu(), then set cpu_callout_mask to allow it
+ * to proceed. The AP will then proceed past setting its 'callin' bit
+ * and end up waiting in check_tsc_sync_target() until we reach
+ * do_wait_cpu_online() to tend to it.
+ */
+static int wait_cpu_initialized(unsigned int cpu)
+{
+ /*
+ * Wait for first sign of life from AP.
+ */
+ if (wait_cpu_cpumask(cpu, cpu_initialized_mask))
+ return -1;
+
+ cpumask_set_cpu(cpu, cpu_callout_mask);
+ return 0;
}
-int native_cpu_up(unsigned int cpu, struct task_struct *tidle)
+/*
+ * Bringup step three: Wait for the target AP to reach smp_callin().
+ * The AP is not waiting for us here so we don't need to parallelise
+ * this step. Not entirely clear why we care about this, since we just
+ * proceed directly to TSC synchronization which is the next sync
+ * point with the AP anyway.
+ */
+static void wait_cpu_callin(unsigned int cpu)
+{
+ while (!cpumask_test_cpu(cpu, cpu_callin_mask))
+ schedule();
+}
+
+/*
+ * Bringup step four: Synchronize the TSC and wait for the target AP
+ * to reach set_cpu_online() in start_secondary().
+ */
+static void wait_cpu_online(unsigned int cpu)
{
- int apicid = apic->cpu_present_to_apicid(cpu);
unsigned long flags;
+
+ /*
+ * Check TSC synchronization with the AP (keep irqs disabled
+ * while doing so):
+ */
+ local_irq_save(flags);
+ check_tsc_sync_source(cpu);
+ local_irq_restore(flags);
+
+ /*
+ * Wait for the AP to mark itself online, so the core caller
+ * can drop sparse_irq_lock.
+ */
+ while (!cpu_online(cpu))
+ schedule();
+}
+
+static int native_kick_ap(unsigned int cpu, struct task_struct *tidle)
+{
+ int apicid = apic->cpu_present_to_apicid(cpu);
int err;
lockdep_assert_irqs_enabled();
@@ -1142,25 +1189,33 @@ int native_cpu_up(unsigned int cpu, stru
return err;
err = do_boot_cpu(apicid, cpu, tidle);
- if (err) {
+ if (err)
pr_err("do_boot_cpu failed(%d) to wakeup CPU#%u\n", err, cpu);
- return err;
- }
- /*
- * Check TSC synchronization with the AP (keep irqs disabled
- * while doing so):
- */
- local_irq_save(flags);
- check_tsc_sync_source(cpu);
- local_irq_restore(flags);
+ return err;
+}
- while (!cpu_online(cpu)) {
- cpu_relax();
- touch_nmi_watchdog();
- }
+int native_cpu_up(unsigned int cpu, struct task_struct *tidle)
+{
+ int ret;
- return 0;
+ ret = native_kick_ap(cpu, tidle);
+ if (ret)
+ goto out;
+
+ ret = wait_cpu_initialized(cpu);
+ if (ret)
+ goto out;
+
+ wait_cpu_callin(cpu);
+ wait_cpu_online(cpu);
+
+out:
+ /* Cleanup possible dangling ends... */
+ if (x86_platform.legacy.warm_reset)
+ smpboot_restore_warm_reset_vector();
+
+ return ret;
}
/**
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 08/37] x86/smpboot: Split up native_cpu_up() into separate phases and document them
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
From: David Woodhouse <dwmw@amazon.co.uk>
There are four logical parts to what native_cpu_up() does on the BSP (or
on the controlling CPU for a later hotplug):
1) Wake the AP by sending the INIT/SIPI/SIPI sequence.
2) Wait for the AP to make it as far as wait_for_master_cpu() which
sets that CPU's bit in cpu_initialized_mask, then sets the bit in
cpu_callout_mask to let the AP proceed through cpu_init().
3) Wait for the AP to finish cpu_init() and get as far as the
smp_callin() call, which sets that CPU's bit in cpu_callin_mask.
4) Perform the TSC synchronization and wait for the AP to actually
mark itself online in cpu_online_mask.
In preparation to allow these phases to operate in parallel on multiple
APs, split them out into separate functions and document the interactions
a little more clearly in both the BP and AP code paths.
No functional change intended.
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: Usama Arif <usama.arif@bytedance.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/kernel/smpboot.c | 187 +++++++++++++++++++++++++++++-----------------
1 file changed, 121 insertions(+), 66 deletions(-)
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -193,6 +193,10 @@ static void smp_callin(void)
wmb();
+ /*
+ * This runs the AP through all the cpuhp states to its target
+ * state (CPUHP_ONLINE in the case of serial bringup).
+ */
notify_cpu_starting(cpuid);
/*
@@ -233,14 +237,31 @@ static void notrace start_secondary(void
load_cr3(swapper_pg_dir);
__flush_tlb_all();
#endif
+ /*
+ * Sync point with wait_cpu_initialized(). Before proceeding through
+ * cpu_init(), the AP will call wait_for_master_cpu() which sets its
+ * own bit in cpu_initialized_mask and then waits for the BSP to set
+ * its bit in cpu_callout_mask to release it.
+ */
cpu_init_secondary();
rcu_cpu_starting(raw_smp_processor_id());
x86_cpuinit.early_percpu_clock_init();
+
+ /*
+ * Sync point with wait_cpu_callin(). The AP doesn't wait here
+ * but just sets the bit to let the controlling CPU (BSP) know that
+ * it's got this far.
+ */
smp_callin();
- /* otherwise gcc will move up smp_processor_id before the cpu_init */
+ /* Otherwise gcc will move up smp_processor_id() before cpu_init() */
barrier();
- /* Check TSC synchronization with the control CPU: */
+
+ /*
+ * Check TSC synchronization with the control CPU, which will do
+ * its part of this from wait_cpu_online(), making it an implicit
+ * synchronization point.
+ */
check_tsc_sync_target();
/*
@@ -259,6 +280,7 @@ static void notrace start_secondary(void
* half valid vector space.
*/
lock_vector_lock();
+ /* Sync point with do_wait_cpu_online() */
set_cpu_online(smp_processor_id(), true);
lapic_online();
unlock_vector_lock();
@@ -981,17 +1003,13 @@ int common_cpu_up(unsigned int cpu, stru
/*
* NOTE - on most systems this is a PHYSICAL apic ID, but on multiquad
* (ie clustered apic addressing mode), this is a LOGICAL apic ID.
- * Returns zero if CPU booted OK, else error code from
+ * Returns zero if startup was successfully sent, else error code from
* ->wakeup_secondary_cpu.
*/
static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle)
{
- /* start_ip had better be page-aligned! */
unsigned long start_ip = real_mode_header->trampoline_start;
- unsigned long boot_error = 0;
- unsigned long timeout;
-
#ifdef CONFIG_X86_64
/* If 64-bit wakeup method exists, use the 64-bit mode trampoline IP */
if (apic->wakeup_secondary_cpu_64)
@@ -1048,60 +1066,89 @@ static int do_boot_cpu(int apicid, int c
* - Use an INIT boot APIC message
*/
if (apic->wakeup_secondary_cpu_64)
- boot_error = apic->wakeup_secondary_cpu_64(apicid, start_ip);
+ return apic->wakeup_secondary_cpu_64(apicid, start_ip);
else if (apic->wakeup_secondary_cpu)
- boot_error = apic->wakeup_secondary_cpu(apicid, start_ip);
- else
- boot_error = wakeup_secondary_cpu_via_init(apicid, start_ip);
+ return apic->wakeup_secondary_cpu(apicid, start_ip);
- if (!boot_error) {
- /*
- * Wait 10s total for first sign of life from AP
- */
- boot_error = -1;
- timeout = jiffies + 10*HZ;
- while (time_before(jiffies, timeout)) {
- if (cpumask_test_cpu(cpu, cpu_initialized_mask)) {
- /*
- * Tell AP to proceed with initialization
- */
- cpumask_set_cpu(cpu, cpu_callout_mask);
- boot_error = 0;
- break;
- }
- schedule();
- }
- }
+ return wakeup_secondary_cpu_via_init(apicid, start_ip);
+}
- if (!boot_error) {
- /*
- * Wait till AP completes initial initialization
- */
- while (!cpumask_test_cpu(cpu, cpu_callin_mask)) {
- /*
- * Allow other tasks to run while we wait for the
- * AP to come online. This also gives a chance
- * for the MTRR work(triggered by the AP coming online)
- * to be completed in the stop machine context.
- */
- schedule();
- }
- }
+static int wait_cpu_cpumask(unsigned int cpu, const struct cpumask *mask)
+{
+ unsigned long timeout;
- if (x86_platform.legacy.warm_reset) {
- /*
- * Cleanup possible dangling ends...
- */
- smpboot_restore_warm_reset_vector();
+ /*
+ * Wait up to 10s for the CPU to report in.
+ */
+ timeout = jiffies + 10*HZ;
+ while (time_before(jiffies, timeout)) {
+ if (cpumask_test_cpu(cpu, mask))
+ return 0;
+
+ schedule();
}
+ return -1;
+}
- return boot_error;
+/*
+ * Bringup step two: Wait for the target AP to reach cpu_init_secondary()
+ * and thus wait_for_master_cpu(), then set cpu_callout_mask to allow it
+ * to proceed. The AP will then proceed past setting its 'callin' bit
+ * and end up waiting in check_tsc_sync_target() until we reach
+ * do_wait_cpu_online() to tend to it.
+ */
+static int wait_cpu_initialized(unsigned int cpu)
+{
+ /*
+ * Wait for first sign of life from AP.
+ */
+ if (wait_cpu_cpumask(cpu, cpu_initialized_mask))
+ return -1;
+
+ cpumask_set_cpu(cpu, cpu_callout_mask);
+ return 0;
}
-int native_cpu_up(unsigned int cpu, struct task_struct *tidle)
+/*
+ * Bringup step three: Wait for the target AP to reach smp_callin().
+ * The AP is not waiting for us here so we don't need to parallelise
+ * this step. Not entirely clear why we care about this, since we just
+ * proceed directly to TSC synchronization which is the next sync
+ * point with the AP anyway.
+ */
+static void wait_cpu_callin(unsigned int cpu)
+{
+ while (!cpumask_test_cpu(cpu, cpu_callin_mask))
+ schedule();
+}
+
+/*
+ * Bringup step four: Synchronize the TSC and wait for the target AP
+ * to reach set_cpu_online() in start_secondary().
+ */
+static void wait_cpu_online(unsigned int cpu)
{
- int apicid = apic->cpu_present_to_apicid(cpu);
unsigned long flags;
+
+ /*
+ * Check TSC synchronization with the AP (keep irqs disabled
+ * while doing so):
+ */
+ local_irq_save(flags);
+ check_tsc_sync_source(cpu);
+ local_irq_restore(flags);
+
+ /*
+ * Wait for the AP to mark itself online, so the core caller
+ * can drop sparse_irq_lock.
+ */
+ while (!cpu_online(cpu))
+ schedule();
+}
+
+static int native_kick_ap(unsigned int cpu, struct task_struct *tidle)
+{
+ int apicid = apic->cpu_present_to_apicid(cpu);
int err;
lockdep_assert_irqs_enabled();
@@ -1142,25 +1189,33 @@ int native_cpu_up(unsigned int cpu, stru
return err;
err = do_boot_cpu(apicid, cpu, tidle);
- if (err) {
+ if (err)
pr_err("do_boot_cpu failed(%d) to wakeup CPU#%u\n", err, cpu);
- return err;
- }
- /*
- * Check TSC synchronization with the AP (keep irqs disabled
- * while doing so):
- */
- local_irq_save(flags);
- check_tsc_sync_source(cpu);
- local_irq_restore(flags);
+ return err;
+}
- while (!cpu_online(cpu)) {
- cpu_relax();
- touch_nmi_watchdog();
- }
+int native_cpu_up(unsigned int cpu, struct task_struct *tidle)
+{
+ int ret;
- return 0;
+ ret = native_kick_ap(cpu, tidle);
+ if (ret)
+ goto out;
+
+ ret = wait_cpu_initialized(cpu);
+ if (ret)
+ goto out;
+
+ wait_cpu_callin(cpu);
+ wait_cpu_online(cpu);
+
+out:
+ /* Cleanup possible dangling ends... */
+ if (x86_platform.legacy.warm_reset)
+ smpboot_restore_warm_reset_vector();
+
+ return ret;
}
/**
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 08/37] x86/smpboot: Split up native_cpu_up() into separate phases and document them
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
From: David Woodhouse <dwmw@amazon.co.uk>
There are four logical parts to what native_cpu_up() does on the BSP (or
on the controlling CPU for a later hotplug):
1) Wake the AP by sending the INIT/SIPI/SIPI sequence.
2) Wait for the AP to make it as far as wait_for_master_cpu() which
sets that CPU's bit in cpu_initialized_mask, then sets the bit in
cpu_callout_mask to let the AP proceed through cpu_init().
3) Wait for the AP to finish cpu_init() and get as far as the
smp_callin() call, which sets that CPU's bit in cpu_callin_mask.
4) Perform the TSC synchronization and wait for the AP to actually
mark itself online in cpu_online_mask.
In preparation to allow these phases to operate in parallel on multiple
APs, split them out into separate functions and document the interactions
a little more clearly in both the BP and AP code paths.
No functional change intended.
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: Usama Arif <usama.arif@bytedance.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/kernel/smpboot.c | 187 +++++++++++++++++++++++++++++-----------------
1 file changed, 121 insertions(+), 66 deletions(-)
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -193,6 +193,10 @@ static void smp_callin(void)
wmb();
+ /*
+ * This runs the AP through all the cpuhp states to its target
+ * state (CPUHP_ONLINE in the case of serial bringup).
+ */
notify_cpu_starting(cpuid);
/*
@@ -233,14 +237,31 @@ static void notrace start_secondary(void
load_cr3(swapper_pg_dir);
__flush_tlb_all();
#endif
+ /*
+ * Sync point with wait_cpu_initialized(). Before proceeding through
+ * cpu_init(), the AP will call wait_for_master_cpu() which sets its
+ * own bit in cpu_initialized_mask and then waits for the BSP to set
+ * its bit in cpu_callout_mask to release it.
+ */
cpu_init_secondary();
rcu_cpu_starting(raw_smp_processor_id());
x86_cpuinit.early_percpu_clock_init();
+
+ /*
+ * Sync point with wait_cpu_callin(). The AP doesn't wait here
+ * but just sets the bit to let the controlling CPU (BSP) know that
+ * it's got this far.
+ */
smp_callin();
- /* otherwise gcc will move up smp_processor_id before the cpu_init */
+ /* Otherwise gcc will move up smp_processor_id() before cpu_init() */
barrier();
- /* Check TSC synchronization with the control CPU: */
+
+ /*
+ * Check TSC synchronization with the control CPU, which will do
+ * its part of this from wait_cpu_online(), making it an implicit
+ * synchronization point.
+ */
check_tsc_sync_target();
/*
@@ -259,6 +280,7 @@ static void notrace start_secondary(void
* half valid vector space.
*/
lock_vector_lock();
+ /* Sync point with do_wait_cpu_online() */
set_cpu_online(smp_processor_id(), true);
lapic_online();
unlock_vector_lock();
@@ -981,17 +1003,13 @@ int common_cpu_up(unsigned int cpu, stru
/*
* NOTE - on most systems this is a PHYSICAL apic ID, but on multiquad
* (ie clustered apic addressing mode), this is a LOGICAL apic ID.
- * Returns zero if CPU booted OK, else error code from
+ * Returns zero if startup was successfully sent, else error code from
* ->wakeup_secondary_cpu.
*/
static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle)
{
- /* start_ip had better be page-aligned! */
unsigned long start_ip = real_mode_header->trampoline_start;
- unsigned long boot_error = 0;
- unsigned long timeout;
-
#ifdef CONFIG_X86_64
/* If 64-bit wakeup method exists, use the 64-bit mode trampoline IP */
if (apic->wakeup_secondary_cpu_64)
@@ -1048,60 +1066,89 @@ static int do_boot_cpu(int apicid, int c
* - Use an INIT boot APIC message
*/
if (apic->wakeup_secondary_cpu_64)
- boot_error = apic->wakeup_secondary_cpu_64(apicid, start_ip);
+ return apic->wakeup_secondary_cpu_64(apicid, start_ip);
else if (apic->wakeup_secondary_cpu)
- boot_error = apic->wakeup_secondary_cpu(apicid, start_ip);
- else
- boot_error = wakeup_secondary_cpu_via_init(apicid, start_ip);
+ return apic->wakeup_secondary_cpu(apicid, start_ip);
- if (!boot_error) {
- /*
- * Wait 10s total for first sign of life from AP
- */
- boot_error = -1;
- timeout = jiffies + 10*HZ;
- while (time_before(jiffies, timeout)) {
- if (cpumask_test_cpu(cpu, cpu_initialized_mask)) {
- /*
- * Tell AP to proceed with initialization
- */
- cpumask_set_cpu(cpu, cpu_callout_mask);
- boot_error = 0;
- break;
- }
- schedule();
- }
- }
+ return wakeup_secondary_cpu_via_init(apicid, start_ip);
+}
- if (!boot_error) {
- /*
- * Wait till AP completes initial initialization
- */
- while (!cpumask_test_cpu(cpu, cpu_callin_mask)) {
- /*
- * Allow other tasks to run while we wait for the
- * AP to come online. This also gives a chance
- * for the MTRR work(triggered by the AP coming online)
- * to be completed in the stop machine context.
- */
- schedule();
- }
- }
+static int wait_cpu_cpumask(unsigned int cpu, const struct cpumask *mask)
+{
+ unsigned long timeout;
- if (x86_platform.legacy.warm_reset) {
- /*
- * Cleanup possible dangling ends...
- */
- smpboot_restore_warm_reset_vector();
+ /*
+ * Wait up to 10s for the CPU to report in.
+ */
+ timeout = jiffies + 10*HZ;
+ while (time_before(jiffies, timeout)) {
+ if (cpumask_test_cpu(cpu, mask))
+ return 0;
+
+ schedule();
}
+ return -1;
+}
- return boot_error;
+/*
+ * Bringup step two: Wait for the target AP to reach cpu_init_secondary()
+ * and thus wait_for_master_cpu(), then set cpu_callout_mask to allow it
+ * to proceed. The AP will then proceed past setting its 'callin' bit
+ * and end up waiting in check_tsc_sync_target() until we reach
+ * do_wait_cpu_online() to tend to it.
+ */
+static int wait_cpu_initialized(unsigned int cpu)
+{
+ /*
+ * Wait for first sign of life from AP.
+ */
+ if (wait_cpu_cpumask(cpu, cpu_initialized_mask))
+ return -1;
+
+ cpumask_set_cpu(cpu, cpu_callout_mask);
+ return 0;
}
-int native_cpu_up(unsigned int cpu, struct task_struct *tidle)
+/*
+ * Bringup step three: Wait for the target AP to reach smp_callin().
+ * The AP is not waiting for us here so we don't need to parallelise
+ * this step. Not entirely clear why we care about this, since we just
+ * proceed directly to TSC synchronization which is the next sync
+ * point with the AP anyway.
+ */
+static void wait_cpu_callin(unsigned int cpu)
+{
+ while (!cpumask_test_cpu(cpu, cpu_callin_mask))
+ schedule();
+}
+
+/*
+ * Bringup step four: Synchronize the TSC and wait for the target AP
+ * to reach set_cpu_online() in start_secondary().
+ */
+static void wait_cpu_online(unsigned int cpu)
{
- int apicid = apic->cpu_present_to_apicid(cpu);
unsigned long flags;
+
+ /*
+ * Check TSC synchronization with the AP (keep irqs disabled
+ * while doing so):
+ */
+ local_irq_save(flags);
+ check_tsc_sync_source(cpu);
+ local_irq_restore(flags);
+
+ /*
+ * Wait for the AP to mark itself online, so the core caller
+ * can drop sparse_irq_lock.
+ */
+ while (!cpu_online(cpu))
+ schedule();
+}
+
+static int native_kick_ap(unsigned int cpu, struct task_struct *tidle)
+{
+ int apicid = apic->cpu_present_to_apicid(cpu);
int err;
lockdep_assert_irqs_enabled();
@@ -1142,25 +1189,33 @@ int native_cpu_up(unsigned int cpu, stru
return err;
err = do_boot_cpu(apicid, cpu, tidle);
- if (err) {
+ if (err)
pr_err("do_boot_cpu failed(%d) to wakeup CPU#%u\n", err, cpu);
- return err;
- }
- /*
- * Check TSC synchronization with the AP (keep irqs disabled
- * while doing so):
- */
- local_irq_save(flags);
- check_tsc_sync_source(cpu);
- local_irq_restore(flags);
+ return err;
+}
- while (!cpu_online(cpu)) {
- cpu_relax();
- touch_nmi_watchdog();
- }
+int native_cpu_up(unsigned int cpu, struct task_struct *tidle)
+{
+ int ret;
- return 0;
+ ret = native_kick_ap(cpu, tidle);
+ if (ret)
+ goto out;
+
+ ret = wait_cpu_initialized(cpu);
+ if (ret)
+ goto out;
+
+ wait_cpu_callin(cpu);
+ wait_cpu_online(cpu);
+
+out:
+ /* Cleanup possible dangling ends... */
+ if (x86_platform.legacy.warm_reset)
+ smpboot_restore_warm_reset_vector();
+
+ return ret;
}
/**
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 09/37] x86/smpboot: Get rid of cpu_init_secondary()
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-14 23:44 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
The synchronization of the AP with the control CPU is a SMP boot problem
and has nothing to do with cpu_init().
Open code cpu_init_secondary() in start_secondary() and move
wait_for_master_cpu() into the SMP boot code.
No functional change.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/include/asm/processor.h | 1 -
arch/x86/kernel/cpu/common.c | 27 ---------------------------
arch/x86/kernel/smpboot.c | 24 +++++++++++++++++++-----
3 files changed, 19 insertions(+), 33 deletions(-)
--- a/arch/x86/include/asm/processor.h
+++ b/arch/x86/include/asm/processor.h
@@ -551,7 +551,6 @@ extern void switch_gdt_and_percpu_base(i
extern void load_direct_gdt(int);
extern void load_fixmap_gdt(int);
extern void cpu_init(void);
-extern void cpu_init_secondary(void);
extern void cpu_init_exception_handling(void);
extern void cr4_init(void);
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -2122,19 +2122,6 @@ static void dbg_restore_debug_regs(void)
#define dbg_restore_debug_regs()
#endif /* ! CONFIG_KGDB */
-static void wait_for_master_cpu(int cpu)
-{
-#ifdef CONFIG_SMP
- /*
- * wait for ACK from master CPU before continuing
- * with AP initialization
- */
- WARN_ON(cpumask_test_and_set_cpu(cpu, cpu_initialized_mask));
- while (!cpumask_test_cpu(cpu, cpu_callout_mask))
- cpu_relax();
-#endif
-}
-
static inline void setup_getcpu(int cpu)
{
unsigned long cpudata = vdso_encode_cpunode(cpu, early_cpu_to_node(cpu));
@@ -2238,8 +2225,6 @@ void cpu_init(void)
struct task_struct *cur = current;
int cpu = raw_smp_processor_id();
- wait_for_master_cpu(cpu);
-
ucode_cpu_init(cpu);
#ifdef CONFIG_NUMA
@@ -2292,18 +2277,6 @@ void cpu_init(void)
load_fixmap_gdt(cpu);
}
-#ifdef CONFIG_SMP
-void cpu_init_secondary(void)
-{
- /*
- * Relies on the BP having set-up the IDT tables, which are loaded
- * on this CPU in cpu_init_exception_handling().
- */
- cpu_init_exception_handling();
- cpu_init();
-}
-#endif
-
#ifdef CONFIG_MICROCODE_LATE_LOADING
/**
* store_cpu_caps() - Store a snapshot of CPU capabilities
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -220,6 +220,17 @@ static void ap_calibrate_delay(void)
cpu_data(smp_processor_id()).loops_per_jiffy = loops_per_jiffy;
}
+static void wait_for_master_cpu(int cpu)
+{
+ /*
+ * Wait for release by control CPU before continuing with AP
+ * initialization.
+ */
+ WARN_ON(cpumask_test_and_set_cpu(cpu, cpu_initialized_mask));
+ while (!cpumask_test_cpu(cpu, cpu_callout_mask))
+ cpu_relax();
+}
+
/*
* Activate a secondary processor.
*/
@@ -237,13 +248,16 @@ static void notrace start_secondary(void
load_cr3(swapper_pg_dir);
__flush_tlb_all();
#endif
+ cpu_init_exception_handling();
+
/*
- * Sync point with wait_cpu_initialized(). Before proceeding through
- * cpu_init(), the AP will call wait_for_master_cpu() which sets its
- * own bit in cpu_initialized_mask and then waits for the BSP to set
- * its bit in cpu_callout_mask to release it.
+ * Sync point with wait_cpu_initialized(). Sets AP in
+ * cpu_initialized_mask and then waits for the control CPU
+ * to release it.
*/
- cpu_init_secondary();
+ wait_for_master_cpu(raw_smp_processor_id());
+
+ cpu_init();
rcu_cpu_starting(raw_smp_processor_id());
x86_cpuinit.early_percpu_clock_init();
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 09/37] x86/smpboot: Get rid of cpu_init_secondary()
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
The synchronization of the AP with the control CPU is a SMP boot problem
and has nothing to do with cpu_init().
Open code cpu_init_secondary() in start_secondary() and move
wait_for_master_cpu() into the SMP boot code.
No functional change.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/include/asm/processor.h | 1 -
arch/x86/kernel/cpu/common.c | 27 ---------------------------
arch/x86/kernel/smpboot.c | 24 +++++++++++++++++++-----
3 files changed, 19 insertions(+), 33 deletions(-)
--- a/arch/x86/include/asm/processor.h
+++ b/arch/x86/include/asm/processor.h
@@ -551,7 +551,6 @@ extern void switch_gdt_and_percpu_base(i
extern void load_direct_gdt(int);
extern void load_fixmap_gdt(int);
extern void cpu_init(void);
-extern void cpu_init_secondary(void);
extern void cpu_init_exception_handling(void);
extern void cr4_init(void);
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -2122,19 +2122,6 @@ static void dbg_restore_debug_regs(void)
#define dbg_restore_debug_regs()
#endif /* ! CONFIG_KGDB */
-static void wait_for_master_cpu(int cpu)
-{
-#ifdef CONFIG_SMP
- /*
- * wait for ACK from master CPU before continuing
- * with AP initialization
- */
- WARN_ON(cpumask_test_and_set_cpu(cpu, cpu_initialized_mask));
- while (!cpumask_test_cpu(cpu, cpu_callout_mask))
- cpu_relax();
-#endif
-}
-
static inline void setup_getcpu(int cpu)
{
unsigned long cpudata = vdso_encode_cpunode(cpu, early_cpu_to_node(cpu));
@@ -2238,8 +2225,6 @@ void cpu_init(void)
struct task_struct *cur = current;
int cpu = raw_smp_processor_id();
- wait_for_master_cpu(cpu);
-
ucode_cpu_init(cpu);
#ifdef CONFIG_NUMA
@@ -2292,18 +2277,6 @@ void cpu_init(void)
load_fixmap_gdt(cpu);
}
-#ifdef CONFIG_SMP
-void cpu_init_secondary(void)
-{
- /*
- * Relies on the BP having set-up the IDT tables, which are loaded
- * on this CPU in cpu_init_exception_handling().
- */
- cpu_init_exception_handling();
- cpu_init();
-}
-#endif
-
#ifdef CONFIG_MICROCODE_LATE_LOADING
/**
* store_cpu_caps() - Store a snapshot of CPU capabilities
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -220,6 +220,17 @@ static void ap_calibrate_delay(void)
cpu_data(smp_processor_id()).loops_per_jiffy = loops_per_jiffy;
}
+static void wait_for_master_cpu(int cpu)
+{
+ /*
+ * Wait for release by control CPU before continuing with AP
+ * initialization.
+ */
+ WARN_ON(cpumask_test_and_set_cpu(cpu, cpu_initialized_mask));
+ while (!cpumask_test_cpu(cpu, cpu_callout_mask))
+ cpu_relax();
+}
+
/*
* Activate a secondary processor.
*/
@@ -237,13 +248,16 @@ static void notrace start_secondary(void
load_cr3(swapper_pg_dir);
__flush_tlb_all();
#endif
+ cpu_init_exception_handling();
+
/*
- * Sync point with wait_cpu_initialized(). Before proceeding through
- * cpu_init(), the AP will call wait_for_master_cpu() which sets its
- * own bit in cpu_initialized_mask and then waits for the BSP to set
- * its bit in cpu_callout_mask to release it.
+ * Sync point with wait_cpu_initialized(). Sets AP in
+ * cpu_initialized_mask and then waits for the control CPU
+ * to release it.
*/
- cpu_init_secondary();
+ wait_for_master_cpu(raw_smp_processor_id());
+
+ cpu_init();
rcu_cpu_starting(raw_smp_processor_id());
x86_cpuinit.early_percpu_clock_init();
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 09/37] x86/smpboot: Get rid of cpu_init_secondary()
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
The synchronization of the AP with the control CPU is a SMP boot problem
and has nothing to do with cpu_init().
Open code cpu_init_secondary() in start_secondary() and move
wait_for_master_cpu() into the SMP boot code.
No functional change.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/include/asm/processor.h | 1 -
arch/x86/kernel/cpu/common.c | 27 ---------------------------
arch/x86/kernel/smpboot.c | 24 +++++++++++++++++++-----
3 files changed, 19 insertions(+), 33 deletions(-)
--- a/arch/x86/include/asm/processor.h
+++ b/arch/x86/include/asm/processor.h
@@ -551,7 +551,6 @@ extern void switch_gdt_and_percpu_base(i
extern void load_direct_gdt(int);
extern void load_fixmap_gdt(int);
extern void cpu_init(void);
-extern void cpu_init_secondary(void);
extern void cpu_init_exception_handling(void);
extern void cr4_init(void);
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -2122,19 +2122,6 @@ static void dbg_restore_debug_regs(void)
#define dbg_restore_debug_regs()
#endif /* ! CONFIG_KGDB */
-static void wait_for_master_cpu(int cpu)
-{
-#ifdef CONFIG_SMP
- /*
- * wait for ACK from master CPU before continuing
- * with AP initialization
- */
- WARN_ON(cpumask_test_and_set_cpu(cpu, cpu_initialized_mask));
- while (!cpumask_test_cpu(cpu, cpu_callout_mask))
- cpu_relax();
-#endif
-}
-
static inline void setup_getcpu(int cpu)
{
unsigned long cpudata = vdso_encode_cpunode(cpu, early_cpu_to_node(cpu));
@@ -2238,8 +2225,6 @@ void cpu_init(void)
struct task_struct *cur = current;
int cpu = raw_smp_processor_id();
- wait_for_master_cpu(cpu);
-
ucode_cpu_init(cpu);
#ifdef CONFIG_NUMA
@@ -2292,18 +2277,6 @@ void cpu_init(void)
load_fixmap_gdt(cpu);
}
-#ifdef CONFIG_SMP
-void cpu_init_secondary(void)
-{
- /*
- * Relies on the BP having set-up the IDT tables, which are loaded
- * on this CPU in cpu_init_exception_handling().
- */
- cpu_init_exception_handling();
- cpu_init();
-}
-#endif
-
#ifdef CONFIG_MICROCODE_LATE_LOADING
/**
* store_cpu_caps() - Store a snapshot of CPU capabilities
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -220,6 +220,17 @@ static void ap_calibrate_delay(void)
cpu_data(smp_processor_id()).loops_per_jiffy = loops_per_jiffy;
}
+static void wait_for_master_cpu(int cpu)
+{
+ /*
+ * Wait for release by control CPU before continuing with AP
+ * initialization.
+ */
+ WARN_ON(cpumask_test_and_set_cpu(cpu, cpu_initialized_mask));
+ while (!cpumask_test_cpu(cpu, cpu_callout_mask))
+ cpu_relax();
+}
+
/*
* Activate a secondary processor.
*/
@@ -237,13 +248,16 @@ static void notrace start_secondary(void
load_cr3(swapper_pg_dir);
__flush_tlb_all();
#endif
+ cpu_init_exception_handling();
+
/*
- * Sync point with wait_cpu_initialized(). Before proceeding through
- * cpu_init(), the AP will call wait_for_master_cpu() which sets its
- * own bit in cpu_initialized_mask and then waits for the BSP to set
- * its bit in cpu_callout_mask to release it.
+ * Sync point with wait_cpu_initialized(). Sets AP in
+ * cpu_initialized_mask and then waits for the control CPU
+ * to release it.
*/
- cpu_init_secondary();
+ wait_for_master_cpu(raw_smp_processor_id());
+
+ cpu_init();
rcu_cpu_starting(raw_smp_processor_id());
x86_cpuinit.early_percpu_clock_init();
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 10/37] x86/cpu/cacheinfo: Remove cpu_callout_mask dependency
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-14 23:44 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
cpu_callout_mask is used for the stop machine based MTRR/PAT init.
In preparation of moving the BP/AP synchronization to the core hotplug
code, use a private CPU mask for cacheinfo and manage it in the
starting/dying hotplug state.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/kernel/cpu/cacheinfo.c | 21 +++++++++++++++++----
1 file changed, 17 insertions(+), 4 deletions(-)
--- a/arch/x86/kernel/cpu/cacheinfo.c
+++ b/arch/x86/kernel/cpu/cacheinfo.c
@@ -39,6 +39,8 @@ DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t
/* Shared L2 cache maps */
DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_l2c_shared_map);
+static cpumask_var_t cpu_cacheinfo_mask;
+
/* Kernel controls MTRR and/or PAT MSRs. */
unsigned int memory_caching_control __ro_after_init;
@@ -1172,8 +1174,10 @@ void cache_bp_restore(void)
cache_cpu_init();
}
-static int cache_ap_init(unsigned int cpu)
+static int cache_ap_online(unsigned int cpu)
{
+ cpumask_set_cpu(cpu, cpu_cacheinfo_mask);
+
if (!memory_caching_control || get_cache_aps_delayed_init())
return 0;
@@ -1191,11 +1195,17 @@ static int cache_ap_init(unsigned int cp
* lock to prevent MTRR entry changes
*/
stop_machine_from_inactive_cpu(cache_rendezvous_handler, NULL,
- cpu_callout_mask);
+ cpu_cacheinfo_mask);
return 0;
}
+static int cache_ap_offline(unsigned int cpu)
+{
+ cpumask_clear_cpu(cpu, cpu_cacheinfo_mask);
+ return 0;
+}
+
/*
* Delayed cache initialization for all AP's
*/
@@ -1210,9 +1220,12 @@ void cache_aps_init(void)
static int __init cache_ap_register(void)
{
+ zalloc_cpumask_var(&cpu_cacheinfo_mask, GFP_KERNEL);
+ cpumask_set_cpu(smp_processor_id(), cpu_cacheinfo_mask);
+
cpuhp_setup_state_nocalls(CPUHP_AP_CACHECTRL_STARTING,
"x86/cachectrl:starting",
- cache_ap_init, NULL);
+ cache_ap_online, cache_ap_offline);
return 0;
}
-core_initcall(cache_ap_register);
+early_initcall(cache_ap_register);
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 10/37] x86/cpu/cacheinfo: Remove cpu_callout_mask dependency
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
cpu_callout_mask is used for the stop machine based MTRR/PAT init.
In preparation of moving the BP/AP synchronization to the core hotplug
code, use a private CPU mask for cacheinfo and manage it in the
starting/dying hotplug state.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/kernel/cpu/cacheinfo.c | 21 +++++++++++++++++----
1 file changed, 17 insertions(+), 4 deletions(-)
--- a/arch/x86/kernel/cpu/cacheinfo.c
+++ b/arch/x86/kernel/cpu/cacheinfo.c
@@ -39,6 +39,8 @@ DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t
/* Shared L2 cache maps */
DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_l2c_shared_map);
+static cpumask_var_t cpu_cacheinfo_mask;
+
/* Kernel controls MTRR and/or PAT MSRs. */
unsigned int memory_caching_control __ro_after_init;
@@ -1172,8 +1174,10 @@ void cache_bp_restore(void)
cache_cpu_init();
}
-static int cache_ap_init(unsigned int cpu)
+static int cache_ap_online(unsigned int cpu)
{
+ cpumask_set_cpu(cpu, cpu_cacheinfo_mask);
+
if (!memory_caching_control || get_cache_aps_delayed_init())
return 0;
@@ -1191,11 +1195,17 @@ static int cache_ap_init(unsigned int cp
* lock to prevent MTRR entry changes
*/
stop_machine_from_inactive_cpu(cache_rendezvous_handler, NULL,
- cpu_callout_mask);
+ cpu_cacheinfo_mask);
return 0;
}
+static int cache_ap_offline(unsigned int cpu)
+{
+ cpumask_clear_cpu(cpu, cpu_cacheinfo_mask);
+ return 0;
+}
+
/*
* Delayed cache initialization for all AP's
*/
@@ -1210,9 +1220,12 @@ void cache_aps_init(void)
static int __init cache_ap_register(void)
{
+ zalloc_cpumask_var(&cpu_cacheinfo_mask, GFP_KERNEL);
+ cpumask_set_cpu(smp_processor_id(), cpu_cacheinfo_mask);
+
cpuhp_setup_state_nocalls(CPUHP_AP_CACHECTRL_STARTING,
"x86/cachectrl:starting",
- cache_ap_init, NULL);
+ cache_ap_online, cache_ap_offline);
return 0;
}
-core_initcall(cache_ap_register);
+early_initcall(cache_ap_register);
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 10/37] x86/cpu/cacheinfo: Remove cpu_callout_mask dependency
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
cpu_callout_mask is used for the stop machine based MTRR/PAT init.
In preparation of moving the BP/AP synchronization to the core hotplug
code, use a private CPU mask for cacheinfo and manage it in the
starting/dying hotplug state.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/kernel/cpu/cacheinfo.c | 21 +++++++++++++++++----
1 file changed, 17 insertions(+), 4 deletions(-)
--- a/arch/x86/kernel/cpu/cacheinfo.c
+++ b/arch/x86/kernel/cpu/cacheinfo.c
@@ -39,6 +39,8 @@ DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t
/* Shared L2 cache maps */
DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_l2c_shared_map);
+static cpumask_var_t cpu_cacheinfo_mask;
+
/* Kernel controls MTRR and/or PAT MSRs. */
unsigned int memory_caching_control __ro_after_init;
@@ -1172,8 +1174,10 @@ void cache_bp_restore(void)
cache_cpu_init();
}
-static int cache_ap_init(unsigned int cpu)
+static int cache_ap_online(unsigned int cpu)
{
+ cpumask_set_cpu(cpu, cpu_cacheinfo_mask);
+
if (!memory_caching_control || get_cache_aps_delayed_init())
return 0;
@@ -1191,11 +1195,17 @@ static int cache_ap_init(unsigned int cp
* lock to prevent MTRR entry changes
*/
stop_machine_from_inactive_cpu(cache_rendezvous_handler, NULL,
- cpu_callout_mask);
+ cpu_cacheinfo_mask);
return 0;
}
+static int cache_ap_offline(unsigned int cpu)
+{
+ cpumask_clear_cpu(cpu, cpu_cacheinfo_mask);
+ return 0;
+}
+
/*
* Delayed cache initialization for all AP's
*/
@@ -1210,9 +1220,12 @@ void cache_aps_init(void)
static int __init cache_ap_register(void)
{
+ zalloc_cpumask_var(&cpu_cacheinfo_mask, GFP_KERNEL);
+ cpumask_set_cpu(smp_processor_id(), cpu_cacheinfo_mask);
+
cpuhp_setup_state_nocalls(CPUHP_AP_CACHECTRL_STARTING,
"x86/cachectrl:starting",
- cache_ap_init, NULL);
+ cache_ap_online, cache_ap_offline);
return 0;
}
-core_initcall(cache_ap_register);
+early_initcall(cache_ap_register);
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 11/37] x86/smpboot: Move synchronization masks to SMP boot code
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-14 23:44 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
The usage is in smpboot.c and not in the CPU initialization code.
The XEN_PV usage of cpu_callout_mask is obsolete as cpu_init() not longer
waits and cacheinfo has its own CPU mask now, so cpu_callout_mask can be
made static too.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/include/asm/cpumask.h | 5 -----
arch/x86/kernel/cpu/common.c | 17 -----------------
arch/x86/kernel/smpboot.c | 16 ++++++++++++++++
arch/x86/xen/smp_pv.c | 3 ---
4 files changed, 16 insertions(+), 25 deletions(-)
--- a/arch/x86/include/asm/cpumask.h
+++ b/arch/x86/include/asm/cpumask.h
@@ -4,11 +4,6 @@
#ifndef __ASSEMBLY__
#include <linux/cpumask.h>
-extern cpumask_var_t cpu_callin_mask;
-extern cpumask_var_t cpu_callout_mask;
-extern cpumask_var_t cpu_initialized_mask;
-extern cpumask_var_t cpu_sibling_setup_mask;
-
extern void setup_cpu_local_masks(void);
/*
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -67,14 +67,6 @@
u32 elf_hwcap2 __read_mostly;
-/* all of these masks are initialized in setup_cpu_local_masks() */
-cpumask_var_t cpu_initialized_mask;
-cpumask_var_t cpu_callout_mask;
-cpumask_var_t cpu_callin_mask;
-
-/* representing cpus for which sibling maps can be computed */
-cpumask_var_t cpu_sibling_setup_mask;
-
/* Number of siblings per CPU package */
int smp_num_siblings = 1;
EXPORT_SYMBOL(smp_num_siblings);
@@ -168,15 +160,6 @@ static void ppin_init(struct cpuinfo_x86
clear_cpu_cap(c, info->feature);
}
-/* correctly size the local cpu masks */
-void __init setup_cpu_local_masks(void)
-{
- alloc_bootmem_cpumask_var(&cpu_initialized_mask);
- alloc_bootmem_cpumask_var(&cpu_callin_mask);
- alloc_bootmem_cpumask_var(&cpu_callout_mask);
- alloc_bootmem_cpumask_var(&cpu_sibling_setup_mask);
-}
-
static void default_init(struct cpuinfo_x86 *c)
{
#ifdef CONFIG_X86_64
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -101,6 +101,13 @@ EXPORT_PER_CPU_SYMBOL(cpu_die_map);
DEFINE_PER_CPU_READ_MOSTLY(struct cpuinfo_x86, cpu_info);
EXPORT_PER_CPU_SYMBOL(cpu_info);
+/* All of these masks are initialized in setup_cpu_local_masks() */
+static cpumask_var_t cpu_initialized_mask;
+static cpumask_var_t cpu_callout_mask;
+static cpumask_var_t cpu_callin_mask;
+/* Representing CPUs for which sibling maps can be computed */
+static cpumask_var_t cpu_sibling_setup_mask;
+
/* Logical package management. We might want to allocate that dynamically */
unsigned int __max_logical_packages __read_mostly;
EXPORT_SYMBOL(__max_logical_packages);
@@ -1548,6 +1555,15 @@ early_param("possible_cpus", _setup_poss
set_cpu_possible(i, true);
}
+/* correctly size the local cpu masks */
+void __init setup_cpu_local_masks(void)
+{
+ alloc_bootmem_cpumask_var(&cpu_initialized_mask);
+ alloc_bootmem_cpumask_var(&cpu_callin_mask);
+ alloc_bootmem_cpumask_var(&cpu_callout_mask);
+ alloc_bootmem_cpumask_var(&cpu_sibling_setup_mask);
+}
+
#ifdef CONFIG_HOTPLUG_CPU
/* Recompute SMT state for all CPUs on offline */
--- a/arch/x86/xen/smp_pv.c
+++ b/arch/x86/xen/smp_pv.c
@@ -254,15 +254,12 @@ cpu_initialize_context(unsigned int cpu,
struct desc_struct *gdt;
unsigned long gdt_mfn;
- /* used to tell cpu_init() that it can proceed with initialization */
- cpumask_set_cpu(cpu, cpu_callout_mask);
if (cpumask_test_and_set_cpu(cpu, xen_cpu_initialized_map))
return 0;
ctxt = kzalloc(sizeof(*ctxt), GFP_KERNEL);
if (ctxt == NULL) {
cpumask_clear_cpu(cpu, xen_cpu_initialized_map);
- cpumask_clear_cpu(cpu, cpu_callout_mask);
return -ENOMEM;
}
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 11/37] x86/smpboot: Move synchronization masks to SMP boot code
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
The usage is in smpboot.c and not in the CPU initialization code.
The XEN_PV usage of cpu_callout_mask is obsolete as cpu_init() not longer
waits and cacheinfo has its own CPU mask now, so cpu_callout_mask can be
made static too.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/include/asm/cpumask.h | 5 -----
arch/x86/kernel/cpu/common.c | 17 -----------------
arch/x86/kernel/smpboot.c | 16 ++++++++++++++++
arch/x86/xen/smp_pv.c | 3 ---
4 files changed, 16 insertions(+), 25 deletions(-)
--- a/arch/x86/include/asm/cpumask.h
+++ b/arch/x86/include/asm/cpumask.h
@@ -4,11 +4,6 @@
#ifndef __ASSEMBLY__
#include <linux/cpumask.h>
-extern cpumask_var_t cpu_callin_mask;
-extern cpumask_var_t cpu_callout_mask;
-extern cpumask_var_t cpu_initialized_mask;
-extern cpumask_var_t cpu_sibling_setup_mask;
-
extern void setup_cpu_local_masks(void);
/*
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -67,14 +67,6 @@
u32 elf_hwcap2 __read_mostly;
-/* all of these masks are initialized in setup_cpu_local_masks() */
-cpumask_var_t cpu_initialized_mask;
-cpumask_var_t cpu_callout_mask;
-cpumask_var_t cpu_callin_mask;
-
-/* representing cpus for which sibling maps can be computed */
-cpumask_var_t cpu_sibling_setup_mask;
-
/* Number of siblings per CPU package */
int smp_num_siblings = 1;
EXPORT_SYMBOL(smp_num_siblings);
@@ -168,15 +160,6 @@ static void ppin_init(struct cpuinfo_x86
clear_cpu_cap(c, info->feature);
}
-/* correctly size the local cpu masks */
-void __init setup_cpu_local_masks(void)
-{
- alloc_bootmem_cpumask_var(&cpu_initialized_mask);
- alloc_bootmem_cpumask_var(&cpu_callin_mask);
- alloc_bootmem_cpumask_var(&cpu_callout_mask);
- alloc_bootmem_cpumask_var(&cpu_sibling_setup_mask);
-}
-
static void default_init(struct cpuinfo_x86 *c)
{
#ifdef CONFIG_X86_64
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -101,6 +101,13 @@ EXPORT_PER_CPU_SYMBOL(cpu_die_map);
DEFINE_PER_CPU_READ_MOSTLY(struct cpuinfo_x86, cpu_info);
EXPORT_PER_CPU_SYMBOL(cpu_info);
+/* All of these masks are initialized in setup_cpu_local_masks() */
+static cpumask_var_t cpu_initialized_mask;
+static cpumask_var_t cpu_callout_mask;
+static cpumask_var_t cpu_callin_mask;
+/* Representing CPUs for which sibling maps can be computed */
+static cpumask_var_t cpu_sibling_setup_mask;
+
/* Logical package management. We might want to allocate that dynamically */
unsigned int __max_logical_packages __read_mostly;
EXPORT_SYMBOL(__max_logical_packages);
@@ -1548,6 +1555,15 @@ early_param("possible_cpus", _setup_poss
set_cpu_possible(i, true);
}
+/* correctly size the local cpu masks */
+void __init setup_cpu_local_masks(void)
+{
+ alloc_bootmem_cpumask_var(&cpu_initialized_mask);
+ alloc_bootmem_cpumask_var(&cpu_callin_mask);
+ alloc_bootmem_cpumask_var(&cpu_callout_mask);
+ alloc_bootmem_cpumask_var(&cpu_sibling_setup_mask);
+}
+
#ifdef CONFIG_HOTPLUG_CPU
/* Recompute SMT state for all CPUs on offline */
--- a/arch/x86/xen/smp_pv.c
+++ b/arch/x86/xen/smp_pv.c
@@ -254,15 +254,12 @@ cpu_initialize_context(unsigned int cpu,
struct desc_struct *gdt;
unsigned long gdt_mfn;
- /* used to tell cpu_init() that it can proceed with initialization */
- cpumask_set_cpu(cpu, cpu_callout_mask);
if (cpumask_test_and_set_cpu(cpu, xen_cpu_initialized_map))
return 0;
ctxt = kzalloc(sizeof(*ctxt), GFP_KERNEL);
if (ctxt == NULL) {
cpumask_clear_cpu(cpu, xen_cpu_initialized_map);
- cpumask_clear_cpu(cpu, cpu_callout_mask);
return -ENOMEM;
}
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 11/37] x86/smpboot: Move synchronization masks to SMP boot code
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
The usage is in smpboot.c and not in the CPU initialization code.
The XEN_PV usage of cpu_callout_mask is obsolete as cpu_init() not longer
waits and cacheinfo has its own CPU mask now, so cpu_callout_mask can be
made static too.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/include/asm/cpumask.h | 5 -----
arch/x86/kernel/cpu/common.c | 17 -----------------
arch/x86/kernel/smpboot.c | 16 ++++++++++++++++
arch/x86/xen/smp_pv.c | 3 ---
4 files changed, 16 insertions(+), 25 deletions(-)
--- a/arch/x86/include/asm/cpumask.h
+++ b/arch/x86/include/asm/cpumask.h
@@ -4,11 +4,6 @@
#ifndef __ASSEMBLY__
#include <linux/cpumask.h>
-extern cpumask_var_t cpu_callin_mask;
-extern cpumask_var_t cpu_callout_mask;
-extern cpumask_var_t cpu_initialized_mask;
-extern cpumask_var_t cpu_sibling_setup_mask;
-
extern void setup_cpu_local_masks(void);
/*
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -67,14 +67,6 @@
u32 elf_hwcap2 __read_mostly;
-/* all of these masks are initialized in setup_cpu_local_masks() */
-cpumask_var_t cpu_initialized_mask;
-cpumask_var_t cpu_callout_mask;
-cpumask_var_t cpu_callin_mask;
-
-/* representing cpus for which sibling maps can be computed */
-cpumask_var_t cpu_sibling_setup_mask;
-
/* Number of siblings per CPU package */
int smp_num_siblings = 1;
EXPORT_SYMBOL(smp_num_siblings);
@@ -168,15 +160,6 @@ static void ppin_init(struct cpuinfo_x86
clear_cpu_cap(c, info->feature);
}
-/* correctly size the local cpu masks */
-void __init setup_cpu_local_masks(void)
-{
- alloc_bootmem_cpumask_var(&cpu_initialized_mask);
- alloc_bootmem_cpumask_var(&cpu_callin_mask);
- alloc_bootmem_cpumask_var(&cpu_callout_mask);
- alloc_bootmem_cpumask_var(&cpu_sibling_setup_mask);
-}
-
static void default_init(struct cpuinfo_x86 *c)
{
#ifdef CONFIG_X86_64
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -101,6 +101,13 @@ EXPORT_PER_CPU_SYMBOL(cpu_die_map);
DEFINE_PER_CPU_READ_MOSTLY(struct cpuinfo_x86, cpu_info);
EXPORT_PER_CPU_SYMBOL(cpu_info);
+/* All of these masks are initialized in setup_cpu_local_masks() */
+static cpumask_var_t cpu_initialized_mask;
+static cpumask_var_t cpu_callout_mask;
+static cpumask_var_t cpu_callin_mask;
+/* Representing CPUs for which sibling maps can be computed */
+static cpumask_var_t cpu_sibling_setup_mask;
+
/* Logical package management. We might want to allocate that dynamically */
unsigned int __max_logical_packages __read_mostly;
EXPORT_SYMBOL(__max_logical_packages);
@@ -1548,6 +1555,15 @@ early_param("possible_cpus", _setup_poss
set_cpu_possible(i, true);
}
+/* correctly size the local cpu masks */
+void __init setup_cpu_local_masks(void)
+{
+ alloc_bootmem_cpumask_var(&cpu_initialized_mask);
+ alloc_bootmem_cpumask_var(&cpu_callin_mask);
+ alloc_bootmem_cpumask_var(&cpu_callout_mask);
+ alloc_bootmem_cpumask_var(&cpu_sibling_setup_mask);
+}
+
#ifdef CONFIG_HOTPLUG_CPU
/* Recompute SMT state for all CPUs on offline */
--- a/arch/x86/xen/smp_pv.c
+++ b/arch/x86/xen/smp_pv.c
@@ -254,15 +254,12 @@ cpu_initialize_context(unsigned int cpu,
struct desc_struct *gdt;
unsigned long gdt_mfn;
- /* used to tell cpu_init() that it can proceed with initialization */
- cpumask_set_cpu(cpu, cpu_callout_mask);
if (cpumask_test_and_set_cpu(cpu, xen_cpu_initialized_map))
return 0;
ctxt = kzalloc(sizeof(*ctxt), GFP_KERNEL);
if (ctxt == NULL) {
cpumask_clear_cpu(cpu, xen_cpu_initialized_map);
- cpumask_clear_cpu(cpu, cpu_callout_mask);
return -ENOMEM;
}
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 12/37] x86/smpboot: Make TSC synchronization function call based
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-14 23:44 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Spin-waiting on the control CPU until the AP reaches the TSC
synchronization is just a waste especially in the case that there is no
synchronization required.
As the synchronization has to run with interrupts disabled the control CPU
part can just be done from a SMP function call. The upcoming AP issues that
call async only in the case that synchronization is required.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/include/asm/tsc.h | 2 --
arch/x86/kernel/smpboot.c | 20 +++-----------------
arch/x86/kernel/tsc_sync.c | 36 +++++++++++-------------------------
3 files changed, 14 insertions(+), 44 deletions(-)
--- a/arch/x86/include/asm/tsc.h
+++ b/arch/x86/include/asm/tsc.h
@@ -55,12 +55,10 @@ extern bool tsc_async_resets;
#ifdef CONFIG_X86_TSC
extern bool tsc_store_and_check_tsc_adjust(bool bootcpu);
extern void tsc_verify_tsc_adjust(bool resume);
-extern void check_tsc_sync_source(int cpu);
extern void check_tsc_sync_target(void);
#else
static inline bool tsc_store_and_check_tsc_adjust(bool bootcpu) { return false; }
static inline void tsc_verify_tsc_adjust(bool resume) { }
-static inline void check_tsc_sync_source(int cpu) { }
static inline void check_tsc_sync_target(void) { }
#endif
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -278,11 +278,7 @@ static void notrace start_secondary(void
/* Otherwise gcc will move up smp_processor_id() before cpu_init() */
barrier();
- /*
- * Check TSC synchronization with the control CPU, which will do
- * its part of this from wait_cpu_online(), making it an implicit
- * synchronization point.
- */
+ /* Check TSC synchronization with the control CPU. */
check_tsc_sync_target();
/*
@@ -1144,21 +1140,11 @@ static void wait_cpu_callin(unsigned int
}
/*
- * Bringup step four: Synchronize the TSC and wait for the target AP
- * to reach set_cpu_online() in start_secondary().
+ * Bringup step four: Wait for the target AP to reach set_cpu_online() in
+ * start_secondary().
*/
static void wait_cpu_online(unsigned int cpu)
{
- unsigned long flags;
-
- /*
- * Check TSC synchronization with the AP (keep irqs disabled
- * while doing so):
- */
- local_irq_save(flags);
- check_tsc_sync_source(cpu);
- local_irq_restore(flags);
-
/*
* Wait for the AP to mark itself online, so the core caller
* can drop sparse_irq_lock.
--- a/arch/x86/kernel/tsc_sync.c
+++ b/arch/x86/kernel/tsc_sync.c
@@ -245,7 +245,6 @@ bool tsc_store_and_check_tsc_adjust(bool
*/
static atomic_t start_count;
static atomic_t stop_count;
-static atomic_t skip_test;
static atomic_t test_runs;
/*
@@ -344,21 +343,14 @@ static inline unsigned int loop_timeout(
}
/*
- * Source CPU calls into this - it waits for the freshly booted
- * target CPU to arrive and then starts the measurement:
+ * The freshly booted CPU initiates this via an async SMP function call.
*/
-void check_tsc_sync_source(int cpu)
+static void check_tsc_sync_source(void *__cpu)
{
+ unsigned int cpu = (unsigned long)__cpu;
int cpus = 2;
/*
- * No need to check if we already know that the TSC is not
- * synchronized or if we have no TSC.
- */
- if (unsynchronized_tsc())
- return;
-
- /*
* Set the maximum number of test runs to
* 1 if the CPU does not provide the TSC_ADJUST MSR
* 3 if the MSR is available, so the target can try to adjust
@@ -368,16 +360,9 @@ void check_tsc_sync_source(int cpu)
else
atomic_set(&test_runs, 3);
retry:
- /*
- * Wait for the target to start or to skip the test:
- */
- while (atomic_read(&start_count) != cpus - 1) {
- if (atomic_read(&skip_test) > 0) {
- atomic_set(&skip_test, 0);
- return;
- }
+ /* Wait for the target to start. */
+ while (atomic_read(&start_count) != cpus - 1)
cpu_relax();
- }
/*
* Trigger the target to continue into the measurement too:
@@ -397,14 +382,14 @@ void check_tsc_sync_source(int cpu)
if (!nr_warps) {
atomic_set(&test_runs, 0);
- pr_debug("TSC synchronization [CPU#%d -> CPU#%d]: passed\n",
+ pr_debug("TSC synchronization [CPU#%d -> CPU#%u]: passed\n",
smp_processor_id(), cpu);
} else if (atomic_dec_and_test(&test_runs) || random_warps) {
/* Force it to 0 if random warps brought us here */
atomic_set(&test_runs, 0);
- pr_warn("TSC synchronization [CPU#%d -> CPU#%d]:\n",
+ pr_warn("TSC synchronization [CPU#%d -> CPU#%u]:\n",
smp_processor_id(), cpu);
pr_warn("Measured %Ld cycles TSC warp between CPUs, "
"turning off TSC clock.\n", max_warp);
@@ -457,11 +442,12 @@ void check_tsc_sync_target(void)
* SoCs the TSC is frequency synchronized, but still the TSC ADJUST
* register might have been wreckaged by the BIOS..
*/
- if (tsc_store_and_check_tsc_adjust(false) || tsc_clocksource_reliable) {
- atomic_inc(&skip_test);
+ if (tsc_store_and_check_tsc_adjust(false) || tsc_clocksource_reliable)
return;
- }
+ /* Kick the control CPU into the TSC synchronization function */
+ smp_call_function_single(cpumask_first(cpu_online_mask), check_tsc_sync_source,
+ (unsigned long *)(unsigned long)cpu, 0);
retry:
/*
* Register this CPU's participation and wait for the
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 12/37] x86/smpboot: Make TSC synchronization function call based
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Spin-waiting on the control CPU until the AP reaches the TSC
synchronization is just a waste especially in the case that there is no
synchronization required.
As the synchronization has to run with interrupts disabled the control CPU
part can just be done from a SMP function call. The upcoming AP issues that
call async only in the case that synchronization is required.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/include/asm/tsc.h | 2 --
arch/x86/kernel/smpboot.c | 20 +++-----------------
arch/x86/kernel/tsc_sync.c | 36 +++++++++++-------------------------
3 files changed, 14 insertions(+), 44 deletions(-)
--- a/arch/x86/include/asm/tsc.h
+++ b/arch/x86/include/asm/tsc.h
@@ -55,12 +55,10 @@ extern bool tsc_async_resets;
#ifdef CONFIG_X86_TSC
extern bool tsc_store_and_check_tsc_adjust(bool bootcpu);
extern void tsc_verify_tsc_adjust(bool resume);
-extern void check_tsc_sync_source(int cpu);
extern void check_tsc_sync_target(void);
#else
static inline bool tsc_store_and_check_tsc_adjust(bool bootcpu) { return false; }
static inline void tsc_verify_tsc_adjust(bool resume) { }
-static inline void check_tsc_sync_source(int cpu) { }
static inline void check_tsc_sync_target(void) { }
#endif
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -278,11 +278,7 @@ static void notrace start_secondary(void
/* Otherwise gcc will move up smp_processor_id() before cpu_init() */
barrier();
- /*
- * Check TSC synchronization with the control CPU, which will do
- * its part of this from wait_cpu_online(), making it an implicit
- * synchronization point.
- */
+ /* Check TSC synchronization with the control CPU. */
check_tsc_sync_target();
/*
@@ -1144,21 +1140,11 @@ static void wait_cpu_callin(unsigned int
}
/*
- * Bringup step four: Synchronize the TSC and wait for the target AP
- * to reach set_cpu_online() in start_secondary().
+ * Bringup step four: Wait for the target AP to reach set_cpu_online() in
+ * start_secondary().
*/
static void wait_cpu_online(unsigned int cpu)
{
- unsigned long flags;
-
- /*
- * Check TSC synchronization with the AP (keep irqs disabled
- * while doing so):
- */
- local_irq_save(flags);
- check_tsc_sync_source(cpu);
- local_irq_restore(flags);
-
/*
* Wait for the AP to mark itself online, so the core caller
* can drop sparse_irq_lock.
--- a/arch/x86/kernel/tsc_sync.c
+++ b/arch/x86/kernel/tsc_sync.c
@@ -245,7 +245,6 @@ bool tsc_store_and_check_tsc_adjust(bool
*/
static atomic_t start_count;
static atomic_t stop_count;
-static atomic_t skip_test;
static atomic_t test_runs;
/*
@@ -344,21 +343,14 @@ static inline unsigned int loop_timeout(
}
/*
- * Source CPU calls into this - it waits for the freshly booted
- * target CPU to arrive and then starts the measurement:
+ * The freshly booted CPU initiates this via an async SMP function call.
*/
-void check_tsc_sync_source(int cpu)
+static void check_tsc_sync_source(void *__cpu)
{
+ unsigned int cpu = (unsigned long)__cpu;
int cpus = 2;
/*
- * No need to check if we already know that the TSC is not
- * synchronized or if we have no TSC.
- */
- if (unsynchronized_tsc())
- return;
-
- /*
* Set the maximum number of test runs to
* 1 if the CPU does not provide the TSC_ADJUST MSR
* 3 if the MSR is available, so the target can try to adjust
@@ -368,16 +360,9 @@ void check_tsc_sync_source(int cpu)
else
atomic_set(&test_runs, 3);
retry:
- /*
- * Wait for the target to start or to skip the test:
- */
- while (atomic_read(&start_count) != cpus - 1) {
- if (atomic_read(&skip_test) > 0) {
- atomic_set(&skip_test, 0);
- return;
- }
+ /* Wait for the target to start. */
+ while (atomic_read(&start_count) != cpus - 1)
cpu_relax();
- }
/*
* Trigger the target to continue into the measurement too:
@@ -397,14 +382,14 @@ void check_tsc_sync_source(int cpu)
if (!nr_warps) {
atomic_set(&test_runs, 0);
- pr_debug("TSC synchronization [CPU#%d -> CPU#%d]: passed\n",
+ pr_debug("TSC synchronization [CPU#%d -> CPU#%u]: passed\n",
smp_processor_id(), cpu);
} else if (atomic_dec_and_test(&test_runs) || random_warps) {
/* Force it to 0 if random warps brought us here */
atomic_set(&test_runs, 0);
- pr_warn("TSC synchronization [CPU#%d -> CPU#%d]:\n",
+ pr_warn("TSC synchronization [CPU#%d -> CPU#%u]:\n",
smp_processor_id(), cpu);
pr_warn("Measured %Ld cycles TSC warp between CPUs, "
"turning off TSC clock.\n", max_warp);
@@ -457,11 +442,12 @@ void check_tsc_sync_target(void)
* SoCs the TSC is frequency synchronized, but still the TSC ADJUST
* register might have been wreckaged by the BIOS..
*/
- if (tsc_store_and_check_tsc_adjust(false) || tsc_clocksource_reliable) {
- atomic_inc(&skip_test);
+ if (tsc_store_and_check_tsc_adjust(false) || tsc_clocksource_reliable)
return;
- }
+ /* Kick the control CPU into the TSC synchronization function */
+ smp_call_function_single(cpumask_first(cpu_online_mask), check_tsc_sync_source,
+ (unsigned long *)(unsigned long)cpu, 0);
retry:
/*
* Register this CPU's participation and wait for the
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 12/37] x86/smpboot: Make TSC synchronization function call based
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Spin-waiting on the control CPU until the AP reaches the TSC
synchronization is just a waste especially in the case that there is no
synchronization required.
As the synchronization has to run with interrupts disabled the control CPU
part can just be done from a SMP function call. The upcoming AP issues that
call async only in the case that synchronization is required.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/include/asm/tsc.h | 2 --
arch/x86/kernel/smpboot.c | 20 +++-----------------
arch/x86/kernel/tsc_sync.c | 36 +++++++++++-------------------------
3 files changed, 14 insertions(+), 44 deletions(-)
--- a/arch/x86/include/asm/tsc.h
+++ b/arch/x86/include/asm/tsc.h
@@ -55,12 +55,10 @@ extern bool tsc_async_resets;
#ifdef CONFIG_X86_TSC
extern bool tsc_store_and_check_tsc_adjust(bool bootcpu);
extern void tsc_verify_tsc_adjust(bool resume);
-extern void check_tsc_sync_source(int cpu);
extern void check_tsc_sync_target(void);
#else
static inline bool tsc_store_and_check_tsc_adjust(bool bootcpu) { return false; }
static inline void tsc_verify_tsc_adjust(bool resume) { }
-static inline void check_tsc_sync_source(int cpu) { }
static inline void check_tsc_sync_target(void) { }
#endif
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -278,11 +278,7 @@ static void notrace start_secondary(void
/* Otherwise gcc will move up smp_processor_id() before cpu_init() */
barrier();
- /*
- * Check TSC synchronization with the control CPU, which will do
- * its part of this from wait_cpu_online(), making it an implicit
- * synchronization point.
- */
+ /* Check TSC synchronization with the control CPU. */
check_tsc_sync_target();
/*
@@ -1144,21 +1140,11 @@ static void wait_cpu_callin(unsigned int
}
/*
- * Bringup step four: Synchronize the TSC and wait for the target AP
- * to reach set_cpu_online() in start_secondary().
+ * Bringup step four: Wait for the target AP to reach set_cpu_online() in
+ * start_secondary().
*/
static void wait_cpu_online(unsigned int cpu)
{
- unsigned long flags;
-
- /*
- * Check TSC synchronization with the AP (keep irqs disabled
- * while doing so):
- */
- local_irq_save(flags);
- check_tsc_sync_source(cpu);
- local_irq_restore(flags);
-
/*
* Wait for the AP to mark itself online, so the core caller
* can drop sparse_irq_lock.
--- a/arch/x86/kernel/tsc_sync.c
+++ b/arch/x86/kernel/tsc_sync.c
@@ -245,7 +245,6 @@ bool tsc_store_and_check_tsc_adjust(bool
*/
static atomic_t start_count;
static atomic_t stop_count;
-static atomic_t skip_test;
static atomic_t test_runs;
/*
@@ -344,21 +343,14 @@ static inline unsigned int loop_timeout(
}
/*
- * Source CPU calls into this - it waits for the freshly booted
- * target CPU to arrive and then starts the measurement:
+ * The freshly booted CPU initiates this via an async SMP function call.
*/
-void check_tsc_sync_source(int cpu)
+static void check_tsc_sync_source(void *__cpu)
{
+ unsigned int cpu = (unsigned long)__cpu;
int cpus = 2;
/*
- * No need to check if we already know that the TSC is not
- * synchronized or if we have no TSC.
- */
- if (unsynchronized_tsc())
- return;
-
- /*
* Set the maximum number of test runs to
* 1 if the CPU does not provide the TSC_ADJUST MSR
* 3 if the MSR is available, so the target can try to adjust
@@ -368,16 +360,9 @@ void check_tsc_sync_source(int cpu)
else
atomic_set(&test_runs, 3);
retry:
- /*
- * Wait for the target to start or to skip the test:
- */
- while (atomic_read(&start_count) != cpus - 1) {
- if (atomic_read(&skip_test) > 0) {
- atomic_set(&skip_test, 0);
- return;
- }
+ /* Wait for the target to start. */
+ while (atomic_read(&start_count) != cpus - 1)
cpu_relax();
- }
/*
* Trigger the target to continue into the measurement too:
@@ -397,14 +382,14 @@ void check_tsc_sync_source(int cpu)
if (!nr_warps) {
atomic_set(&test_runs, 0);
- pr_debug("TSC synchronization [CPU#%d -> CPU#%d]: passed\n",
+ pr_debug("TSC synchronization [CPU#%d -> CPU#%u]: passed\n",
smp_processor_id(), cpu);
} else if (atomic_dec_and_test(&test_runs) || random_warps) {
/* Force it to 0 if random warps brought us here */
atomic_set(&test_runs, 0);
- pr_warn("TSC synchronization [CPU#%d -> CPU#%d]:\n",
+ pr_warn("TSC synchronization [CPU#%d -> CPU#%u]:\n",
smp_processor_id(), cpu);
pr_warn("Measured %Ld cycles TSC warp between CPUs, "
"turning off TSC clock.\n", max_warp);
@@ -457,11 +442,12 @@ void check_tsc_sync_target(void)
* SoCs the TSC is frequency synchronized, but still the TSC ADJUST
* register might have been wreckaged by the BIOS..
*/
- if (tsc_store_and_check_tsc_adjust(false) || tsc_clocksource_reliable) {
- atomic_inc(&skip_test);
+ if (tsc_store_and_check_tsc_adjust(false) || tsc_clocksource_reliable)
return;
- }
+ /* Kick the control CPU into the TSC synchronization function */
+ smp_call_function_single(cpumask_first(cpu_online_mask), check_tsc_sync_source,
+ (unsigned long *)(unsigned long)cpu, 0);
retry:
/*
* Register this CPU's participation and wait for the
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 13/37] x86/smpboot: Remove cpu_callin_mask
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-14 23:44 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Now that TSC synchronization is SMP function call based there is no reason
to wait for the AP to be set in smp_callin_mask. The control CPU waits for
the AP to set itself in the online mask anyway.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/kernel/smpboot.c | 61 +++++++---------------------------------------
1 file changed, 10 insertions(+), 51 deletions(-)
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -104,7 +104,6 @@ EXPORT_PER_CPU_SYMBOL(cpu_info);
/* All of these masks are initialized in setup_cpu_local_masks() */
static cpumask_var_t cpu_initialized_mask;
static cpumask_var_t cpu_callout_mask;
-static cpumask_var_t cpu_callin_mask;
/* Representing CPUs for which sibling maps can be computed */
static cpumask_var_t cpu_sibling_setup_mask;
@@ -167,21 +166,16 @@ static inline void smpboot_restore_warm_
*/
static void smp_callin(void)
{
- int cpuid;
+ int cpuid = smp_processor_id();
/*
* If waken up by an INIT in an 82489DX configuration
- * cpu_callout_mask guarantees we don't get here before
- * an INIT_deassert IPI reaches our local APIC, so it is
- * now safe to touch our local APIC.
- */
- cpuid = smp_processor_id();
-
- /*
- * the boot CPU has finished the init stage and is spinning
- * on callin_map until we finish. We are free to set up this
- * CPU, first the APIC. (this is probably redundant on most
- * boards)
+ * cpu_callout_mask guarantees we don't get here before an
+ * INIT_deassert IPI reaches our local APIC, so it is now safe to
+ * touch our local APIC.
+ *
+ * Set up this CPU, first the APIC, which is probably redundant on
+ * most boards.
*/
apic_ap_setup();
@@ -192,7 +186,7 @@ static void smp_callin(void)
* The topology information must be up to date before
* calibrate_delay() and notify_cpu_starting().
*/
- set_cpu_sibling_map(raw_smp_processor_id());
+ set_cpu_sibling_map(cpuid);
ap_init_aperfmperf();
@@ -205,11 +199,6 @@ static void smp_callin(void)
* state (CPUHP_ONLINE in the case of serial bringup).
*/
notify_cpu_starting(cpuid);
-
- /*
- * Allow the master to continue.
- */
- cpumask_set_cpu(cpuid, cpu_callin_mask);
}
static void ap_calibrate_delay(void)
@@ -268,11 +257,6 @@ static void notrace start_secondary(void
rcu_cpu_starting(raw_smp_processor_id());
x86_cpuinit.early_percpu_clock_init();
- /*
- * Sync point with wait_cpu_callin(). The AP doesn't wait here
- * but just sets the bit to let the controlling CPU (BSP) know that
- * it's got this far.
- */
smp_callin();
/* Otherwise gcc will move up smp_processor_id() before cpu_init() */
@@ -1112,7 +1096,7 @@ static int wait_cpu_cpumask(unsigned int
* and thus wait_for_master_cpu(), then set cpu_callout_mask to allow it
* to proceed. The AP will then proceed past setting its 'callin' bit
* and end up waiting in check_tsc_sync_target() until we reach
- * do_wait_cpu_online() to tend to it.
+ * wait_cpu_online() to tend to it.
*/
static int wait_cpu_initialized(unsigned int cpu)
{
@@ -1127,20 +1111,7 @@ static int wait_cpu_initialized(unsigned
}
/*
- * Bringup step three: Wait for the target AP to reach smp_callin().
- * The AP is not waiting for us here so we don't need to parallelise
- * this step. Not entirely clear why we care about this, since we just
- * proceed directly to TSC synchronization which is the next sync
- * point with the AP anyway.
- */
-static void wait_cpu_callin(unsigned int cpu)
-{
- while (!cpumask_test_cpu(cpu, cpu_callin_mask))
- schedule();
-}
-
-/*
- * Bringup step four: Wait for the target AP to reach set_cpu_online() in
+ * Bringup step three: Wait for the target AP to reach set_cpu_online() in
* start_secondary().
*/
static void wait_cpu_online(unsigned int cpu)
@@ -1170,14 +1141,6 @@ static int native_kick_ap(unsigned int c
}
/*
- * Already booted CPU?
- */
- if (cpumask_test_cpu(cpu, cpu_callin_mask)) {
- pr_debug("do_boot_cpu %d Already started\n", cpu);
- return -ENOSYS;
- }
-
- /*
* Save current MTRR state in case it was changed since early boot
* (e.g. by the ACPI SMI) to initialize new CPUs with MTRRs in sync:
*/
@@ -1214,7 +1177,6 @@ int native_cpu_up(unsigned int cpu, stru
if (ret)
goto out;
- wait_cpu_callin(cpu);
wait_cpu_online(cpu);
out:
@@ -1330,7 +1292,6 @@ void __init smp_prepare_cpus_common(void
* Setup boot CPU information
*/
smp_store_boot_cpu_info(); /* Final full version of the data */
- cpumask_copy(cpu_callin_mask, cpumask_of(0));
mb();
for_each_possible_cpu(i) {
@@ -1545,7 +1506,6 @@ early_param("possible_cpus", _setup_poss
void __init setup_cpu_local_masks(void)
{
alloc_bootmem_cpumask_var(&cpu_initialized_mask);
- alloc_bootmem_cpumask_var(&cpu_callin_mask);
alloc_bootmem_cpumask_var(&cpu_callout_mask);
alloc_bootmem_cpumask_var(&cpu_sibling_setup_mask);
}
@@ -1609,7 +1569,6 @@ static void remove_cpu_from_maps(int cpu
{
set_cpu_online(cpu, false);
cpumask_clear_cpu(cpu, cpu_callout_mask);
- cpumask_clear_cpu(cpu, cpu_callin_mask);
/* was set by cpu_init() */
cpumask_clear_cpu(cpu, cpu_initialized_mask);
numa_remove_cpu(cpu);
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 13/37] x86/smpboot: Remove cpu_callin_mask
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Now that TSC synchronization is SMP function call based there is no reason
to wait for the AP to be set in smp_callin_mask. The control CPU waits for
the AP to set itself in the online mask anyway.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/kernel/smpboot.c | 61 +++++++---------------------------------------
1 file changed, 10 insertions(+), 51 deletions(-)
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -104,7 +104,6 @@ EXPORT_PER_CPU_SYMBOL(cpu_info);
/* All of these masks are initialized in setup_cpu_local_masks() */
static cpumask_var_t cpu_initialized_mask;
static cpumask_var_t cpu_callout_mask;
-static cpumask_var_t cpu_callin_mask;
/* Representing CPUs for which sibling maps can be computed */
static cpumask_var_t cpu_sibling_setup_mask;
@@ -167,21 +166,16 @@ static inline void smpboot_restore_warm_
*/
static void smp_callin(void)
{
- int cpuid;
+ int cpuid = smp_processor_id();
/*
* If waken up by an INIT in an 82489DX configuration
- * cpu_callout_mask guarantees we don't get here before
- * an INIT_deassert IPI reaches our local APIC, so it is
- * now safe to touch our local APIC.
- */
- cpuid = smp_processor_id();
-
- /*
- * the boot CPU has finished the init stage and is spinning
- * on callin_map until we finish. We are free to set up this
- * CPU, first the APIC. (this is probably redundant on most
- * boards)
+ * cpu_callout_mask guarantees we don't get here before an
+ * INIT_deassert IPI reaches our local APIC, so it is now safe to
+ * touch our local APIC.
+ *
+ * Set up this CPU, first the APIC, which is probably redundant on
+ * most boards.
*/
apic_ap_setup();
@@ -192,7 +186,7 @@ static void smp_callin(void)
* The topology information must be up to date before
* calibrate_delay() and notify_cpu_starting().
*/
- set_cpu_sibling_map(raw_smp_processor_id());
+ set_cpu_sibling_map(cpuid);
ap_init_aperfmperf();
@@ -205,11 +199,6 @@ static void smp_callin(void)
* state (CPUHP_ONLINE in the case of serial bringup).
*/
notify_cpu_starting(cpuid);
-
- /*
- * Allow the master to continue.
- */
- cpumask_set_cpu(cpuid, cpu_callin_mask);
}
static void ap_calibrate_delay(void)
@@ -268,11 +257,6 @@ static void notrace start_secondary(void
rcu_cpu_starting(raw_smp_processor_id());
x86_cpuinit.early_percpu_clock_init();
- /*
- * Sync point with wait_cpu_callin(). The AP doesn't wait here
- * but just sets the bit to let the controlling CPU (BSP) know that
- * it's got this far.
- */
smp_callin();
/* Otherwise gcc will move up smp_processor_id() before cpu_init() */
@@ -1112,7 +1096,7 @@ static int wait_cpu_cpumask(unsigned int
* and thus wait_for_master_cpu(), then set cpu_callout_mask to allow it
* to proceed. The AP will then proceed past setting its 'callin' bit
* and end up waiting in check_tsc_sync_target() until we reach
- * do_wait_cpu_online() to tend to it.
+ * wait_cpu_online() to tend to it.
*/
static int wait_cpu_initialized(unsigned int cpu)
{
@@ -1127,20 +1111,7 @@ static int wait_cpu_initialized(unsigned
}
/*
- * Bringup step three: Wait for the target AP to reach smp_callin().
- * The AP is not waiting for us here so we don't need to parallelise
- * this step. Not entirely clear why we care about this, since we just
- * proceed directly to TSC synchronization which is the next sync
- * point with the AP anyway.
- */
-static void wait_cpu_callin(unsigned int cpu)
-{
- while (!cpumask_test_cpu(cpu, cpu_callin_mask))
- schedule();
-}
-
-/*
- * Bringup step four: Wait for the target AP to reach set_cpu_online() in
+ * Bringup step three: Wait for the target AP to reach set_cpu_online() in
* start_secondary().
*/
static void wait_cpu_online(unsigned int cpu)
@@ -1170,14 +1141,6 @@ static int native_kick_ap(unsigned int c
}
/*
- * Already booted CPU?
- */
- if (cpumask_test_cpu(cpu, cpu_callin_mask)) {
- pr_debug("do_boot_cpu %d Already started\n", cpu);
- return -ENOSYS;
- }
-
- /*
* Save current MTRR state in case it was changed since early boot
* (e.g. by the ACPI SMI) to initialize new CPUs with MTRRs in sync:
*/
@@ -1214,7 +1177,6 @@ int native_cpu_up(unsigned int cpu, stru
if (ret)
goto out;
- wait_cpu_callin(cpu);
wait_cpu_online(cpu);
out:
@@ -1330,7 +1292,6 @@ void __init smp_prepare_cpus_common(void
* Setup boot CPU information
*/
smp_store_boot_cpu_info(); /* Final full version of the data */
- cpumask_copy(cpu_callin_mask, cpumask_of(0));
mb();
for_each_possible_cpu(i) {
@@ -1545,7 +1506,6 @@ early_param("possible_cpus", _setup_poss
void __init setup_cpu_local_masks(void)
{
alloc_bootmem_cpumask_var(&cpu_initialized_mask);
- alloc_bootmem_cpumask_var(&cpu_callin_mask);
alloc_bootmem_cpumask_var(&cpu_callout_mask);
alloc_bootmem_cpumask_var(&cpu_sibling_setup_mask);
}
@@ -1609,7 +1569,6 @@ static void remove_cpu_from_maps(int cpu
{
set_cpu_online(cpu, false);
cpumask_clear_cpu(cpu, cpu_callout_mask);
- cpumask_clear_cpu(cpu, cpu_callin_mask);
/* was set by cpu_init() */
cpumask_clear_cpu(cpu, cpu_initialized_mask);
numa_remove_cpu(cpu);
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 13/37] x86/smpboot: Remove cpu_callin_mask
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Now that TSC synchronization is SMP function call based there is no reason
to wait for the AP to be set in smp_callin_mask. The control CPU waits for
the AP to set itself in the online mask anyway.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/kernel/smpboot.c | 61 +++++++---------------------------------------
1 file changed, 10 insertions(+), 51 deletions(-)
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -104,7 +104,6 @@ EXPORT_PER_CPU_SYMBOL(cpu_info);
/* All of these masks are initialized in setup_cpu_local_masks() */
static cpumask_var_t cpu_initialized_mask;
static cpumask_var_t cpu_callout_mask;
-static cpumask_var_t cpu_callin_mask;
/* Representing CPUs for which sibling maps can be computed */
static cpumask_var_t cpu_sibling_setup_mask;
@@ -167,21 +166,16 @@ static inline void smpboot_restore_warm_
*/
static void smp_callin(void)
{
- int cpuid;
+ int cpuid = smp_processor_id();
/*
* If waken up by an INIT in an 82489DX configuration
- * cpu_callout_mask guarantees we don't get here before
- * an INIT_deassert IPI reaches our local APIC, so it is
- * now safe to touch our local APIC.
- */
- cpuid = smp_processor_id();
-
- /*
- * the boot CPU has finished the init stage and is spinning
- * on callin_map until we finish. We are free to set up this
- * CPU, first the APIC. (this is probably redundant on most
- * boards)
+ * cpu_callout_mask guarantees we don't get here before an
+ * INIT_deassert IPI reaches our local APIC, so it is now safe to
+ * touch our local APIC.
+ *
+ * Set up this CPU, first the APIC, which is probably redundant on
+ * most boards.
*/
apic_ap_setup();
@@ -192,7 +186,7 @@ static void smp_callin(void)
* The topology information must be up to date before
* calibrate_delay() and notify_cpu_starting().
*/
- set_cpu_sibling_map(raw_smp_processor_id());
+ set_cpu_sibling_map(cpuid);
ap_init_aperfmperf();
@@ -205,11 +199,6 @@ static void smp_callin(void)
* state (CPUHP_ONLINE in the case of serial bringup).
*/
notify_cpu_starting(cpuid);
-
- /*
- * Allow the master to continue.
- */
- cpumask_set_cpu(cpuid, cpu_callin_mask);
}
static void ap_calibrate_delay(void)
@@ -268,11 +257,6 @@ static void notrace start_secondary(void
rcu_cpu_starting(raw_smp_processor_id());
x86_cpuinit.early_percpu_clock_init();
- /*
- * Sync point with wait_cpu_callin(). The AP doesn't wait here
- * but just sets the bit to let the controlling CPU (BSP) know that
- * it's got this far.
- */
smp_callin();
/* Otherwise gcc will move up smp_processor_id() before cpu_init() */
@@ -1112,7 +1096,7 @@ static int wait_cpu_cpumask(unsigned int
* and thus wait_for_master_cpu(), then set cpu_callout_mask to allow it
* to proceed. The AP will then proceed past setting its 'callin' bit
* and end up waiting in check_tsc_sync_target() until we reach
- * do_wait_cpu_online() to tend to it.
+ * wait_cpu_online() to tend to it.
*/
static int wait_cpu_initialized(unsigned int cpu)
{
@@ -1127,20 +1111,7 @@ static int wait_cpu_initialized(unsigned
}
/*
- * Bringup step three: Wait for the target AP to reach smp_callin().
- * The AP is not waiting for us here so we don't need to parallelise
- * this step. Not entirely clear why we care about this, since we just
- * proceed directly to TSC synchronization which is the next sync
- * point with the AP anyway.
- */
-static void wait_cpu_callin(unsigned int cpu)
-{
- while (!cpumask_test_cpu(cpu, cpu_callin_mask))
- schedule();
-}
-
-/*
- * Bringup step four: Wait for the target AP to reach set_cpu_online() in
+ * Bringup step three: Wait for the target AP to reach set_cpu_online() in
* start_secondary().
*/
static void wait_cpu_online(unsigned int cpu)
@@ -1170,14 +1141,6 @@ static int native_kick_ap(unsigned int c
}
/*
- * Already booted CPU?
- */
- if (cpumask_test_cpu(cpu, cpu_callin_mask)) {
- pr_debug("do_boot_cpu %d Already started\n", cpu);
- return -ENOSYS;
- }
-
- /*
* Save current MTRR state in case it was changed since early boot
* (e.g. by the ACPI SMI) to initialize new CPUs with MTRRs in sync:
*/
@@ -1214,7 +1177,6 @@ int native_cpu_up(unsigned int cpu, stru
if (ret)
goto out;
- wait_cpu_callin(cpu);
wait_cpu_online(cpu);
out:
@@ -1330,7 +1292,6 @@ void __init smp_prepare_cpus_common(void
* Setup boot CPU information
*/
smp_store_boot_cpu_info(); /* Final full version of the data */
- cpumask_copy(cpu_callin_mask, cpumask_of(0));
mb();
for_each_possible_cpu(i) {
@@ -1545,7 +1506,6 @@ early_param("possible_cpus", _setup_poss
void __init setup_cpu_local_masks(void)
{
alloc_bootmem_cpumask_var(&cpu_initialized_mask);
- alloc_bootmem_cpumask_var(&cpu_callin_mask);
alloc_bootmem_cpumask_var(&cpu_callout_mask);
alloc_bootmem_cpumask_var(&cpu_sibling_setup_mask);
}
@@ -1609,7 +1569,6 @@ static void remove_cpu_from_maps(int cpu
{
set_cpu_online(cpu, false);
cpumask_clear_cpu(cpu, cpu_callout_mask);
- cpumask_clear_cpu(cpu, cpu_callin_mask);
/* was set by cpu_init() */
cpumask_clear_cpu(cpu, cpu_initialized_mask);
numa_remove_cpu(cpu);
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 14/37] cpu/hotplug: Rework sparse_irq locking in bringup_cpu()
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-14 23:44 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
There is no harm to hold sparse_irq lock until the upcoming CPU completes
in cpuhp_online_idle(). This allows to remove cpu_online() synchronization
from architecture code.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
kernel/cpu.c | 28 +++++++++++++++++++---------
1 file changed, 19 insertions(+), 9 deletions(-)
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -558,7 +558,7 @@ static int cpuhp_kick_ap(int cpu, struct
return ret;
}
-static int bringup_wait_for_ap(unsigned int cpu)
+static int bringup_wait_for_ap_online(unsigned int cpu)
{
struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
@@ -579,15 +579,12 @@ static int bringup_wait_for_ap(unsigned
*/
if (!cpu_smt_allowed(cpu))
return -ECANCELED;
-
- if (st->target <= CPUHP_AP_ONLINE_IDLE)
- return 0;
-
- return cpuhp_kick_ap(cpu, st, st->target);
+ return 0;
}
static int bringup_cpu(unsigned int cpu)
{
+ struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
struct task_struct *idle = idle_thread_get(cpu);
int ret;
@@ -606,10 +603,23 @@ static int bringup_cpu(unsigned int cpu)
/* Arch-specific enabling code. */
ret = __cpu_up(cpu, idle);
- irq_unlock_sparse();
if (ret)
- return ret;
- return bringup_wait_for_ap(cpu);
+ goto out_unlock;
+
+ ret = bringup_wait_for_ap_online(cpu);
+ if (ret)
+ goto out_unlock;
+
+ irq_unlock_sparse();
+
+ if (st->target <= CPUHP_AP_ONLINE_IDLE)
+ return 0;
+
+ return cpuhp_kick_ap(cpu, st, st->target);
+
+out_unlock:
+ irq_unlock_sparse();
+ return ret;
}
static int finish_cpu(unsigned int cpu)
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 14/37] cpu/hotplug: Rework sparse_irq locking in bringup_cpu()
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
There is no harm to hold sparse_irq lock until the upcoming CPU completes
in cpuhp_online_idle(). This allows to remove cpu_online() synchronization
from architecture code.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
kernel/cpu.c | 28 +++++++++++++++++++---------
1 file changed, 19 insertions(+), 9 deletions(-)
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -558,7 +558,7 @@ static int cpuhp_kick_ap(int cpu, struct
return ret;
}
-static int bringup_wait_for_ap(unsigned int cpu)
+static int bringup_wait_for_ap_online(unsigned int cpu)
{
struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
@@ -579,15 +579,12 @@ static int bringup_wait_for_ap(unsigned
*/
if (!cpu_smt_allowed(cpu))
return -ECANCELED;
-
- if (st->target <= CPUHP_AP_ONLINE_IDLE)
- return 0;
-
- return cpuhp_kick_ap(cpu, st, st->target);
+ return 0;
}
static int bringup_cpu(unsigned int cpu)
{
+ struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
struct task_struct *idle = idle_thread_get(cpu);
int ret;
@@ -606,10 +603,23 @@ static int bringup_cpu(unsigned int cpu)
/* Arch-specific enabling code. */
ret = __cpu_up(cpu, idle);
- irq_unlock_sparse();
if (ret)
- return ret;
- return bringup_wait_for_ap(cpu);
+ goto out_unlock;
+
+ ret = bringup_wait_for_ap_online(cpu);
+ if (ret)
+ goto out_unlock;
+
+ irq_unlock_sparse();
+
+ if (st->target <= CPUHP_AP_ONLINE_IDLE)
+ return 0;
+
+ return cpuhp_kick_ap(cpu, st, st->target);
+
+out_unlock:
+ irq_unlock_sparse();
+ return ret;
}
static int finish_cpu(unsigned int cpu)
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 14/37] cpu/hotplug: Rework sparse_irq locking in bringup_cpu()
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
There is no harm to hold sparse_irq lock until the upcoming CPU completes
in cpuhp_online_idle(). This allows to remove cpu_online() synchronization
from architecture code.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
kernel/cpu.c | 28 +++++++++++++++++++---------
1 file changed, 19 insertions(+), 9 deletions(-)
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -558,7 +558,7 @@ static int cpuhp_kick_ap(int cpu, struct
return ret;
}
-static int bringup_wait_for_ap(unsigned int cpu)
+static int bringup_wait_for_ap_online(unsigned int cpu)
{
struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
@@ -579,15 +579,12 @@ static int bringup_wait_for_ap(unsigned
*/
if (!cpu_smt_allowed(cpu))
return -ECANCELED;
-
- if (st->target <= CPUHP_AP_ONLINE_IDLE)
- return 0;
-
- return cpuhp_kick_ap(cpu, st, st->target);
+ return 0;
}
static int bringup_cpu(unsigned int cpu)
{
+ struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
struct task_struct *idle = idle_thread_get(cpu);
int ret;
@@ -606,10 +603,23 @@ static int bringup_cpu(unsigned int cpu)
/* Arch-specific enabling code. */
ret = __cpu_up(cpu, idle);
- irq_unlock_sparse();
if (ret)
- return ret;
- return bringup_wait_for_ap(cpu);
+ goto out_unlock;
+
+ ret = bringup_wait_for_ap_online(cpu);
+ if (ret)
+ goto out_unlock;
+
+ irq_unlock_sparse();
+
+ if (st->target <= CPUHP_AP_ONLINE_IDLE)
+ return 0;
+
+ return cpuhp_kick_ap(cpu, st, st->target);
+
+out_unlock:
+ irq_unlock_sparse();
+ return ret;
}
static int finish_cpu(unsigned int cpu)
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 15/37] x86/smpboot: Remove wait for cpu_online()
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-14 23:44 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Now that the core code drops sparse_irq_lock after the idle thread
synchronized, it's pointless to wait for the AP to mark itself online.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/kernel/smpboot.c | 25 ++-----------------------
1 file changed, 2 insertions(+), 23 deletions(-)
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -1110,20 +1110,6 @@ static int wait_cpu_initialized(unsigned
return 0;
}
-/*
- * Bringup step three: Wait for the target AP to reach set_cpu_online() in
- * start_secondary().
- */
-static void wait_cpu_online(unsigned int cpu)
-{
- /*
- * Wait for the AP to mark itself online, so the core caller
- * can drop sparse_irq_lock.
- */
- while (!cpu_online(cpu))
- schedule();
-}
-
static int native_kick_ap(unsigned int cpu, struct task_struct *tidle)
{
int apicid = apic->cpu_present_to_apicid(cpu);
@@ -1170,16 +1156,9 @@ int native_cpu_up(unsigned int cpu, stru
int ret;
ret = native_kick_ap(cpu, tidle);
- if (ret)
- goto out;
-
- ret = wait_cpu_initialized(cpu);
- if (ret)
- goto out;
-
- wait_cpu_online(cpu);
+ if (!ret)
+ ret = wait_cpu_initialized(cpu);
-out:
/* Cleanup possible dangling ends... */
if (x86_platform.legacy.warm_reset)
smpboot_restore_warm_reset_vector();
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 15/37] x86/smpboot: Remove wait for cpu_online()
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Now that the core code drops sparse_irq_lock after the idle thread
synchronized, it's pointless to wait for the AP to mark itself online.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/kernel/smpboot.c | 25 ++-----------------------
1 file changed, 2 insertions(+), 23 deletions(-)
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -1110,20 +1110,6 @@ static int wait_cpu_initialized(unsigned
return 0;
}
-/*
- * Bringup step three: Wait for the target AP to reach set_cpu_online() in
- * start_secondary().
- */
-static void wait_cpu_online(unsigned int cpu)
-{
- /*
- * Wait for the AP to mark itself online, so the core caller
- * can drop sparse_irq_lock.
- */
- while (!cpu_online(cpu))
- schedule();
-}
-
static int native_kick_ap(unsigned int cpu, struct task_struct *tidle)
{
int apicid = apic->cpu_present_to_apicid(cpu);
@@ -1170,16 +1156,9 @@ int native_cpu_up(unsigned int cpu, stru
int ret;
ret = native_kick_ap(cpu, tidle);
- if (ret)
- goto out;
-
- ret = wait_cpu_initialized(cpu);
- if (ret)
- goto out;
-
- wait_cpu_online(cpu);
+ if (!ret)
+ ret = wait_cpu_initialized(cpu);
-out:
/* Cleanup possible dangling ends... */
if (x86_platform.legacy.warm_reset)
smpboot_restore_warm_reset_vector();
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 15/37] x86/smpboot: Remove wait for cpu_online()
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Now that the core code drops sparse_irq_lock after the idle thread
synchronized, it's pointless to wait for the AP to mark itself online.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/kernel/smpboot.c | 25 ++-----------------------
1 file changed, 2 insertions(+), 23 deletions(-)
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -1110,20 +1110,6 @@ static int wait_cpu_initialized(unsigned
return 0;
}
-/*
- * Bringup step three: Wait for the target AP to reach set_cpu_online() in
- * start_secondary().
- */
-static void wait_cpu_online(unsigned int cpu)
-{
- /*
- * Wait for the AP to mark itself online, so the core caller
- * can drop sparse_irq_lock.
- */
- while (!cpu_online(cpu))
- schedule();
-}
-
static int native_kick_ap(unsigned int cpu, struct task_struct *tidle)
{
int apicid = apic->cpu_present_to_apicid(cpu);
@@ -1170,16 +1156,9 @@ int native_cpu_up(unsigned int cpu, stru
int ret;
ret = native_kick_ap(cpu, tidle);
- if (ret)
- goto out;
-
- ret = wait_cpu_initialized(cpu);
- if (ret)
- goto out;
-
- wait_cpu_online(cpu);
+ if (!ret)
+ ret = wait_cpu_initialized(cpu);
-out:
/* Cleanup possible dangling ends... */
if (x86_platform.legacy.warm_reset)
smpboot_restore_warm_reset_vector();
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 16/37] x86/xen/smp_pv: Remove wait for CPU online
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-14 23:44 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Juergen Gross,
Boris Ostrovsky, xen-devel, David Woodhouse, Usama Arif,
Russell King, Arnd Bergmann, linux-arm-kernel, Catalin Marinas,
Will Deacon, Guo Ren, linux-csky, Thomas Bogendoerfer,
linux-mips, James E.J. Bottomley, Helge Deller, linux-parisc,
Paul Walmsley, Palmer Dabbelt, linux-riscv, Mark Rutland,
Sabin Rapan
Now that the core code drops sparse_irq_lock after the idle thread
synchronized, it's pointless to wait for the AP to mark itself online.
Whether the control CPU runs in a wait loop or sleeps in the core code
waiting for the online operation to complete makes no difference.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Juergen Gross <jgross@suse.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: xen-devel@lists.xenproject.org
---
arch/x86/xen/smp_pv.c | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
--- a/arch/x86/xen/smp_pv.c
+++ b/arch/x86/xen/smp_pv.c
@@ -340,11 +340,11 @@ static int xen_pv_cpu_up(unsigned int cp
xen_pmu_init(cpu);
- rc = HYPERVISOR_vcpu_op(VCPUOP_up, xen_vcpu_nr(cpu), NULL);
- BUG_ON(rc);
-
- while (cpu_report_state(cpu) != CPU_ONLINE)
- HYPERVISOR_sched_op(SCHEDOP_yield, NULL);
+ /*
+ * Why is this a BUG? If the hypercall fails then everything can be
+ * rolled back, no?
+ */
+ BUG_ON(HYPERVISOR_vcpu_op(VCPUOP_up, xen_vcpu_nr(cpu), NULL));
return 0;
}
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 16/37] x86/xen/smp_pv: Remove wait for CPU online
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Juergen Gross,
Boris Ostrovsky, xen-devel, David Woodhouse, Usama Arif,
Russell King, Arnd Bergmann, linux-arm-kernel, Catalin Marinas,
Will Deacon, Guo Ren, linux-csky, Thomas Bogendoerfer,
linux-mips, James E.J. Bottomley, Helge Deller, linux-parisc,
Paul Walmsley, Palmer Dabbelt, linux-riscv, Mark Rutland,
Sabin Rapan
Now that the core code drops sparse_irq_lock after the idle thread
synchronized, it's pointless to wait for the AP to mark itself online.
Whether the control CPU runs in a wait loop or sleeps in the core code
waiting for the online operation to complete makes no difference.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Juergen Gross <jgross@suse.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: xen-devel@lists.xenproject.org
---
arch/x86/xen/smp_pv.c | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
--- a/arch/x86/xen/smp_pv.c
+++ b/arch/x86/xen/smp_pv.c
@@ -340,11 +340,11 @@ static int xen_pv_cpu_up(unsigned int cp
xen_pmu_init(cpu);
- rc = HYPERVISOR_vcpu_op(VCPUOP_up, xen_vcpu_nr(cpu), NULL);
- BUG_ON(rc);
-
- while (cpu_report_state(cpu) != CPU_ONLINE)
- HYPERVISOR_sched_op(SCHEDOP_yield, NULL);
+ /*
+ * Why is this a BUG? If the hypercall fails then everything can be
+ * rolled back, no?
+ */
+ BUG_ON(HYPERVISOR_vcpu_op(VCPUOP_up, xen_vcpu_nr(cpu), NULL));
return 0;
}
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 16/37] x86/xen/smp_pv: Remove wait for CPU online
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Juergen Gross,
Boris Ostrovsky, xen-devel, David Woodhouse, Usama Arif,
Russell King, Arnd Bergmann, linux-arm-kernel, Catalin Marinas,
Will Deacon, Guo Ren, linux-csky, Thomas Bogendoerfer,
linux-mips, James E.J. Bottomley, Helge Deller, linux-parisc,
Paul Walmsley, Palmer Dabbelt, linux-riscv, Mark Rutland,
Sabin Rapan
Now that the core code drops sparse_irq_lock after the idle thread
synchronized, it's pointless to wait for the AP to mark itself online.
Whether the control CPU runs in a wait loop or sleeps in the core code
waiting for the online operation to complete makes no difference.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Juergen Gross <jgross@suse.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: xen-devel@lists.xenproject.org
---
arch/x86/xen/smp_pv.c | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
--- a/arch/x86/xen/smp_pv.c
+++ b/arch/x86/xen/smp_pv.c
@@ -340,11 +340,11 @@ static int xen_pv_cpu_up(unsigned int cp
xen_pmu_init(cpu);
- rc = HYPERVISOR_vcpu_op(VCPUOP_up, xen_vcpu_nr(cpu), NULL);
- BUG_ON(rc);
-
- while (cpu_report_state(cpu) != CPU_ONLINE)
- HYPERVISOR_sched_op(SCHEDOP_yield, NULL);
+ /*
+ * Why is this a BUG? If the hypercall fails then everything can be
+ * rolled back, no?
+ */
+ BUG_ON(HYPERVISOR_vcpu_op(VCPUOP_up, xen_vcpu_nr(cpu), NULL));
return 0;
}
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 17/37] x86/xen/hvm: Get rid of DEAD_FROZEN handling
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-14 23:44 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Juergen Gross,
Boris Ostrovsky, xen-devel, David Woodhouse, Usama Arif,
Russell King, Arnd Bergmann, linux-arm-kernel, Catalin Marinas,
Will Deacon, Guo Ren, linux-csky, Thomas Bogendoerfer,
linux-mips, James E.J. Bottomley, Helge Deller, linux-parisc,
Paul Walmsley, Palmer Dabbelt, linux-riscv, Mark Rutland,
Sabin Rapan
No point in this conditional voodoo. Un-initializing the lock mechanism is
safe to be called unconditionally even if it was already invoked when the
CPU died.
Remove the invocation of xen_smp_intr_free() as that has been already
cleaned up in xen_cpu_dead_hvm().
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Juergen Gross <jgross@suse.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: xen-devel@lists.xenproject.org
---
arch/x86/xen/enlighten_hvm.c | 11 +++++------
1 file changed, 5 insertions(+), 6 deletions(-)
--- a/arch/x86/xen/enlighten_hvm.c
+++ b/arch/x86/xen/enlighten_hvm.c
@@ -161,13 +161,12 @@ static int xen_cpu_up_prepare_hvm(unsign
int rc = 0;
/*
- * This can happen if CPU was offlined earlier and
- * offlining timed out in common_cpu_die().
+ * If a CPU was offlined earlier and offlining timed out then the
+ * lock mechanism is still initialized. Uninit it unconditionally
+ * as it's safe to call even if already uninited. Interrupts and
+ * timer have already been handled in xen_cpu_dead_hvm().
*/
- if (cpu_report_state(cpu) == CPU_DEAD_FROZEN) {
- xen_smp_intr_free(cpu);
- xen_uninit_lock_cpu(cpu);
- }
+ xen_uninit_lock_cpu(cpu);
if (cpu_acpi_id(cpu) != U32_MAX)
per_cpu(xen_vcpu_id, cpu) = cpu_acpi_id(cpu);
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 17/37] x86/xen/hvm: Get rid of DEAD_FROZEN handling
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Juergen Gross,
Boris Ostrovsky, xen-devel, David Woodhouse, Usama Arif,
Russell King, Arnd Bergmann, linux-arm-kernel, Catalin Marinas,
Will Deacon, Guo Ren, linux-csky, Thomas Bogendoerfer,
linux-mips, James E.J. Bottomley, Helge Deller, linux-parisc,
Paul Walmsley, Palmer Dabbelt, linux-riscv, Mark Rutland,
Sabin Rapan
No point in this conditional voodoo. Un-initializing the lock mechanism is
safe to be called unconditionally even if it was already invoked when the
CPU died.
Remove the invocation of xen_smp_intr_free() as that has been already
cleaned up in xen_cpu_dead_hvm().
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Juergen Gross <jgross@suse.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: xen-devel@lists.xenproject.org
---
arch/x86/xen/enlighten_hvm.c | 11 +++++------
1 file changed, 5 insertions(+), 6 deletions(-)
--- a/arch/x86/xen/enlighten_hvm.c
+++ b/arch/x86/xen/enlighten_hvm.c
@@ -161,13 +161,12 @@ static int xen_cpu_up_prepare_hvm(unsign
int rc = 0;
/*
- * This can happen if CPU was offlined earlier and
- * offlining timed out in common_cpu_die().
+ * If a CPU was offlined earlier and offlining timed out then the
+ * lock mechanism is still initialized. Uninit it unconditionally
+ * as it's safe to call even if already uninited. Interrupts and
+ * timer have already been handled in xen_cpu_dead_hvm().
*/
- if (cpu_report_state(cpu) == CPU_DEAD_FROZEN) {
- xen_smp_intr_free(cpu);
- xen_uninit_lock_cpu(cpu);
- }
+ xen_uninit_lock_cpu(cpu);
if (cpu_acpi_id(cpu) != U32_MAX)
per_cpu(xen_vcpu_id, cpu) = cpu_acpi_id(cpu);
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 17/37] x86/xen/hvm: Get rid of DEAD_FROZEN handling
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Juergen Gross,
Boris Ostrovsky, xen-devel, David Woodhouse, Usama Arif,
Russell King, Arnd Bergmann, linux-arm-kernel, Catalin Marinas,
Will Deacon, Guo Ren, linux-csky, Thomas Bogendoerfer,
linux-mips, James E.J. Bottomley, Helge Deller, linux-parisc,
Paul Walmsley, Palmer Dabbelt, linux-riscv, Mark Rutland,
Sabin Rapan
No point in this conditional voodoo. Un-initializing the lock mechanism is
safe to be called unconditionally even if it was already invoked when the
CPU died.
Remove the invocation of xen_smp_intr_free() as that has been already
cleaned up in xen_cpu_dead_hvm().
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Juergen Gross <jgross@suse.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: xen-devel@lists.xenproject.org
---
arch/x86/xen/enlighten_hvm.c | 11 +++++------
1 file changed, 5 insertions(+), 6 deletions(-)
--- a/arch/x86/xen/enlighten_hvm.c
+++ b/arch/x86/xen/enlighten_hvm.c
@@ -161,13 +161,12 @@ static int xen_cpu_up_prepare_hvm(unsign
int rc = 0;
/*
- * This can happen if CPU was offlined earlier and
- * offlining timed out in common_cpu_die().
+ * If a CPU was offlined earlier and offlining timed out then the
+ * lock mechanism is still initialized. Uninit it unconditionally
+ * as it's safe to call even if already uninited. Interrupts and
+ * timer have already been handled in xen_cpu_dead_hvm().
*/
- if (cpu_report_state(cpu) == CPU_DEAD_FROZEN) {
- xen_smp_intr_free(cpu);
- xen_uninit_lock_cpu(cpu);
- }
+ xen_uninit_lock_cpu(cpu);
if (cpu_acpi_id(cpu) != U32_MAX)
per_cpu(xen_vcpu_id, cpu) = cpu_acpi_id(cpu);
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 18/37] cpu/hotplug: Add CPU state tracking and synchronization
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-14 23:44 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
The CPU state tracking and synchronization mechanism in smpboot.c is
completely independent of the hotplug code and all logic around it is
implemented in architecture specific code.
Except for the state reporting of the AP there is absolutely nothing
architecture specific and the sychronization and decision functions can be
moved into the generic hotplug core code.
Provide an integrated variant and add the core synchronization and decision
points. This comes in two flavours:
1) DEAD state synchronization
Updated by the architecture code once the AP reaches the point where
it is ready to be torn down by the control CPU, e.g. by removing power
or clocks or tear down via the hypervisor.
The control CPU waits for this state to be reached with a timeout. If
the state is reached an architecture specific cleanup function is
invoked.
2) Full state synchronization
This extends #1 with AP alive synchronization. This is new
functionality, which allows to replace architecture specific wait
mechanims, e.g. cpumasks, completely.
It also prevents that an AP which is in a limbo state can be brought
up again. This can happen when an AP failed to report dead state
during a previous off-line operation.
The dead synchronization is what most architectures use. Only x86 makes a
bringup decision based on that state at the moment.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/Kconfig | 15 +++
include/linux/cpuhotplug.h | 12 ++
kernel/cpu.c | 193 ++++++++++++++++++++++++++++++++++++++++++++-
kernel/smpboot.c | 2
4 files changed, 221 insertions(+), 1 deletion(-)
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -34,6 +34,21 @@ config ARCH_HAS_SUBPAGE_FAULTS
config HOTPLUG_SMT
bool
+# Selected by HOTPLUG_CORE_SYNC_DEAD or HOTPLUG_CORE_SYNC_FULL
+config HOTPLUG_CORE_SYNC
+ bool
+
+# Basic CPU dead synchronization selected by architecture
+config HOTPLUG_CORE_SYNC_DEAD
+ bool
+ select HOTPLUG_CORE_SYNC
+
+# Full CPU synchronization with alive state selected by architecture
+config HOTPLUG_CORE_SYNC_FULL
+ bool
+ select HOTPLUG_CORE_SYNC_DEAD if HOTPLUG_CPU
+ select HOTPLUG_CORE_SYNC
+
config GENERIC_ENTRY
bool
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -519,4 +519,16 @@ void cpuhp_online_idle(enum cpuhp_state
static inline void cpuhp_online_idle(enum cpuhp_state state) { }
#endif
+void cpuhp_ap_sync_alive(void);
+void arch_cpuhp_sync_state_poll(void);
+void arch_cpuhp_cleanup_kick_cpu(unsigned int cpu);
+
+#ifdef CONFIG_HOTPLUG_CORE_SYNC_DEAD
+void cpuhp_ap_report_dead(void);
+void arch_cpuhp_cleanup_dead_cpu(unsigned int cpu);
+#else
+static inline void cpuhp_ap_report_dead(void) { }
+static inline void arch_cpuhp_cleanup_dead_cpu(unsigned int cpu) { }
+#endif
+
#endif
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -17,6 +17,7 @@
#include <linux/cpu.h>
#include <linux/oom.h>
#include <linux/rcupdate.h>
+#include <linux/delay.h>
#include <linux/export.h>
#include <linux/bug.h>
#include <linux/kthread.h>
@@ -59,6 +60,7 @@
* @last: For multi-instance rollback, remember how far we got
* @cb_state: The state for a single callback (install/uninstall)
* @result: Result of the operation
+ * @ap_sync_state: State for AP synchronization
* @done_up: Signal completion to the issuer of the task for cpu-up
* @done_down: Signal completion to the issuer of the task for cpu-down
*/
@@ -76,6 +78,7 @@ struct cpuhp_cpu_state {
struct hlist_node *last;
enum cpuhp_state cb_state;
int result;
+ atomic_t ap_sync_state;
struct completion done_up;
struct completion done_down;
#endif
@@ -276,6 +279,182 @@ static bool cpuhp_is_atomic_state(enum c
return CPUHP_AP_IDLE_DEAD <= state && state < CPUHP_AP_ONLINE;
}
+/* Synchronization state management */
+enum cpuhp_sync_state {
+ SYNC_STATE_DEAD,
+ SYNC_STATE_KICKED,
+ SYNC_STATE_SHOULD_DIE,
+ SYNC_STATE_ALIVE,
+ SYNC_STATE_SHOULD_ONLINE,
+ SYNC_STATE_ONLINE,
+};
+
+#ifdef CONFIG_HOTPLUG_CORE_SYNC
+/**
+ * cpuhp_ap_update_sync_state - Update synchronization state during bringup/teardown
+ * @state: The synchronization state to set
+ *
+ * No synchronization point. Just update of the synchronization state.
+ */
+static inline void cpuhp_ap_update_sync_state(enum cpuhp_sync_state state)
+{
+ atomic_t *st = this_cpu_ptr(&cpuhp_state.ap_sync_state);
+ int sync = atomic_read(st);
+
+ while (!atomic_try_cmpxchg(st, &sync, state));
+}
+
+void __weak arch_cpuhp_sync_state_poll(void) { cpu_relax(); }
+
+static bool cpuhp_wait_for_sync_state(unsigned int cpu, enum cpuhp_sync_state state,
+ enum cpuhp_sync_state next_state)
+{
+ atomic_t *st = per_cpu_ptr(&cpuhp_state.ap_sync_state, cpu);
+ ktime_t now, end, start = ktime_get();
+ int sync;
+
+ end = start + 10ULL * NSEC_PER_SEC;
+
+ sync = atomic_read(st);
+ while (1) {
+ if (sync == state) {
+ if (!atomic_try_cmpxchg(st, &sync, next_state))
+ continue;
+ return true;
+ }
+
+ now = ktime_get();
+ if (now > end) {
+ /* Timeout. Leave the state unchanged */
+ return false;
+ } else if (now - start < NSEC_PER_MSEC) {
+ /* Poll for one millisecond */
+ arch_cpuhp_sync_state_poll();
+ } else {
+ usleep_range_state(USEC_PER_MSEC, 2 * USEC_PER_MSEC, TASK_UNINTERRUPTIBLE);
+ }
+ sync = atomic_read(st);
+ }
+ return true;
+}
+#else /* CONFIG_HOTPLUG_CORE_SYNC */
+static inline void cpuhp_ap_update_sync_state(enum cpuhp_sync_state state) { }
+#endif /* !CONFIG_HOTPLUG_CORE_SYNC */
+
+#ifdef CONFIG_HOTPLUG_CORE_SYNC_DEAD
+/**
+ * cpuhp_ap_report_dead - Update synchronization state to DEAD
+ *
+ * No synchronization point. Just update of the synchronization state.
+ */
+void cpuhp_ap_report_dead(void)
+{
+ cpuhp_ap_update_sync_state(SYNC_STATE_DEAD);
+}
+
+void __weak arch_cpuhp_cleanup_dead_cpu(unsigned int cpu) { }
+
+/*
+ * Late CPU shutdown synchronization point. Cannot use cpuhp_state::done_down
+ * because the AP cannot issue complete() at this stage.
+ */
+static void cpuhp_bp_sync_dead(unsigned int cpu)
+{
+ atomic_t *st = per_cpu_ptr(&cpuhp_state.ap_sync_state, cpu);
+ int sync = atomic_read(st);
+
+ do {
+ /* CPU can have reported dead already. Don't overwrite that! */
+ if (sync == SYNC_STATE_DEAD)
+ break;
+ } while (!atomic_try_cmpxchg(st, &sync, SYNC_STATE_SHOULD_DIE));
+
+ if (cpuhp_wait_for_sync_state(cpu, SYNC_STATE_DEAD, SYNC_STATE_DEAD)) {
+ /* CPU reached dead state. Invoke the cleanup function */
+ arch_cpuhp_cleanup_dead_cpu(cpu);
+ return;
+ }
+
+ /* No further action possible. Emit message and give up. */
+ pr_err("CPU%u failed to report dead state\n", cpu);
+}
+#else /* CONFIG_HOTPLUG_CORE_SYNC_DEAD */
+static inline void cpuhp_bp_sync_dead(unsigned int cpu) { }
+#endif /* !CONFIG_HOTPLUG_CORE_SYNC_DEAD */
+
+#ifdef CONFIG_HOTPLUG_CORE_SYNC_FULL
+/**
+ * cpuhp_ap_sync_alive - Synchronize AP with the control CPU once it is alive
+ *
+ * Updates the AP synchronization state to SYNC_STATE_ALIVE and waits
+ * for the BP to release it.
+ */
+void cpuhp_ap_sync_alive(void)
+{
+ atomic_t *st = this_cpu_ptr(&cpuhp_state.ap_sync_state);
+
+ cpuhp_ap_update_sync_state(SYNC_STATE_ALIVE);
+
+ /* Wait for the control CPU to release it. */
+ while (atomic_read(st) != SYNC_STATE_SHOULD_ONLINE)
+ cpu_relax();
+}
+
+static bool cpuhp_can_boot_ap(unsigned int cpu)
+{
+ atomic_t *st = per_cpu_ptr(&cpuhp_state.ap_sync_state, cpu);
+ int sync = atomic_read(st);
+
+again:
+ switch (sync) {
+ case SYNC_STATE_DEAD:
+ /* CPU is properly dead */
+ break;
+ case SYNC_STATE_KICKED:
+ /* CPU did not come up in previous attempt */
+ break;
+ case SYNC_STATE_ALIVE:
+ /* CPU is stuck cpuhp_ap_sync_alive(). */
+ break;
+ default:
+ /* CPU failed to report online or dead and is in limbo state. */
+ return false;
+ }
+
+ /* Prepare for booting */
+ if (!atomic_try_cmpxchg(st, &sync, SYNC_STATE_KICKED))
+ goto again;
+
+ return true;
+}
+
+void __weak arch_cpuhp_cleanup_kick_cpu(unsigned int cpu) { }
+
+/*
+ * Early CPU bringup synchronization point. Cannot use cpuhp_state::done_up
+ * because the AP cannot issue complete() so early in the bringup.
+ */
+static int cpuhp_bp_sync_alive(unsigned int cpu)
+{
+ int ret = 0;
+
+ if (!IS_ENABLED(CONFIG_HOTPLUG_CORE_SYNC_FULL))
+ return 0;
+
+ if (!cpuhp_wait_for_sync_state(cpu, SYNC_STATE_ALIVE, SYNC_STATE_SHOULD_ONLINE)) {
+ pr_err("CPU%u failed to report alive state\n", cpu);
+ ret = -EIO;
+ }
+
+ /* Let the architecture cleanup the kick alive mechanics. */
+ arch_cpuhp_cleanup_kick_cpu(cpu);
+ return ret;
+}
+#else /* CONFIG_HOTPLUG_CORE_SYNC_FULL */
+static inline int cpuhp_bp_sync_alive(unsigned int cpu) { return 0; }
+static inline bool cpuhp_can_boot_ap(unsigned int cpu) { return true; }
+#endif /* !CONFIG_HOTPLUG_CORE_SYNC_FULL */
+
/* Serializes the updates to cpu_online_mask, cpu_present_mask */
static DEFINE_MUTEX(cpu_add_remove_lock);
bool cpuhp_tasks_frozen;
@@ -588,6 +767,9 @@ static int bringup_cpu(unsigned int cpu)
struct task_struct *idle = idle_thread_get(cpu);
int ret;
+ if (!cpuhp_can_boot_ap(cpu))
+ return -EAGAIN;
+
/*
* Reset stale stack state from the last time this CPU was online.
*/
@@ -606,6 +788,10 @@ static int bringup_cpu(unsigned int cpu)
if (ret)
goto out_unlock;
+ ret = cpuhp_bp_sync_alive(cpu);
+ if (ret)
+ goto out_unlock;
+
ret = bringup_wait_for_ap_online(cpu);
if (ret)
goto out_unlock;
@@ -1109,6 +1295,8 @@ static int takedown_cpu(unsigned int cpu
/* This actually kills the CPU. */
__cpu_die(cpu);
+ cpuhp_bp_sync_dead(cpu);
+
tick_cleanup_dead_cpu(cpu);
rcutree_migrate_callbacks(cpu);
return 0;
@@ -1355,8 +1543,10 @@ void cpuhp_online_idle(enum cpuhp_state
if (state != CPUHP_AP_ONLINE_IDLE)
return;
+ cpuhp_ap_update_sync_state(SYNC_STATE_ONLINE);
+
/*
- * Unpart the stopper thread before we start the idle loop (and start
+ * Unpark the stopper thread before we start the idle loop (and start
* scheduling); this ensures the stopper task is always available.
*/
stop_machine_unpark(smp_processor_id());
@@ -2722,6 +2912,7 @@ void __init boot_cpu_hotplug_init(void)
{
#ifdef CONFIG_SMP
cpumask_set_cpu(smp_processor_id(), &cpus_booted_once_mask);
+ atomic_set(this_cpu_ptr(&cpuhp_state.ap_sync_state), SYNC_STATE_ONLINE);
#endif
this_cpu_write(cpuhp_state.state, CPUHP_ONLINE);
this_cpu_write(cpuhp_state.target, CPUHP_ONLINE);
--- a/kernel/smpboot.c
+++ b/kernel/smpboot.c
@@ -326,6 +326,7 @@ void smpboot_unregister_percpu_thread(st
}
EXPORT_SYMBOL_GPL(smpboot_unregister_percpu_thread);
+#ifndef CONFIG_HOTPLUG_CORE_SYNC
static DEFINE_PER_CPU(atomic_t, cpu_hotplug_state) = ATOMIC_INIT(CPU_POST_DEAD);
/*
@@ -488,3 +489,4 @@ bool cpu_report_death(void)
}
#endif /* #ifdef CONFIG_HOTPLUG_CPU */
+#endif /* !CONFIG_HOTPLUG_CORE_SYNC */
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 18/37] cpu/hotplug: Add CPU state tracking and synchronization
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
The CPU state tracking and synchronization mechanism in smpboot.c is
completely independent of the hotplug code and all logic around it is
implemented in architecture specific code.
Except for the state reporting of the AP there is absolutely nothing
architecture specific and the sychronization and decision functions can be
moved into the generic hotplug core code.
Provide an integrated variant and add the core synchronization and decision
points. This comes in two flavours:
1) DEAD state synchronization
Updated by the architecture code once the AP reaches the point where
it is ready to be torn down by the control CPU, e.g. by removing power
or clocks or tear down via the hypervisor.
The control CPU waits for this state to be reached with a timeout. If
the state is reached an architecture specific cleanup function is
invoked.
2) Full state synchronization
This extends #1 with AP alive synchronization. This is new
functionality, which allows to replace architecture specific wait
mechanims, e.g. cpumasks, completely.
It also prevents that an AP which is in a limbo state can be brought
up again. This can happen when an AP failed to report dead state
during a previous off-line operation.
The dead synchronization is what most architectures use. Only x86 makes a
bringup decision based on that state at the moment.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/Kconfig | 15 +++
include/linux/cpuhotplug.h | 12 ++
kernel/cpu.c | 193 ++++++++++++++++++++++++++++++++++++++++++++-
kernel/smpboot.c | 2
4 files changed, 221 insertions(+), 1 deletion(-)
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -34,6 +34,21 @@ config ARCH_HAS_SUBPAGE_FAULTS
config HOTPLUG_SMT
bool
+# Selected by HOTPLUG_CORE_SYNC_DEAD or HOTPLUG_CORE_SYNC_FULL
+config HOTPLUG_CORE_SYNC
+ bool
+
+# Basic CPU dead synchronization selected by architecture
+config HOTPLUG_CORE_SYNC_DEAD
+ bool
+ select HOTPLUG_CORE_SYNC
+
+# Full CPU synchronization with alive state selected by architecture
+config HOTPLUG_CORE_SYNC_FULL
+ bool
+ select HOTPLUG_CORE_SYNC_DEAD if HOTPLUG_CPU
+ select HOTPLUG_CORE_SYNC
+
config GENERIC_ENTRY
bool
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -519,4 +519,16 @@ void cpuhp_online_idle(enum cpuhp_state
static inline void cpuhp_online_idle(enum cpuhp_state state) { }
#endif
+void cpuhp_ap_sync_alive(void);
+void arch_cpuhp_sync_state_poll(void);
+void arch_cpuhp_cleanup_kick_cpu(unsigned int cpu);
+
+#ifdef CONFIG_HOTPLUG_CORE_SYNC_DEAD
+void cpuhp_ap_report_dead(void);
+void arch_cpuhp_cleanup_dead_cpu(unsigned int cpu);
+#else
+static inline void cpuhp_ap_report_dead(void) { }
+static inline void arch_cpuhp_cleanup_dead_cpu(unsigned int cpu) { }
+#endif
+
#endif
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -17,6 +17,7 @@
#include <linux/cpu.h>
#include <linux/oom.h>
#include <linux/rcupdate.h>
+#include <linux/delay.h>
#include <linux/export.h>
#include <linux/bug.h>
#include <linux/kthread.h>
@@ -59,6 +60,7 @@
* @last: For multi-instance rollback, remember how far we got
* @cb_state: The state for a single callback (install/uninstall)
* @result: Result of the operation
+ * @ap_sync_state: State for AP synchronization
* @done_up: Signal completion to the issuer of the task for cpu-up
* @done_down: Signal completion to the issuer of the task for cpu-down
*/
@@ -76,6 +78,7 @@ struct cpuhp_cpu_state {
struct hlist_node *last;
enum cpuhp_state cb_state;
int result;
+ atomic_t ap_sync_state;
struct completion done_up;
struct completion done_down;
#endif
@@ -276,6 +279,182 @@ static bool cpuhp_is_atomic_state(enum c
return CPUHP_AP_IDLE_DEAD <= state && state < CPUHP_AP_ONLINE;
}
+/* Synchronization state management */
+enum cpuhp_sync_state {
+ SYNC_STATE_DEAD,
+ SYNC_STATE_KICKED,
+ SYNC_STATE_SHOULD_DIE,
+ SYNC_STATE_ALIVE,
+ SYNC_STATE_SHOULD_ONLINE,
+ SYNC_STATE_ONLINE,
+};
+
+#ifdef CONFIG_HOTPLUG_CORE_SYNC
+/**
+ * cpuhp_ap_update_sync_state - Update synchronization state during bringup/teardown
+ * @state: The synchronization state to set
+ *
+ * No synchronization point. Just update of the synchronization state.
+ */
+static inline void cpuhp_ap_update_sync_state(enum cpuhp_sync_state state)
+{
+ atomic_t *st = this_cpu_ptr(&cpuhp_state.ap_sync_state);
+ int sync = atomic_read(st);
+
+ while (!atomic_try_cmpxchg(st, &sync, state));
+}
+
+void __weak arch_cpuhp_sync_state_poll(void) { cpu_relax(); }
+
+static bool cpuhp_wait_for_sync_state(unsigned int cpu, enum cpuhp_sync_state state,
+ enum cpuhp_sync_state next_state)
+{
+ atomic_t *st = per_cpu_ptr(&cpuhp_state.ap_sync_state, cpu);
+ ktime_t now, end, start = ktime_get();
+ int sync;
+
+ end = start + 10ULL * NSEC_PER_SEC;
+
+ sync = atomic_read(st);
+ while (1) {
+ if (sync == state) {
+ if (!atomic_try_cmpxchg(st, &sync, next_state))
+ continue;
+ return true;
+ }
+
+ now = ktime_get();
+ if (now > end) {
+ /* Timeout. Leave the state unchanged */
+ return false;
+ } else if (now - start < NSEC_PER_MSEC) {
+ /* Poll for one millisecond */
+ arch_cpuhp_sync_state_poll();
+ } else {
+ usleep_range_state(USEC_PER_MSEC, 2 * USEC_PER_MSEC, TASK_UNINTERRUPTIBLE);
+ }
+ sync = atomic_read(st);
+ }
+ return true;
+}
+#else /* CONFIG_HOTPLUG_CORE_SYNC */
+static inline void cpuhp_ap_update_sync_state(enum cpuhp_sync_state state) { }
+#endif /* !CONFIG_HOTPLUG_CORE_SYNC */
+
+#ifdef CONFIG_HOTPLUG_CORE_SYNC_DEAD
+/**
+ * cpuhp_ap_report_dead - Update synchronization state to DEAD
+ *
+ * No synchronization point. Just update of the synchronization state.
+ */
+void cpuhp_ap_report_dead(void)
+{
+ cpuhp_ap_update_sync_state(SYNC_STATE_DEAD);
+}
+
+void __weak arch_cpuhp_cleanup_dead_cpu(unsigned int cpu) { }
+
+/*
+ * Late CPU shutdown synchronization point. Cannot use cpuhp_state::done_down
+ * because the AP cannot issue complete() at this stage.
+ */
+static void cpuhp_bp_sync_dead(unsigned int cpu)
+{
+ atomic_t *st = per_cpu_ptr(&cpuhp_state.ap_sync_state, cpu);
+ int sync = atomic_read(st);
+
+ do {
+ /* CPU can have reported dead already. Don't overwrite that! */
+ if (sync == SYNC_STATE_DEAD)
+ break;
+ } while (!atomic_try_cmpxchg(st, &sync, SYNC_STATE_SHOULD_DIE));
+
+ if (cpuhp_wait_for_sync_state(cpu, SYNC_STATE_DEAD, SYNC_STATE_DEAD)) {
+ /* CPU reached dead state. Invoke the cleanup function */
+ arch_cpuhp_cleanup_dead_cpu(cpu);
+ return;
+ }
+
+ /* No further action possible. Emit message and give up. */
+ pr_err("CPU%u failed to report dead state\n", cpu);
+}
+#else /* CONFIG_HOTPLUG_CORE_SYNC_DEAD */
+static inline void cpuhp_bp_sync_dead(unsigned int cpu) { }
+#endif /* !CONFIG_HOTPLUG_CORE_SYNC_DEAD */
+
+#ifdef CONFIG_HOTPLUG_CORE_SYNC_FULL
+/**
+ * cpuhp_ap_sync_alive - Synchronize AP with the control CPU once it is alive
+ *
+ * Updates the AP synchronization state to SYNC_STATE_ALIVE and waits
+ * for the BP to release it.
+ */
+void cpuhp_ap_sync_alive(void)
+{
+ atomic_t *st = this_cpu_ptr(&cpuhp_state.ap_sync_state);
+
+ cpuhp_ap_update_sync_state(SYNC_STATE_ALIVE);
+
+ /* Wait for the control CPU to release it. */
+ while (atomic_read(st) != SYNC_STATE_SHOULD_ONLINE)
+ cpu_relax();
+}
+
+static bool cpuhp_can_boot_ap(unsigned int cpu)
+{
+ atomic_t *st = per_cpu_ptr(&cpuhp_state.ap_sync_state, cpu);
+ int sync = atomic_read(st);
+
+again:
+ switch (sync) {
+ case SYNC_STATE_DEAD:
+ /* CPU is properly dead */
+ break;
+ case SYNC_STATE_KICKED:
+ /* CPU did not come up in previous attempt */
+ break;
+ case SYNC_STATE_ALIVE:
+ /* CPU is stuck cpuhp_ap_sync_alive(). */
+ break;
+ default:
+ /* CPU failed to report online or dead and is in limbo state. */
+ return false;
+ }
+
+ /* Prepare for booting */
+ if (!atomic_try_cmpxchg(st, &sync, SYNC_STATE_KICKED))
+ goto again;
+
+ return true;
+}
+
+void __weak arch_cpuhp_cleanup_kick_cpu(unsigned int cpu) { }
+
+/*
+ * Early CPU bringup synchronization point. Cannot use cpuhp_state::done_up
+ * because the AP cannot issue complete() so early in the bringup.
+ */
+static int cpuhp_bp_sync_alive(unsigned int cpu)
+{
+ int ret = 0;
+
+ if (!IS_ENABLED(CONFIG_HOTPLUG_CORE_SYNC_FULL))
+ return 0;
+
+ if (!cpuhp_wait_for_sync_state(cpu, SYNC_STATE_ALIVE, SYNC_STATE_SHOULD_ONLINE)) {
+ pr_err("CPU%u failed to report alive state\n", cpu);
+ ret = -EIO;
+ }
+
+ /* Let the architecture cleanup the kick alive mechanics. */
+ arch_cpuhp_cleanup_kick_cpu(cpu);
+ return ret;
+}
+#else /* CONFIG_HOTPLUG_CORE_SYNC_FULL */
+static inline int cpuhp_bp_sync_alive(unsigned int cpu) { return 0; }
+static inline bool cpuhp_can_boot_ap(unsigned int cpu) { return true; }
+#endif /* !CONFIG_HOTPLUG_CORE_SYNC_FULL */
+
/* Serializes the updates to cpu_online_mask, cpu_present_mask */
static DEFINE_MUTEX(cpu_add_remove_lock);
bool cpuhp_tasks_frozen;
@@ -588,6 +767,9 @@ static int bringup_cpu(unsigned int cpu)
struct task_struct *idle = idle_thread_get(cpu);
int ret;
+ if (!cpuhp_can_boot_ap(cpu))
+ return -EAGAIN;
+
/*
* Reset stale stack state from the last time this CPU was online.
*/
@@ -606,6 +788,10 @@ static int bringup_cpu(unsigned int cpu)
if (ret)
goto out_unlock;
+ ret = cpuhp_bp_sync_alive(cpu);
+ if (ret)
+ goto out_unlock;
+
ret = bringup_wait_for_ap_online(cpu);
if (ret)
goto out_unlock;
@@ -1109,6 +1295,8 @@ static int takedown_cpu(unsigned int cpu
/* This actually kills the CPU. */
__cpu_die(cpu);
+ cpuhp_bp_sync_dead(cpu);
+
tick_cleanup_dead_cpu(cpu);
rcutree_migrate_callbacks(cpu);
return 0;
@@ -1355,8 +1543,10 @@ void cpuhp_online_idle(enum cpuhp_state
if (state != CPUHP_AP_ONLINE_IDLE)
return;
+ cpuhp_ap_update_sync_state(SYNC_STATE_ONLINE);
+
/*
- * Unpart the stopper thread before we start the idle loop (and start
+ * Unpark the stopper thread before we start the idle loop (and start
* scheduling); this ensures the stopper task is always available.
*/
stop_machine_unpark(smp_processor_id());
@@ -2722,6 +2912,7 @@ void __init boot_cpu_hotplug_init(void)
{
#ifdef CONFIG_SMP
cpumask_set_cpu(smp_processor_id(), &cpus_booted_once_mask);
+ atomic_set(this_cpu_ptr(&cpuhp_state.ap_sync_state), SYNC_STATE_ONLINE);
#endif
this_cpu_write(cpuhp_state.state, CPUHP_ONLINE);
this_cpu_write(cpuhp_state.target, CPUHP_ONLINE);
--- a/kernel/smpboot.c
+++ b/kernel/smpboot.c
@@ -326,6 +326,7 @@ void smpboot_unregister_percpu_thread(st
}
EXPORT_SYMBOL_GPL(smpboot_unregister_percpu_thread);
+#ifndef CONFIG_HOTPLUG_CORE_SYNC
static DEFINE_PER_CPU(atomic_t, cpu_hotplug_state) = ATOMIC_INIT(CPU_POST_DEAD);
/*
@@ -488,3 +489,4 @@ bool cpu_report_death(void)
}
#endif /* #ifdef CONFIG_HOTPLUG_CPU */
+#endif /* !CONFIG_HOTPLUG_CORE_SYNC */
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 18/37] cpu/hotplug: Add CPU state tracking and synchronization
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
The CPU state tracking and synchronization mechanism in smpboot.c is
completely independent of the hotplug code and all logic around it is
implemented in architecture specific code.
Except for the state reporting of the AP there is absolutely nothing
architecture specific and the sychronization and decision functions can be
moved into the generic hotplug core code.
Provide an integrated variant and add the core synchronization and decision
points. This comes in two flavours:
1) DEAD state synchronization
Updated by the architecture code once the AP reaches the point where
it is ready to be torn down by the control CPU, e.g. by removing power
or clocks or tear down via the hypervisor.
The control CPU waits for this state to be reached with a timeout. If
the state is reached an architecture specific cleanup function is
invoked.
2) Full state synchronization
This extends #1 with AP alive synchronization. This is new
functionality, which allows to replace architecture specific wait
mechanims, e.g. cpumasks, completely.
It also prevents that an AP which is in a limbo state can be brought
up again. This can happen when an AP failed to report dead state
during a previous off-line operation.
The dead synchronization is what most architectures use. Only x86 makes a
bringup decision based on that state at the moment.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/Kconfig | 15 +++
include/linux/cpuhotplug.h | 12 ++
kernel/cpu.c | 193 ++++++++++++++++++++++++++++++++++++++++++++-
kernel/smpboot.c | 2
4 files changed, 221 insertions(+), 1 deletion(-)
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -34,6 +34,21 @@ config ARCH_HAS_SUBPAGE_FAULTS
config HOTPLUG_SMT
bool
+# Selected by HOTPLUG_CORE_SYNC_DEAD or HOTPLUG_CORE_SYNC_FULL
+config HOTPLUG_CORE_SYNC
+ bool
+
+# Basic CPU dead synchronization selected by architecture
+config HOTPLUG_CORE_SYNC_DEAD
+ bool
+ select HOTPLUG_CORE_SYNC
+
+# Full CPU synchronization with alive state selected by architecture
+config HOTPLUG_CORE_SYNC_FULL
+ bool
+ select HOTPLUG_CORE_SYNC_DEAD if HOTPLUG_CPU
+ select HOTPLUG_CORE_SYNC
+
config GENERIC_ENTRY
bool
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -519,4 +519,16 @@ void cpuhp_online_idle(enum cpuhp_state
static inline void cpuhp_online_idle(enum cpuhp_state state) { }
#endif
+void cpuhp_ap_sync_alive(void);
+void arch_cpuhp_sync_state_poll(void);
+void arch_cpuhp_cleanup_kick_cpu(unsigned int cpu);
+
+#ifdef CONFIG_HOTPLUG_CORE_SYNC_DEAD
+void cpuhp_ap_report_dead(void);
+void arch_cpuhp_cleanup_dead_cpu(unsigned int cpu);
+#else
+static inline void cpuhp_ap_report_dead(void) { }
+static inline void arch_cpuhp_cleanup_dead_cpu(unsigned int cpu) { }
+#endif
+
#endif
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -17,6 +17,7 @@
#include <linux/cpu.h>
#include <linux/oom.h>
#include <linux/rcupdate.h>
+#include <linux/delay.h>
#include <linux/export.h>
#include <linux/bug.h>
#include <linux/kthread.h>
@@ -59,6 +60,7 @@
* @last: For multi-instance rollback, remember how far we got
* @cb_state: The state for a single callback (install/uninstall)
* @result: Result of the operation
+ * @ap_sync_state: State for AP synchronization
* @done_up: Signal completion to the issuer of the task for cpu-up
* @done_down: Signal completion to the issuer of the task for cpu-down
*/
@@ -76,6 +78,7 @@ struct cpuhp_cpu_state {
struct hlist_node *last;
enum cpuhp_state cb_state;
int result;
+ atomic_t ap_sync_state;
struct completion done_up;
struct completion done_down;
#endif
@@ -276,6 +279,182 @@ static bool cpuhp_is_atomic_state(enum c
return CPUHP_AP_IDLE_DEAD <= state && state < CPUHP_AP_ONLINE;
}
+/* Synchronization state management */
+enum cpuhp_sync_state {
+ SYNC_STATE_DEAD,
+ SYNC_STATE_KICKED,
+ SYNC_STATE_SHOULD_DIE,
+ SYNC_STATE_ALIVE,
+ SYNC_STATE_SHOULD_ONLINE,
+ SYNC_STATE_ONLINE,
+};
+
+#ifdef CONFIG_HOTPLUG_CORE_SYNC
+/**
+ * cpuhp_ap_update_sync_state - Update synchronization state during bringup/teardown
+ * @state: The synchronization state to set
+ *
+ * No synchronization point. Just update of the synchronization state.
+ */
+static inline void cpuhp_ap_update_sync_state(enum cpuhp_sync_state state)
+{
+ atomic_t *st = this_cpu_ptr(&cpuhp_state.ap_sync_state);
+ int sync = atomic_read(st);
+
+ while (!atomic_try_cmpxchg(st, &sync, state));
+}
+
+void __weak arch_cpuhp_sync_state_poll(void) { cpu_relax(); }
+
+static bool cpuhp_wait_for_sync_state(unsigned int cpu, enum cpuhp_sync_state state,
+ enum cpuhp_sync_state next_state)
+{
+ atomic_t *st = per_cpu_ptr(&cpuhp_state.ap_sync_state, cpu);
+ ktime_t now, end, start = ktime_get();
+ int sync;
+
+ end = start + 10ULL * NSEC_PER_SEC;
+
+ sync = atomic_read(st);
+ while (1) {
+ if (sync == state) {
+ if (!atomic_try_cmpxchg(st, &sync, next_state))
+ continue;
+ return true;
+ }
+
+ now = ktime_get();
+ if (now > end) {
+ /* Timeout. Leave the state unchanged */
+ return false;
+ } else if (now - start < NSEC_PER_MSEC) {
+ /* Poll for one millisecond */
+ arch_cpuhp_sync_state_poll();
+ } else {
+ usleep_range_state(USEC_PER_MSEC, 2 * USEC_PER_MSEC, TASK_UNINTERRUPTIBLE);
+ }
+ sync = atomic_read(st);
+ }
+ return true;
+}
+#else /* CONFIG_HOTPLUG_CORE_SYNC */
+static inline void cpuhp_ap_update_sync_state(enum cpuhp_sync_state state) { }
+#endif /* !CONFIG_HOTPLUG_CORE_SYNC */
+
+#ifdef CONFIG_HOTPLUG_CORE_SYNC_DEAD
+/**
+ * cpuhp_ap_report_dead - Update synchronization state to DEAD
+ *
+ * No synchronization point. Just update of the synchronization state.
+ */
+void cpuhp_ap_report_dead(void)
+{
+ cpuhp_ap_update_sync_state(SYNC_STATE_DEAD);
+}
+
+void __weak arch_cpuhp_cleanup_dead_cpu(unsigned int cpu) { }
+
+/*
+ * Late CPU shutdown synchronization point. Cannot use cpuhp_state::done_down
+ * because the AP cannot issue complete() at this stage.
+ */
+static void cpuhp_bp_sync_dead(unsigned int cpu)
+{
+ atomic_t *st = per_cpu_ptr(&cpuhp_state.ap_sync_state, cpu);
+ int sync = atomic_read(st);
+
+ do {
+ /* CPU can have reported dead already. Don't overwrite that! */
+ if (sync == SYNC_STATE_DEAD)
+ break;
+ } while (!atomic_try_cmpxchg(st, &sync, SYNC_STATE_SHOULD_DIE));
+
+ if (cpuhp_wait_for_sync_state(cpu, SYNC_STATE_DEAD, SYNC_STATE_DEAD)) {
+ /* CPU reached dead state. Invoke the cleanup function */
+ arch_cpuhp_cleanup_dead_cpu(cpu);
+ return;
+ }
+
+ /* No further action possible. Emit message and give up. */
+ pr_err("CPU%u failed to report dead state\n", cpu);
+}
+#else /* CONFIG_HOTPLUG_CORE_SYNC_DEAD */
+static inline void cpuhp_bp_sync_dead(unsigned int cpu) { }
+#endif /* !CONFIG_HOTPLUG_CORE_SYNC_DEAD */
+
+#ifdef CONFIG_HOTPLUG_CORE_SYNC_FULL
+/**
+ * cpuhp_ap_sync_alive - Synchronize AP with the control CPU once it is alive
+ *
+ * Updates the AP synchronization state to SYNC_STATE_ALIVE and waits
+ * for the BP to release it.
+ */
+void cpuhp_ap_sync_alive(void)
+{
+ atomic_t *st = this_cpu_ptr(&cpuhp_state.ap_sync_state);
+
+ cpuhp_ap_update_sync_state(SYNC_STATE_ALIVE);
+
+ /* Wait for the control CPU to release it. */
+ while (atomic_read(st) != SYNC_STATE_SHOULD_ONLINE)
+ cpu_relax();
+}
+
+static bool cpuhp_can_boot_ap(unsigned int cpu)
+{
+ atomic_t *st = per_cpu_ptr(&cpuhp_state.ap_sync_state, cpu);
+ int sync = atomic_read(st);
+
+again:
+ switch (sync) {
+ case SYNC_STATE_DEAD:
+ /* CPU is properly dead */
+ break;
+ case SYNC_STATE_KICKED:
+ /* CPU did not come up in previous attempt */
+ break;
+ case SYNC_STATE_ALIVE:
+ /* CPU is stuck cpuhp_ap_sync_alive(). */
+ break;
+ default:
+ /* CPU failed to report online or dead and is in limbo state. */
+ return false;
+ }
+
+ /* Prepare for booting */
+ if (!atomic_try_cmpxchg(st, &sync, SYNC_STATE_KICKED))
+ goto again;
+
+ return true;
+}
+
+void __weak arch_cpuhp_cleanup_kick_cpu(unsigned int cpu) { }
+
+/*
+ * Early CPU bringup synchronization point. Cannot use cpuhp_state::done_up
+ * because the AP cannot issue complete() so early in the bringup.
+ */
+static int cpuhp_bp_sync_alive(unsigned int cpu)
+{
+ int ret = 0;
+
+ if (!IS_ENABLED(CONFIG_HOTPLUG_CORE_SYNC_FULL))
+ return 0;
+
+ if (!cpuhp_wait_for_sync_state(cpu, SYNC_STATE_ALIVE, SYNC_STATE_SHOULD_ONLINE)) {
+ pr_err("CPU%u failed to report alive state\n", cpu);
+ ret = -EIO;
+ }
+
+ /* Let the architecture cleanup the kick alive mechanics. */
+ arch_cpuhp_cleanup_kick_cpu(cpu);
+ return ret;
+}
+#else /* CONFIG_HOTPLUG_CORE_SYNC_FULL */
+static inline int cpuhp_bp_sync_alive(unsigned int cpu) { return 0; }
+static inline bool cpuhp_can_boot_ap(unsigned int cpu) { return true; }
+#endif /* !CONFIG_HOTPLUG_CORE_SYNC_FULL */
+
/* Serializes the updates to cpu_online_mask, cpu_present_mask */
static DEFINE_MUTEX(cpu_add_remove_lock);
bool cpuhp_tasks_frozen;
@@ -588,6 +767,9 @@ static int bringup_cpu(unsigned int cpu)
struct task_struct *idle = idle_thread_get(cpu);
int ret;
+ if (!cpuhp_can_boot_ap(cpu))
+ return -EAGAIN;
+
/*
* Reset stale stack state from the last time this CPU was online.
*/
@@ -606,6 +788,10 @@ static int bringup_cpu(unsigned int cpu)
if (ret)
goto out_unlock;
+ ret = cpuhp_bp_sync_alive(cpu);
+ if (ret)
+ goto out_unlock;
+
ret = bringup_wait_for_ap_online(cpu);
if (ret)
goto out_unlock;
@@ -1109,6 +1295,8 @@ static int takedown_cpu(unsigned int cpu
/* This actually kills the CPU. */
__cpu_die(cpu);
+ cpuhp_bp_sync_dead(cpu);
+
tick_cleanup_dead_cpu(cpu);
rcutree_migrate_callbacks(cpu);
return 0;
@@ -1355,8 +1543,10 @@ void cpuhp_online_idle(enum cpuhp_state
if (state != CPUHP_AP_ONLINE_IDLE)
return;
+ cpuhp_ap_update_sync_state(SYNC_STATE_ONLINE);
+
/*
- * Unpart the stopper thread before we start the idle loop (and start
+ * Unpark the stopper thread before we start the idle loop (and start
* scheduling); this ensures the stopper task is always available.
*/
stop_machine_unpark(smp_processor_id());
@@ -2722,6 +2912,7 @@ void __init boot_cpu_hotplug_init(void)
{
#ifdef CONFIG_SMP
cpumask_set_cpu(smp_processor_id(), &cpus_booted_once_mask);
+ atomic_set(this_cpu_ptr(&cpuhp_state.ap_sync_state), SYNC_STATE_ONLINE);
#endif
this_cpu_write(cpuhp_state.state, CPUHP_ONLINE);
this_cpu_write(cpuhp_state.target, CPUHP_ONLINE);
--- a/kernel/smpboot.c
+++ b/kernel/smpboot.c
@@ -326,6 +326,7 @@ void smpboot_unregister_percpu_thread(st
}
EXPORT_SYMBOL_GPL(smpboot_unregister_percpu_thread);
+#ifndef CONFIG_HOTPLUG_CORE_SYNC
static DEFINE_PER_CPU(atomic_t, cpu_hotplug_state) = ATOMIC_INIT(CPU_POST_DEAD);
/*
@@ -488,3 +489,4 @@ bool cpu_report_death(void)
}
#endif /* #ifdef CONFIG_HOTPLUG_CPU */
+#endif /* !CONFIG_HOTPLUG_CORE_SYNC */
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 19/37] x86/smpboot: Switch to hotplug core state synchronization
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-14 23:44 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Juergen Gross,
Boris Ostrovsky, xen-devel, David Woodhouse, Usama Arif,
Russell King, Arnd Bergmann, linux-arm-kernel, Catalin Marinas,
Will Deacon, Guo Ren, linux-csky, Thomas Bogendoerfer,
linux-mips, James E.J. Bottomley, Helge Deller, linux-parisc,
Paul Walmsley, Palmer Dabbelt, linux-riscv, Mark Rutland,
Sabin Rapan
The new AP state tracking and synchronization mechanism in the CPU hotplug
core code allows to remove quite some x86 specific code:
1) The AP alive synchronization based on cpumasks
2) The decision whether an AP can be brought up again
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Juergen Gross <jgross@suse.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: xen-devel@lists.xenproject.org
---
arch/x86/Kconfig | 1
arch/x86/include/asm/smp.h | 7 +
arch/x86/kernel/smp.c | 1
arch/x86/kernel/smpboot.c | 159 ++++++++++-----------------------------------
arch/x86/xen/smp_hvm.c | 16 +---
arch/x86/xen/smp_pv.c | 39 ++++++-----
6 files changed, 72 insertions(+), 151 deletions(-)
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -272,6 +272,7 @@ config X86
select HAVE_UNSTABLE_SCHED_CLOCK
select HAVE_USER_RETURN_NOTIFIER
select HAVE_GENERIC_VDSO
+ select HOTPLUG_CORE_SYNC_FULL if SMP
select HOTPLUG_SMT if SMP
select IRQ_FORCED_THREADING
select NEED_PER_CPU_EMBED_FIRST_CHUNK
--- a/arch/x86/include/asm/smp.h
+++ b/arch/x86/include/asm/smp.h
@@ -38,6 +38,8 @@ struct smp_ops {
void (*crash_stop_other_cpus)(void);
void (*smp_send_reschedule)(int cpu);
+ void (*cleanup_dead_cpu)(unsigned cpu);
+ void (*poll_sync_state)(void);
int (*cpu_up)(unsigned cpu, struct task_struct *tidle);
int (*cpu_disable)(void);
void (*cpu_die)(unsigned int cpu);
@@ -90,7 +92,8 @@ static inline int __cpu_disable(void)
static inline void __cpu_die(unsigned int cpu)
{
- smp_ops.cpu_die(cpu);
+ if (smp_ops.cpu_die)
+ smp_ops.cpu_die(cpu);
}
static inline void play_dead(void)
@@ -122,8 +125,6 @@ void native_smp_cpus_done(unsigned int m
int common_cpu_up(unsigned int cpunum, struct task_struct *tidle);
int native_cpu_up(unsigned int cpunum, struct task_struct *tidle);
int native_cpu_disable(void);
-int common_cpu_die(unsigned int cpu);
-void native_cpu_die(unsigned int cpu);
void hlt_play_dead(void);
void native_play_dead(void);
void play_dead_common(void);
--- a/arch/x86/kernel/smp.c
+++ b/arch/x86/kernel/smp.c
@@ -269,7 +269,6 @@ struct smp_ops smp_ops = {
.smp_send_reschedule = native_smp_send_reschedule,
.cpu_up = native_cpu_up,
- .cpu_die = native_cpu_die,
.cpu_disable = native_cpu_disable,
.play_dead = native_play_dead,
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -57,6 +57,7 @@
#include <linux/pgtable.h>
#include <linux/overflow.h>
#include <linux/stackprotector.h>
+#include <linux/cpuhotplug.h>
#include <asm/acpi.h>
#include <asm/cacheinfo.h>
@@ -101,9 +102,6 @@ EXPORT_PER_CPU_SYMBOL(cpu_die_map);
DEFINE_PER_CPU_READ_MOSTLY(struct cpuinfo_x86, cpu_info);
EXPORT_PER_CPU_SYMBOL(cpu_info);
-/* All of these masks are initialized in setup_cpu_local_masks() */
-static cpumask_var_t cpu_initialized_mask;
-static cpumask_var_t cpu_callout_mask;
/* Representing CPUs for which sibling maps can be computed */
static cpumask_var_t cpu_sibling_setup_mask;
@@ -169,8 +167,8 @@ static void smp_callin(void)
int cpuid = smp_processor_id();
/*
- * If waken up by an INIT in an 82489DX configuration
- * cpu_callout_mask guarantees we don't get here before an
+ * If waken up by an INIT in an 82489DX configuration the alive
+ * synchronization guarantees we don't get here before an
* INIT_deassert IPI reaches our local APIC, so it is now safe to
* touch our local APIC.
*
@@ -216,17 +214,6 @@ static void ap_calibrate_delay(void)
cpu_data(smp_processor_id()).loops_per_jiffy = loops_per_jiffy;
}
-static void wait_for_master_cpu(int cpu)
-{
- /*
- * Wait for release by control CPU before continuing with AP
- * initialization.
- */
- WARN_ON(cpumask_test_and_set_cpu(cpu, cpu_initialized_mask));
- while (!cpumask_test_cpu(cpu, cpu_callout_mask))
- cpu_relax();
-}
-
/*
* Activate a secondary processor.
*/
@@ -247,11 +234,10 @@ static void notrace start_secondary(void
cpu_init_exception_handling();
/*
- * Sync point with wait_cpu_initialized(). Sets AP in
- * cpu_initialized_mask and then waits for the control CPU
- * to release it.
+ * Sync point with the hotplug core. Sets the sync state to ALIVE
+ * and waits for the control CPU to release it.
*/
- wait_for_master_cpu(raw_smp_processor_id());
+ cpuhp_ap_sync_alive();
cpu_init();
rcu_cpu_starting(raw_smp_processor_id());
@@ -285,7 +271,6 @@ static void notrace start_secondary(void
set_cpu_online(smp_processor_id(), true);
lapic_online();
unlock_vector_lock();
- cpu_set_state_online(smp_processor_id());
x86_platform.nmi_init();
/* enable local interrupts */
@@ -736,9 +721,10 @@ static void impress_friends(void)
* Allow the user to impress friends.
*/
pr_debug("Before bogomips\n");
- for_each_possible_cpu(cpu)
- if (cpumask_test_cpu(cpu, cpu_callout_mask))
+ for_each_possible_cpu(cpu) {
+ if (cpumask_test_cpu(cpu, cpu_online_mask))
bogosum += cpu_data(cpu).loops_per_jiffy;
+ }
pr_info("Total of %d processors activated (%lu.%02lu BogoMIPS)\n",
num_online_cpus(),
bogosum/(500000/HZ),
@@ -1010,6 +996,7 @@ int common_cpu_up(unsigned int cpu, stru
static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle)
{
unsigned long start_ip = real_mode_header->trampoline_start;
+ int ret;
#ifdef CONFIG_X86_64
/* If 64-bit wakeup method exists, use the 64-bit mode trampoline IP */
@@ -1050,13 +1037,6 @@ static int do_boot_cpu(int apicid, int c
}
}
- /*
- * AP might wait on cpu_callout_mask in cpu_init() with
- * cpu_initialized_mask set if previous attempt to online
- * it timed-out. Clear cpu_initialized_mask so that after
- * INIT/SIPI it could start with a clean state.
- */
- cpumask_clear_cpu(cpu, cpu_initialized_mask);
smp_mb();
/*
@@ -1067,47 +1047,16 @@ static int do_boot_cpu(int apicid, int c
* - Use an INIT boot APIC message
*/
if (apic->wakeup_secondary_cpu_64)
- return apic->wakeup_secondary_cpu_64(apicid, start_ip);
+ ret = apic->wakeup_secondary_cpu_64(apicid, start_ip);
else if (apic->wakeup_secondary_cpu)
- return apic->wakeup_secondary_cpu(apicid, start_ip);
-
- return wakeup_secondary_cpu_via_init(apicid, start_ip);
-}
-
-static int wait_cpu_cpumask(unsigned int cpu, const struct cpumask *mask)
-{
- unsigned long timeout;
-
- /*
- * Wait up to 10s for the CPU to report in.
- */
- timeout = jiffies + 10*HZ;
- while (time_before(jiffies, timeout)) {
- if (cpumask_test_cpu(cpu, mask))
- return 0;
-
- schedule();
- }
- return -1;
-}
-
-/*
- * Bringup step two: Wait for the target AP to reach cpu_init_secondary()
- * and thus wait_for_master_cpu(), then set cpu_callout_mask to allow it
- * to proceed. The AP will then proceed past setting its 'callin' bit
- * and end up waiting in check_tsc_sync_target() until we reach
- * wait_cpu_online() to tend to it.
- */
-static int wait_cpu_initialized(unsigned int cpu)
-{
- /*
- * Wait for first sign of life from AP.
- */
- if (wait_cpu_cpumask(cpu, cpu_initialized_mask))
- return -1;
+ ret = apic->wakeup_secondary_cpu(apicid, start_ip);
+ else
+ ret = wakeup_secondary_cpu_via_init(apicid, start_ip);
- cpumask_set_cpu(cpu, cpu_callout_mask);
- return 0;
+ /* If the wakeup mechanism failed, cleanup the warm reset vector */
+ if (ret)
+ arch_cpuhp_cleanup_kick_cpu(cpu);
+ return ret;
}
static int native_kick_ap(unsigned int cpu, struct task_struct *tidle)
@@ -1132,11 +1081,6 @@ static int native_kick_ap(unsigned int c
*/
mtrr_save_state();
- /* x86 CPUs take themselves offline, so delayed offline is OK. */
- err = cpu_check_up_prepare(cpu);
- if (err && err != -EBUSY)
- return err;
-
/* the FPU context is blank, nobody can own it */
per_cpu(fpu_fpregs_owner_ctx, cpu) = NULL;
@@ -1153,17 +1097,29 @@ static int native_kick_ap(unsigned int c
int native_cpu_up(unsigned int cpu, struct task_struct *tidle)
{
- int ret;
-
- ret = native_kick_ap(cpu, tidle);
- if (!ret)
- ret = wait_cpu_initialized(cpu);
+ return native_kick_ap(cpu, tidle);
+}
+void arch_cpuhp_cleanup_kick_cpu(unsigned int cpu)
+{
/* Cleanup possible dangling ends... */
- if (x86_platform.legacy.warm_reset)
+ if (smp_ops.cpu_up == native_cpu_up && x86_platform.legacy.warm_reset)
smpboot_restore_warm_reset_vector();
+}
- return ret;
+void arch_cpuhp_cleanup_dead_cpu(unsigned int cpu)
+{
+ if (smp_ops.cleanup_dead_cpu)
+ smp_ops.cleanup_dead_cpu(cpu);
+
+ if (system_state == SYSTEM_RUNNING)
+ pr_info("CPU %u is now offline\n", cpu);
+}
+
+void arch_cpuhp_sync_state_poll(void)
+{
+ if (smp_ops.poll_sync_state)
+ smp_ops.poll_sync_state();
}
/**
@@ -1355,9 +1311,6 @@ void __init native_smp_prepare_boot_cpu(
if (!IS_ENABLED(CONFIG_SMP))
switch_gdt_and_percpu_base(me);
- /* already set me in cpu_online_mask in boot_cpu_init() */
- cpumask_set_cpu(me, cpu_callout_mask);
- cpu_set_state_online(me);
native_pv_lock_init();
}
@@ -1484,8 +1437,6 @@ early_param("possible_cpus", _setup_poss
/* correctly size the local cpu masks */
void __init setup_cpu_local_masks(void)
{
- alloc_bootmem_cpumask_var(&cpu_initialized_mask);
- alloc_bootmem_cpumask_var(&cpu_callout_mask);
alloc_bootmem_cpumask_var(&cpu_sibling_setup_mask);
}
@@ -1547,9 +1498,6 @@ static void remove_siblinginfo(int cpu)
static void remove_cpu_from_maps(int cpu)
{
set_cpu_online(cpu, false);
- cpumask_clear_cpu(cpu, cpu_callout_mask);
- /* was set by cpu_init() */
- cpumask_clear_cpu(cpu, cpu_initialized_mask);
numa_remove_cpu(cpu);
}
@@ -1600,36 +1548,11 @@ int native_cpu_disable(void)
return 0;
}
-int common_cpu_die(unsigned int cpu)
-{
- int ret = 0;
-
- /* We don't do anything here: idle task is faking death itself. */
-
- /* They ack this in play_dead() by setting CPU_DEAD */
- if (cpu_wait_death(cpu, 5)) {
- if (system_state == SYSTEM_RUNNING)
- pr_info("CPU %u is now offline\n", cpu);
- } else {
- pr_err("CPU %u didn't die...\n", cpu);
- ret = -1;
- }
-
- return ret;
-}
-
-void native_cpu_die(unsigned int cpu)
-{
- common_cpu_die(cpu);
-}
-
void play_dead_common(void)
{
idle_task_exit();
- /* Ack it */
- (void)cpu_report_death();
-
+ cpuhp_ap_report_dead();
/*
* With physical CPU hotplug, we should halt the cpu
*/
@@ -1731,12 +1654,6 @@ int native_cpu_disable(void)
return -ENOSYS;
}
-void native_cpu_die(unsigned int cpu)
-{
- /* We said "no" in __cpu_disable */
- BUG();
-}
-
void native_play_dead(void)
{
BUG();
--- a/arch/x86/xen/smp_hvm.c
+++ b/arch/x86/xen/smp_hvm.c
@@ -55,18 +55,16 @@ static void __init xen_hvm_smp_prepare_c
}
#ifdef CONFIG_HOTPLUG_CPU
-static void xen_hvm_cpu_die(unsigned int cpu)
+static void xen_hvm_cleanup_dead_cpu(unsigned int cpu)
{
- if (common_cpu_die(cpu) == 0) {
- if (xen_have_vector_callback) {
- xen_smp_intr_free(cpu);
- xen_uninit_lock_cpu(cpu);
- xen_teardown_timer(cpu);
- }
+ if (xen_have_vector_callback) {
+ xen_smp_intr_free(cpu);
+ xen_uninit_lock_cpu(cpu);
+ xen_teardown_timer(cpu);
}
}
#else
-static void xen_hvm_cpu_die(unsigned int cpu)
+static void xen_hvm_cleanup_dead_cpu(unsigned int cpu)
{
BUG();
}
@@ -77,7 +75,7 @@ void __init xen_hvm_smp_init(void)
smp_ops.smp_prepare_boot_cpu = xen_hvm_smp_prepare_boot_cpu;
smp_ops.smp_prepare_cpus = xen_hvm_smp_prepare_cpus;
smp_ops.smp_cpus_done = xen_smp_cpus_done;
- smp_ops.cpu_die = xen_hvm_cpu_die;
+ smp_ops.cleanup_dead_cpu = xen_hvm_cleanup_dead_cpu;
if (!xen_have_vector_callback) {
#ifdef CONFIG_PARAVIRT_SPINLOCKS
--- a/arch/x86/xen/smp_pv.c
+++ b/arch/x86/xen/smp_pv.c
@@ -62,6 +62,7 @@ static void cpu_bringup(void)
int cpu;
cr4_init();
+ cpuhp_ap_sync_alive();
cpu_init();
touch_softlockup_watchdog();
@@ -83,7 +84,7 @@ static void cpu_bringup(void)
set_cpu_online(cpu, true);
- cpu_set_state_online(cpu); /* Implies full memory barrier. */
+ smp_mb();
/* We can take interrupts now: we're officially "up". */
local_irq_enable();
@@ -323,14 +324,6 @@ static int xen_pv_cpu_up(unsigned int cp
xen_setup_runstate_info(cpu);
- /*
- * PV VCPUs are always successfully taken down (see 'while' loop
- * in xen_cpu_die()), so -EBUSY is an error.
- */
- rc = cpu_check_up_prepare(cpu);
- if (rc)
- return rc;
-
/* make sure interrupts start blocked */
per_cpu(xen_vcpu, cpu)->evtchn_upcall_mask = 1;
@@ -349,6 +342,11 @@ static int xen_pv_cpu_up(unsigned int cp
return 0;
}
+static void xen_pv_poll_sync_state(void)
+{
+ HYPERVISOR_sched_op(SCHEDOP_yield, NULL);
+}
+
#ifdef CONFIG_HOTPLUG_CPU
static int xen_pv_cpu_disable(void)
{
@@ -364,18 +362,18 @@ static int xen_pv_cpu_disable(void)
static void xen_pv_cpu_die(unsigned int cpu)
{
- while (HYPERVISOR_vcpu_op(VCPUOP_is_up,
- xen_vcpu_nr(cpu), NULL)) {
+ while (HYPERVISOR_vcpu_op(VCPUOP_is_up, xen_vcpu_nr(cpu), NULL)) {
__set_current_state(TASK_UNINTERRUPTIBLE);
schedule_timeout(HZ/10);
}
+}
- if (common_cpu_die(cpu) == 0) {
- xen_smp_intr_free(cpu);
- xen_uninit_lock_cpu(cpu);
- xen_teardown_timer(cpu);
- xen_pmu_finish(cpu);
- }
+static void xen_pv_cleanup_dead_cpu(unsigned int cpu)
+{
+ xen_smp_intr_free(cpu);
+ xen_uninit_lock_cpu(cpu);
+ xen_teardown_timer(cpu);
+ xen_pmu_finish(cpu);
}
static void __noreturn xen_pv_play_dead(void) /* used only with HOTPLUG_CPU */
@@ -397,6 +395,11 @@ static void xen_pv_cpu_die(unsigned int
BUG();
}
+static void xen_pv_cleanup_dead_cpu(unsigned int cpu)
+{
+ BUG();
+}
+
static void __noreturn xen_pv_play_dead(void)
{
BUG();
@@ -437,6 +440,8 @@ static const struct smp_ops xen_smp_ops
.cpu_up = xen_pv_cpu_up,
.cpu_die = xen_pv_cpu_die,
+ .cleanup_dead_cpu = xen_pv_cleanup_dead_cpu,
+ .poll_sync_state = xen_pv_poll_sync_state,
.cpu_disable = xen_pv_cpu_disable,
.play_dead = xen_pv_play_dead,
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 19/37] x86/smpboot: Switch to hotplug core state synchronization
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Juergen Gross,
Boris Ostrovsky, xen-devel, David Woodhouse, Usama Arif,
Russell King, Arnd Bergmann, linux-arm-kernel, Catalin Marinas,
Will Deacon, Guo Ren, linux-csky, Thomas Bogendoerfer,
linux-mips, James E.J. Bottomley, Helge Deller, linux-parisc,
Paul Walmsley, Palmer Dabbelt, linux-riscv, Mark Rutland,
Sabin Rapan
The new AP state tracking and synchronization mechanism in the CPU hotplug
core code allows to remove quite some x86 specific code:
1) The AP alive synchronization based on cpumasks
2) The decision whether an AP can be brought up again
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Juergen Gross <jgross@suse.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: xen-devel@lists.xenproject.org
---
arch/x86/Kconfig | 1
arch/x86/include/asm/smp.h | 7 +
arch/x86/kernel/smp.c | 1
arch/x86/kernel/smpboot.c | 159 ++++++++++-----------------------------------
arch/x86/xen/smp_hvm.c | 16 +---
arch/x86/xen/smp_pv.c | 39 ++++++-----
6 files changed, 72 insertions(+), 151 deletions(-)
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -272,6 +272,7 @@ config X86
select HAVE_UNSTABLE_SCHED_CLOCK
select HAVE_USER_RETURN_NOTIFIER
select HAVE_GENERIC_VDSO
+ select HOTPLUG_CORE_SYNC_FULL if SMP
select HOTPLUG_SMT if SMP
select IRQ_FORCED_THREADING
select NEED_PER_CPU_EMBED_FIRST_CHUNK
--- a/arch/x86/include/asm/smp.h
+++ b/arch/x86/include/asm/smp.h
@@ -38,6 +38,8 @@ struct smp_ops {
void (*crash_stop_other_cpus)(void);
void (*smp_send_reschedule)(int cpu);
+ void (*cleanup_dead_cpu)(unsigned cpu);
+ void (*poll_sync_state)(void);
int (*cpu_up)(unsigned cpu, struct task_struct *tidle);
int (*cpu_disable)(void);
void (*cpu_die)(unsigned int cpu);
@@ -90,7 +92,8 @@ static inline int __cpu_disable(void)
static inline void __cpu_die(unsigned int cpu)
{
- smp_ops.cpu_die(cpu);
+ if (smp_ops.cpu_die)
+ smp_ops.cpu_die(cpu);
}
static inline void play_dead(void)
@@ -122,8 +125,6 @@ void native_smp_cpus_done(unsigned int m
int common_cpu_up(unsigned int cpunum, struct task_struct *tidle);
int native_cpu_up(unsigned int cpunum, struct task_struct *tidle);
int native_cpu_disable(void);
-int common_cpu_die(unsigned int cpu);
-void native_cpu_die(unsigned int cpu);
void hlt_play_dead(void);
void native_play_dead(void);
void play_dead_common(void);
--- a/arch/x86/kernel/smp.c
+++ b/arch/x86/kernel/smp.c
@@ -269,7 +269,6 @@ struct smp_ops smp_ops = {
.smp_send_reschedule = native_smp_send_reschedule,
.cpu_up = native_cpu_up,
- .cpu_die = native_cpu_die,
.cpu_disable = native_cpu_disable,
.play_dead = native_play_dead,
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -57,6 +57,7 @@
#include <linux/pgtable.h>
#include <linux/overflow.h>
#include <linux/stackprotector.h>
+#include <linux/cpuhotplug.h>
#include <asm/acpi.h>
#include <asm/cacheinfo.h>
@@ -101,9 +102,6 @@ EXPORT_PER_CPU_SYMBOL(cpu_die_map);
DEFINE_PER_CPU_READ_MOSTLY(struct cpuinfo_x86, cpu_info);
EXPORT_PER_CPU_SYMBOL(cpu_info);
-/* All of these masks are initialized in setup_cpu_local_masks() */
-static cpumask_var_t cpu_initialized_mask;
-static cpumask_var_t cpu_callout_mask;
/* Representing CPUs for which sibling maps can be computed */
static cpumask_var_t cpu_sibling_setup_mask;
@@ -169,8 +167,8 @@ static void smp_callin(void)
int cpuid = smp_processor_id();
/*
- * If waken up by an INIT in an 82489DX configuration
- * cpu_callout_mask guarantees we don't get here before an
+ * If waken up by an INIT in an 82489DX configuration the alive
+ * synchronization guarantees we don't get here before an
* INIT_deassert IPI reaches our local APIC, so it is now safe to
* touch our local APIC.
*
@@ -216,17 +214,6 @@ static void ap_calibrate_delay(void)
cpu_data(smp_processor_id()).loops_per_jiffy = loops_per_jiffy;
}
-static void wait_for_master_cpu(int cpu)
-{
- /*
- * Wait for release by control CPU before continuing with AP
- * initialization.
- */
- WARN_ON(cpumask_test_and_set_cpu(cpu, cpu_initialized_mask));
- while (!cpumask_test_cpu(cpu, cpu_callout_mask))
- cpu_relax();
-}
-
/*
* Activate a secondary processor.
*/
@@ -247,11 +234,10 @@ static void notrace start_secondary(void
cpu_init_exception_handling();
/*
- * Sync point with wait_cpu_initialized(). Sets AP in
- * cpu_initialized_mask and then waits for the control CPU
- * to release it.
+ * Sync point with the hotplug core. Sets the sync state to ALIVE
+ * and waits for the control CPU to release it.
*/
- wait_for_master_cpu(raw_smp_processor_id());
+ cpuhp_ap_sync_alive();
cpu_init();
rcu_cpu_starting(raw_smp_processor_id());
@@ -285,7 +271,6 @@ static void notrace start_secondary(void
set_cpu_online(smp_processor_id(), true);
lapic_online();
unlock_vector_lock();
- cpu_set_state_online(smp_processor_id());
x86_platform.nmi_init();
/* enable local interrupts */
@@ -736,9 +721,10 @@ static void impress_friends(void)
* Allow the user to impress friends.
*/
pr_debug("Before bogomips\n");
- for_each_possible_cpu(cpu)
- if (cpumask_test_cpu(cpu, cpu_callout_mask))
+ for_each_possible_cpu(cpu) {
+ if (cpumask_test_cpu(cpu, cpu_online_mask))
bogosum += cpu_data(cpu).loops_per_jiffy;
+ }
pr_info("Total of %d processors activated (%lu.%02lu BogoMIPS)\n",
num_online_cpus(),
bogosum/(500000/HZ),
@@ -1010,6 +996,7 @@ int common_cpu_up(unsigned int cpu, stru
static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle)
{
unsigned long start_ip = real_mode_header->trampoline_start;
+ int ret;
#ifdef CONFIG_X86_64
/* If 64-bit wakeup method exists, use the 64-bit mode trampoline IP */
@@ -1050,13 +1037,6 @@ static int do_boot_cpu(int apicid, int c
}
}
- /*
- * AP might wait on cpu_callout_mask in cpu_init() with
- * cpu_initialized_mask set if previous attempt to online
- * it timed-out. Clear cpu_initialized_mask so that after
- * INIT/SIPI it could start with a clean state.
- */
- cpumask_clear_cpu(cpu, cpu_initialized_mask);
smp_mb();
/*
@@ -1067,47 +1047,16 @@ static int do_boot_cpu(int apicid, int c
* - Use an INIT boot APIC message
*/
if (apic->wakeup_secondary_cpu_64)
- return apic->wakeup_secondary_cpu_64(apicid, start_ip);
+ ret = apic->wakeup_secondary_cpu_64(apicid, start_ip);
else if (apic->wakeup_secondary_cpu)
- return apic->wakeup_secondary_cpu(apicid, start_ip);
-
- return wakeup_secondary_cpu_via_init(apicid, start_ip);
-}
-
-static int wait_cpu_cpumask(unsigned int cpu, const struct cpumask *mask)
-{
- unsigned long timeout;
-
- /*
- * Wait up to 10s for the CPU to report in.
- */
- timeout = jiffies + 10*HZ;
- while (time_before(jiffies, timeout)) {
- if (cpumask_test_cpu(cpu, mask))
- return 0;
-
- schedule();
- }
- return -1;
-}
-
-/*
- * Bringup step two: Wait for the target AP to reach cpu_init_secondary()
- * and thus wait_for_master_cpu(), then set cpu_callout_mask to allow it
- * to proceed. The AP will then proceed past setting its 'callin' bit
- * and end up waiting in check_tsc_sync_target() until we reach
- * wait_cpu_online() to tend to it.
- */
-static int wait_cpu_initialized(unsigned int cpu)
-{
- /*
- * Wait for first sign of life from AP.
- */
- if (wait_cpu_cpumask(cpu, cpu_initialized_mask))
- return -1;
+ ret = apic->wakeup_secondary_cpu(apicid, start_ip);
+ else
+ ret = wakeup_secondary_cpu_via_init(apicid, start_ip);
- cpumask_set_cpu(cpu, cpu_callout_mask);
- return 0;
+ /* If the wakeup mechanism failed, cleanup the warm reset vector */
+ if (ret)
+ arch_cpuhp_cleanup_kick_cpu(cpu);
+ return ret;
}
static int native_kick_ap(unsigned int cpu, struct task_struct *tidle)
@@ -1132,11 +1081,6 @@ static int native_kick_ap(unsigned int c
*/
mtrr_save_state();
- /* x86 CPUs take themselves offline, so delayed offline is OK. */
- err = cpu_check_up_prepare(cpu);
- if (err && err != -EBUSY)
- return err;
-
/* the FPU context is blank, nobody can own it */
per_cpu(fpu_fpregs_owner_ctx, cpu) = NULL;
@@ -1153,17 +1097,29 @@ static int native_kick_ap(unsigned int c
int native_cpu_up(unsigned int cpu, struct task_struct *tidle)
{
- int ret;
-
- ret = native_kick_ap(cpu, tidle);
- if (!ret)
- ret = wait_cpu_initialized(cpu);
+ return native_kick_ap(cpu, tidle);
+}
+void arch_cpuhp_cleanup_kick_cpu(unsigned int cpu)
+{
/* Cleanup possible dangling ends... */
- if (x86_platform.legacy.warm_reset)
+ if (smp_ops.cpu_up == native_cpu_up && x86_platform.legacy.warm_reset)
smpboot_restore_warm_reset_vector();
+}
- return ret;
+void arch_cpuhp_cleanup_dead_cpu(unsigned int cpu)
+{
+ if (smp_ops.cleanup_dead_cpu)
+ smp_ops.cleanup_dead_cpu(cpu);
+
+ if (system_state == SYSTEM_RUNNING)
+ pr_info("CPU %u is now offline\n", cpu);
+}
+
+void arch_cpuhp_sync_state_poll(void)
+{
+ if (smp_ops.poll_sync_state)
+ smp_ops.poll_sync_state();
}
/**
@@ -1355,9 +1311,6 @@ void __init native_smp_prepare_boot_cpu(
if (!IS_ENABLED(CONFIG_SMP))
switch_gdt_and_percpu_base(me);
- /* already set me in cpu_online_mask in boot_cpu_init() */
- cpumask_set_cpu(me, cpu_callout_mask);
- cpu_set_state_online(me);
native_pv_lock_init();
}
@@ -1484,8 +1437,6 @@ early_param("possible_cpus", _setup_poss
/* correctly size the local cpu masks */
void __init setup_cpu_local_masks(void)
{
- alloc_bootmem_cpumask_var(&cpu_initialized_mask);
- alloc_bootmem_cpumask_var(&cpu_callout_mask);
alloc_bootmem_cpumask_var(&cpu_sibling_setup_mask);
}
@@ -1547,9 +1498,6 @@ static void remove_siblinginfo(int cpu)
static void remove_cpu_from_maps(int cpu)
{
set_cpu_online(cpu, false);
- cpumask_clear_cpu(cpu, cpu_callout_mask);
- /* was set by cpu_init() */
- cpumask_clear_cpu(cpu, cpu_initialized_mask);
numa_remove_cpu(cpu);
}
@@ -1600,36 +1548,11 @@ int native_cpu_disable(void)
return 0;
}
-int common_cpu_die(unsigned int cpu)
-{
- int ret = 0;
-
- /* We don't do anything here: idle task is faking death itself. */
-
- /* They ack this in play_dead() by setting CPU_DEAD */
- if (cpu_wait_death(cpu, 5)) {
- if (system_state == SYSTEM_RUNNING)
- pr_info("CPU %u is now offline\n", cpu);
- } else {
- pr_err("CPU %u didn't die...\n", cpu);
- ret = -1;
- }
-
- return ret;
-}
-
-void native_cpu_die(unsigned int cpu)
-{
- common_cpu_die(cpu);
-}
-
void play_dead_common(void)
{
idle_task_exit();
- /* Ack it */
- (void)cpu_report_death();
-
+ cpuhp_ap_report_dead();
/*
* With physical CPU hotplug, we should halt the cpu
*/
@@ -1731,12 +1654,6 @@ int native_cpu_disable(void)
return -ENOSYS;
}
-void native_cpu_die(unsigned int cpu)
-{
- /* We said "no" in __cpu_disable */
- BUG();
-}
-
void native_play_dead(void)
{
BUG();
--- a/arch/x86/xen/smp_hvm.c
+++ b/arch/x86/xen/smp_hvm.c
@@ -55,18 +55,16 @@ static void __init xen_hvm_smp_prepare_c
}
#ifdef CONFIG_HOTPLUG_CPU
-static void xen_hvm_cpu_die(unsigned int cpu)
+static void xen_hvm_cleanup_dead_cpu(unsigned int cpu)
{
- if (common_cpu_die(cpu) == 0) {
- if (xen_have_vector_callback) {
- xen_smp_intr_free(cpu);
- xen_uninit_lock_cpu(cpu);
- xen_teardown_timer(cpu);
- }
+ if (xen_have_vector_callback) {
+ xen_smp_intr_free(cpu);
+ xen_uninit_lock_cpu(cpu);
+ xen_teardown_timer(cpu);
}
}
#else
-static void xen_hvm_cpu_die(unsigned int cpu)
+static void xen_hvm_cleanup_dead_cpu(unsigned int cpu)
{
BUG();
}
@@ -77,7 +75,7 @@ void __init xen_hvm_smp_init(void)
smp_ops.smp_prepare_boot_cpu = xen_hvm_smp_prepare_boot_cpu;
smp_ops.smp_prepare_cpus = xen_hvm_smp_prepare_cpus;
smp_ops.smp_cpus_done = xen_smp_cpus_done;
- smp_ops.cpu_die = xen_hvm_cpu_die;
+ smp_ops.cleanup_dead_cpu = xen_hvm_cleanup_dead_cpu;
if (!xen_have_vector_callback) {
#ifdef CONFIG_PARAVIRT_SPINLOCKS
--- a/arch/x86/xen/smp_pv.c
+++ b/arch/x86/xen/smp_pv.c
@@ -62,6 +62,7 @@ static void cpu_bringup(void)
int cpu;
cr4_init();
+ cpuhp_ap_sync_alive();
cpu_init();
touch_softlockup_watchdog();
@@ -83,7 +84,7 @@ static void cpu_bringup(void)
set_cpu_online(cpu, true);
- cpu_set_state_online(cpu); /* Implies full memory barrier. */
+ smp_mb();
/* We can take interrupts now: we're officially "up". */
local_irq_enable();
@@ -323,14 +324,6 @@ static int xen_pv_cpu_up(unsigned int cp
xen_setup_runstate_info(cpu);
- /*
- * PV VCPUs are always successfully taken down (see 'while' loop
- * in xen_cpu_die()), so -EBUSY is an error.
- */
- rc = cpu_check_up_prepare(cpu);
- if (rc)
- return rc;
-
/* make sure interrupts start blocked */
per_cpu(xen_vcpu, cpu)->evtchn_upcall_mask = 1;
@@ -349,6 +342,11 @@ static int xen_pv_cpu_up(unsigned int cp
return 0;
}
+static void xen_pv_poll_sync_state(void)
+{
+ HYPERVISOR_sched_op(SCHEDOP_yield, NULL);
+}
+
#ifdef CONFIG_HOTPLUG_CPU
static int xen_pv_cpu_disable(void)
{
@@ -364,18 +362,18 @@ static int xen_pv_cpu_disable(void)
static void xen_pv_cpu_die(unsigned int cpu)
{
- while (HYPERVISOR_vcpu_op(VCPUOP_is_up,
- xen_vcpu_nr(cpu), NULL)) {
+ while (HYPERVISOR_vcpu_op(VCPUOP_is_up, xen_vcpu_nr(cpu), NULL)) {
__set_current_state(TASK_UNINTERRUPTIBLE);
schedule_timeout(HZ/10);
}
+}
- if (common_cpu_die(cpu) == 0) {
- xen_smp_intr_free(cpu);
- xen_uninit_lock_cpu(cpu);
- xen_teardown_timer(cpu);
- xen_pmu_finish(cpu);
- }
+static void xen_pv_cleanup_dead_cpu(unsigned int cpu)
+{
+ xen_smp_intr_free(cpu);
+ xen_uninit_lock_cpu(cpu);
+ xen_teardown_timer(cpu);
+ xen_pmu_finish(cpu);
}
static void __noreturn xen_pv_play_dead(void) /* used only with HOTPLUG_CPU */
@@ -397,6 +395,11 @@ static void xen_pv_cpu_die(unsigned int
BUG();
}
+static void xen_pv_cleanup_dead_cpu(unsigned int cpu)
+{
+ BUG();
+}
+
static void __noreturn xen_pv_play_dead(void)
{
BUG();
@@ -437,6 +440,8 @@ static const struct smp_ops xen_smp_ops
.cpu_up = xen_pv_cpu_up,
.cpu_die = xen_pv_cpu_die,
+ .cleanup_dead_cpu = xen_pv_cleanup_dead_cpu,
+ .poll_sync_state = xen_pv_poll_sync_state,
.cpu_disable = xen_pv_cpu_disable,
.play_dead = xen_pv_play_dead,
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 19/37] x86/smpboot: Switch to hotplug core state synchronization
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Juergen Gross,
Boris Ostrovsky, xen-devel, David Woodhouse, Usama Arif,
Russell King, Arnd Bergmann, linux-arm-kernel, Catalin Marinas,
Will Deacon, Guo Ren, linux-csky, Thomas Bogendoerfer,
linux-mips, James E.J. Bottomley, Helge Deller, linux-parisc,
Paul Walmsley, Palmer Dabbelt, linux-riscv, Mark Rutland,
Sabin Rapan
The new AP state tracking and synchronization mechanism in the CPU hotplug
core code allows to remove quite some x86 specific code:
1) The AP alive synchronization based on cpumasks
2) The decision whether an AP can be brought up again
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Juergen Gross <jgross@suse.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: xen-devel@lists.xenproject.org
---
arch/x86/Kconfig | 1
arch/x86/include/asm/smp.h | 7 +
arch/x86/kernel/smp.c | 1
arch/x86/kernel/smpboot.c | 159 ++++++++++-----------------------------------
arch/x86/xen/smp_hvm.c | 16 +---
arch/x86/xen/smp_pv.c | 39 ++++++-----
6 files changed, 72 insertions(+), 151 deletions(-)
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -272,6 +272,7 @@ config X86
select HAVE_UNSTABLE_SCHED_CLOCK
select HAVE_USER_RETURN_NOTIFIER
select HAVE_GENERIC_VDSO
+ select HOTPLUG_CORE_SYNC_FULL if SMP
select HOTPLUG_SMT if SMP
select IRQ_FORCED_THREADING
select NEED_PER_CPU_EMBED_FIRST_CHUNK
--- a/arch/x86/include/asm/smp.h
+++ b/arch/x86/include/asm/smp.h
@@ -38,6 +38,8 @@ struct smp_ops {
void (*crash_stop_other_cpus)(void);
void (*smp_send_reschedule)(int cpu);
+ void (*cleanup_dead_cpu)(unsigned cpu);
+ void (*poll_sync_state)(void);
int (*cpu_up)(unsigned cpu, struct task_struct *tidle);
int (*cpu_disable)(void);
void (*cpu_die)(unsigned int cpu);
@@ -90,7 +92,8 @@ static inline int __cpu_disable(void)
static inline void __cpu_die(unsigned int cpu)
{
- smp_ops.cpu_die(cpu);
+ if (smp_ops.cpu_die)
+ smp_ops.cpu_die(cpu);
}
static inline void play_dead(void)
@@ -122,8 +125,6 @@ void native_smp_cpus_done(unsigned int m
int common_cpu_up(unsigned int cpunum, struct task_struct *tidle);
int native_cpu_up(unsigned int cpunum, struct task_struct *tidle);
int native_cpu_disable(void);
-int common_cpu_die(unsigned int cpu);
-void native_cpu_die(unsigned int cpu);
void hlt_play_dead(void);
void native_play_dead(void);
void play_dead_common(void);
--- a/arch/x86/kernel/smp.c
+++ b/arch/x86/kernel/smp.c
@@ -269,7 +269,6 @@ struct smp_ops smp_ops = {
.smp_send_reschedule = native_smp_send_reschedule,
.cpu_up = native_cpu_up,
- .cpu_die = native_cpu_die,
.cpu_disable = native_cpu_disable,
.play_dead = native_play_dead,
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -57,6 +57,7 @@
#include <linux/pgtable.h>
#include <linux/overflow.h>
#include <linux/stackprotector.h>
+#include <linux/cpuhotplug.h>
#include <asm/acpi.h>
#include <asm/cacheinfo.h>
@@ -101,9 +102,6 @@ EXPORT_PER_CPU_SYMBOL(cpu_die_map);
DEFINE_PER_CPU_READ_MOSTLY(struct cpuinfo_x86, cpu_info);
EXPORT_PER_CPU_SYMBOL(cpu_info);
-/* All of these masks are initialized in setup_cpu_local_masks() */
-static cpumask_var_t cpu_initialized_mask;
-static cpumask_var_t cpu_callout_mask;
/* Representing CPUs for which sibling maps can be computed */
static cpumask_var_t cpu_sibling_setup_mask;
@@ -169,8 +167,8 @@ static void smp_callin(void)
int cpuid = smp_processor_id();
/*
- * If waken up by an INIT in an 82489DX configuration
- * cpu_callout_mask guarantees we don't get here before an
+ * If waken up by an INIT in an 82489DX configuration the alive
+ * synchronization guarantees we don't get here before an
* INIT_deassert IPI reaches our local APIC, so it is now safe to
* touch our local APIC.
*
@@ -216,17 +214,6 @@ static void ap_calibrate_delay(void)
cpu_data(smp_processor_id()).loops_per_jiffy = loops_per_jiffy;
}
-static void wait_for_master_cpu(int cpu)
-{
- /*
- * Wait for release by control CPU before continuing with AP
- * initialization.
- */
- WARN_ON(cpumask_test_and_set_cpu(cpu, cpu_initialized_mask));
- while (!cpumask_test_cpu(cpu, cpu_callout_mask))
- cpu_relax();
-}
-
/*
* Activate a secondary processor.
*/
@@ -247,11 +234,10 @@ static void notrace start_secondary(void
cpu_init_exception_handling();
/*
- * Sync point with wait_cpu_initialized(). Sets AP in
- * cpu_initialized_mask and then waits for the control CPU
- * to release it.
+ * Sync point with the hotplug core. Sets the sync state to ALIVE
+ * and waits for the control CPU to release it.
*/
- wait_for_master_cpu(raw_smp_processor_id());
+ cpuhp_ap_sync_alive();
cpu_init();
rcu_cpu_starting(raw_smp_processor_id());
@@ -285,7 +271,6 @@ static void notrace start_secondary(void
set_cpu_online(smp_processor_id(), true);
lapic_online();
unlock_vector_lock();
- cpu_set_state_online(smp_processor_id());
x86_platform.nmi_init();
/* enable local interrupts */
@@ -736,9 +721,10 @@ static void impress_friends(void)
* Allow the user to impress friends.
*/
pr_debug("Before bogomips\n");
- for_each_possible_cpu(cpu)
- if (cpumask_test_cpu(cpu, cpu_callout_mask))
+ for_each_possible_cpu(cpu) {
+ if (cpumask_test_cpu(cpu, cpu_online_mask))
bogosum += cpu_data(cpu).loops_per_jiffy;
+ }
pr_info("Total of %d processors activated (%lu.%02lu BogoMIPS)\n",
num_online_cpus(),
bogosum/(500000/HZ),
@@ -1010,6 +996,7 @@ int common_cpu_up(unsigned int cpu, stru
static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle)
{
unsigned long start_ip = real_mode_header->trampoline_start;
+ int ret;
#ifdef CONFIG_X86_64
/* If 64-bit wakeup method exists, use the 64-bit mode trampoline IP */
@@ -1050,13 +1037,6 @@ static int do_boot_cpu(int apicid, int c
}
}
- /*
- * AP might wait on cpu_callout_mask in cpu_init() with
- * cpu_initialized_mask set if previous attempt to online
- * it timed-out. Clear cpu_initialized_mask so that after
- * INIT/SIPI it could start with a clean state.
- */
- cpumask_clear_cpu(cpu, cpu_initialized_mask);
smp_mb();
/*
@@ -1067,47 +1047,16 @@ static int do_boot_cpu(int apicid, int c
* - Use an INIT boot APIC message
*/
if (apic->wakeup_secondary_cpu_64)
- return apic->wakeup_secondary_cpu_64(apicid, start_ip);
+ ret = apic->wakeup_secondary_cpu_64(apicid, start_ip);
else if (apic->wakeup_secondary_cpu)
- return apic->wakeup_secondary_cpu(apicid, start_ip);
-
- return wakeup_secondary_cpu_via_init(apicid, start_ip);
-}
-
-static int wait_cpu_cpumask(unsigned int cpu, const struct cpumask *mask)
-{
- unsigned long timeout;
-
- /*
- * Wait up to 10s for the CPU to report in.
- */
- timeout = jiffies + 10*HZ;
- while (time_before(jiffies, timeout)) {
- if (cpumask_test_cpu(cpu, mask))
- return 0;
-
- schedule();
- }
- return -1;
-}
-
-/*
- * Bringup step two: Wait for the target AP to reach cpu_init_secondary()
- * and thus wait_for_master_cpu(), then set cpu_callout_mask to allow it
- * to proceed. The AP will then proceed past setting its 'callin' bit
- * and end up waiting in check_tsc_sync_target() until we reach
- * wait_cpu_online() to tend to it.
- */
-static int wait_cpu_initialized(unsigned int cpu)
-{
- /*
- * Wait for first sign of life from AP.
- */
- if (wait_cpu_cpumask(cpu, cpu_initialized_mask))
- return -1;
+ ret = apic->wakeup_secondary_cpu(apicid, start_ip);
+ else
+ ret = wakeup_secondary_cpu_via_init(apicid, start_ip);
- cpumask_set_cpu(cpu, cpu_callout_mask);
- return 0;
+ /* If the wakeup mechanism failed, cleanup the warm reset vector */
+ if (ret)
+ arch_cpuhp_cleanup_kick_cpu(cpu);
+ return ret;
}
static int native_kick_ap(unsigned int cpu, struct task_struct *tidle)
@@ -1132,11 +1081,6 @@ static int native_kick_ap(unsigned int c
*/
mtrr_save_state();
- /* x86 CPUs take themselves offline, so delayed offline is OK. */
- err = cpu_check_up_prepare(cpu);
- if (err && err != -EBUSY)
- return err;
-
/* the FPU context is blank, nobody can own it */
per_cpu(fpu_fpregs_owner_ctx, cpu) = NULL;
@@ -1153,17 +1097,29 @@ static int native_kick_ap(unsigned int c
int native_cpu_up(unsigned int cpu, struct task_struct *tidle)
{
- int ret;
-
- ret = native_kick_ap(cpu, tidle);
- if (!ret)
- ret = wait_cpu_initialized(cpu);
+ return native_kick_ap(cpu, tidle);
+}
+void arch_cpuhp_cleanup_kick_cpu(unsigned int cpu)
+{
/* Cleanup possible dangling ends... */
- if (x86_platform.legacy.warm_reset)
+ if (smp_ops.cpu_up == native_cpu_up && x86_platform.legacy.warm_reset)
smpboot_restore_warm_reset_vector();
+}
- return ret;
+void arch_cpuhp_cleanup_dead_cpu(unsigned int cpu)
+{
+ if (smp_ops.cleanup_dead_cpu)
+ smp_ops.cleanup_dead_cpu(cpu);
+
+ if (system_state == SYSTEM_RUNNING)
+ pr_info("CPU %u is now offline\n", cpu);
+}
+
+void arch_cpuhp_sync_state_poll(void)
+{
+ if (smp_ops.poll_sync_state)
+ smp_ops.poll_sync_state();
}
/**
@@ -1355,9 +1311,6 @@ void __init native_smp_prepare_boot_cpu(
if (!IS_ENABLED(CONFIG_SMP))
switch_gdt_and_percpu_base(me);
- /* already set me in cpu_online_mask in boot_cpu_init() */
- cpumask_set_cpu(me, cpu_callout_mask);
- cpu_set_state_online(me);
native_pv_lock_init();
}
@@ -1484,8 +1437,6 @@ early_param("possible_cpus", _setup_poss
/* correctly size the local cpu masks */
void __init setup_cpu_local_masks(void)
{
- alloc_bootmem_cpumask_var(&cpu_initialized_mask);
- alloc_bootmem_cpumask_var(&cpu_callout_mask);
alloc_bootmem_cpumask_var(&cpu_sibling_setup_mask);
}
@@ -1547,9 +1498,6 @@ static void remove_siblinginfo(int cpu)
static void remove_cpu_from_maps(int cpu)
{
set_cpu_online(cpu, false);
- cpumask_clear_cpu(cpu, cpu_callout_mask);
- /* was set by cpu_init() */
- cpumask_clear_cpu(cpu, cpu_initialized_mask);
numa_remove_cpu(cpu);
}
@@ -1600,36 +1548,11 @@ int native_cpu_disable(void)
return 0;
}
-int common_cpu_die(unsigned int cpu)
-{
- int ret = 0;
-
- /* We don't do anything here: idle task is faking death itself. */
-
- /* They ack this in play_dead() by setting CPU_DEAD */
- if (cpu_wait_death(cpu, 5)) {
- if (system_state == SYSTEM_RUNNING)
- pr_info("CPU %u is now offline\n", cpu);
- } else {
- pr_err("CPU %u didn't die...\n", cpu);
- ret = -1;
- }
-
- return ret;
-}
-
-void native_cpu_die(unsigned int cpu)
-{
- common_cpu_die(cpu);
-}
-
void play_dead_common(void)
{
idle_task_exit();
- /* Ack it */
- (void)cpu_report_death();
-
+ cpuhp_ap_report_dead();
/*
* With physical CPU hotplug, we should halt the cpu
*/
@@ -1731,12 +1654,6 @@ int native_cpu_disable(void)
return -ENOSYS;
}
-void native_cpu_die(unsigned int cpu)
-{
- /* We said "no" in __cpu_disable */
- BUG();
-}
-
void native_play_dead(void)
{
BUG();
--- a/arch/x86/xen/smp_hvm.c
+++ b/arch/x86/xen/smp_hvm.c
@@ -55,18 +55,16 @@ static void __init xen_hvm_smp_prepare_c
}
#ifdef CONFIG_HOTPLUG_CPU
-static void xen_hvm_cpu_die(unsigned int cpu)
+static void xen_hvm_cleanup_dead_cpu(unsigned int cpu)
{
- if (common_cpu_die(cpu) == 0) {
- if (xen_have_vector_callback) {
- xen_smp_intr_free(cpu);
- xen_uninit_lock_cpu(cpu);
- xen_teardown_timer(cpu);
- }
+ if (xen_have_vector_callback) {
+ xen_smp_intr_free(cpu);
+ xen_uninit_lock_cpu(cpu);
+ xen_teardown_timer(cpu);
}
}
#else
-static void xen_hvm_cpu_die(unsigned int cpu)
+static void xen_hvm_cleanup_dead_cpu(unsigned int cpu)
{
BUG();
}
@@ -77,7 +75,7 @@ void __init xen_hvm_smp_init(void)
smp_ops.smp_prepare_boot_cpu = xen_hvm_smp_prepare_boot_cpu;
smp_ops.smp_prepare_cpus = xen_hvm_smp_prepare_cpus;
smp_ops.smp_cpus_done = xen_smp_cpus_done;
- smp_ops.cpu_die = xen_hvm_cpu_die;
+ smp_ops.cleanup_dead_cpu = xen_hvm_cleanup_dead_cpu;
if (!xen_have_vector_callback) {
#ifdef CONFIG_PARAVIRT_SPINLOCKS
--- a/arch/x86/xen/smp_pv.c
+++ b/arch/x86/xen/smp_pv.c
@@ -62,6 +62,7 @@ static void cpu_bringup(void)
int cpu;
cr4_init();
+ cpuhp_ap_sync_alive();
cpu_init();
touch_softlockup_watchdog();
@@ -83,7 +84,7 @@ static void cpu_bringup(void)
set_cpu_online(cpu, true);
- cpu_set_state_online(cpu); /* Implies full memory barrier. */
+ smp_mb();
/* We can take interrupts now: we're officially "up". */
local_irq_enable();
@@ -323,14 +324,6 @@ static int xen_pv_cpu_up(unsigned int cp
xen_setup_runstate_info(cpu);
- /*
- * PV VCPUs are always successfully taken down (see 'while' loop
- * in xen_cpu_die()), so -EBUSY is an error.
- */
- rc = cpu_check_up_prepare(cpu);
- if (rc)
- return rc;
-
/* make sure interrupts start blocked */
per_cpu(xen_vcpu, cpu)->evtchn_upcall_mask = 1;
@@ -349,6 +342,11 @@ static int xen_pv_cpu_up(unsigned int cp
return 0;
}
+static void xen_pv_poll_sync_state(void)
+{
+ HYPERVISOR_sched_op(SCHEDOP_yield, NULL);
+}
+
#ifdef CONFIG_HOTPLUG_CPU
static int xen_pv_cpu_disable(void)
{
@@ -364,18 +362,18 @@ static int xen_pv_cpu_disable(void)
static void xen_pv_cpu_die(unsigned int cpu)
{
- while (HYPERVISOR_vcpu_op(VCPUOP_is_up,
- xen_vcpu_nr(cpu), NULL)) {
+ while (HYPERVISOR_vcpu_op(VCPUOP_is_up, xen_vcpu_nr(cpu), NULL)) {
__set_current_state(TASK_UNINTERRUPTIBLE);
schedule_timeout(HZ/10);
}
+}
- if (common_cpu_die(cpu) == 0) {
- xen_smp_intr_free(cpu);
- xen_uninit_lock_cpu(cpu);
- xen_teardown_timer(cpu);
- xen_pmu_finish(cpu);
- }
+static void xen_pv_cleanup_dead_cpu(unsigned int cpu)
+{
+ xen_smp_intr_free(cpu);
+ xen_uninit_lock_cpu(cpu);
+ xen_teardown_timer(cpu);
+ xen_pmu_finish(cpu);
}
static void __noreturn xen_pv_play_dead(void) /* used only with HOTPLUG_CPU */
@@ -397,6 +395,11 @@ static void xen_pv_cpu_die(unsigned int
BUG();
}
+static void xen_pv_cleanup_dead_cpu(unsigned int cpu)
+{
+ BUG();
+}
+
static void __noreturn xen_pv_play_dead(void)
{
BUG();
@@ -437,6 +440,8 @@ static const struct smp_ops xen_smp_ops
.cpu_up = xen_pv_cpu_up,
.cpu_die = xen_pv_cpu_die,
+ .cleanup_dead_cpu = xen_pv_cleanup_dead_cpu,
+ .poll_sync_state = xen_pv_poll_sync_state,
.cpu_disable = xen_pv_cpu_disable,
.play_dead = xen_pv_play_dead,
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 20/37] cpu/hotplug: Remove cpu_report_state() and related unused cruft
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-14 23:44 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
No more users.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/cpu.h | 2 -
kernel/smpboot.c | 90 ----------------------------------------------------
2 files changed, 92 deletions(-)
--- a/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@ -184,8 +184,6 @@ void arch_cpu_idle_enter(void);
void arch_cpu_idle_exit(void);
void arch_cpu_idle_dead(void);
-int cpu_report_state(int cpu);
-int cpu_check_up_prepare(int cpu);
void cpu_set_state_online(int cpu);
void play_idle_precise(u64 duration_ns, u64 latency_ns);
--- a/kernel/smpboot.c
+++ b/kernel/smpboot.c
@@ -329,97 +329,7 @@ EXPORT_SYMBOL_GPL(smpboot_unregister_per
#ifndef CONFIG_HOTPLUG_CORE_SYNC
static DEFINE_PER_CPU(atomic_t, cpu_hotplug_state) = ATOMIC_INIT(CPU_POST_DEAD);
-/*
- * Called to poll specified CPU's state, for example, when waiting for
- * a CPU to come online.
- */
-int cpu_report_state(int cpu)
-{
- return atomic_read(&per_cpu(cpu_hotplug_state, cpu));
-}
-
-/*
- * If CPU has died properly, set its state to CPU_UP_PREPARE and
- * return success. Otherwise, return -EBUSY if the CPU died after
- * cpu_wait_death() timed out. And yet otherwise again, return -EAGAIN
- * if cpu_wait_death() timed out and the CPU still hasn't gotten around
- * to dying. In the latter two cases, the CPU might not be set up
- * properly, but it is up to the arch-specific code to decide.
- * Finally, -EIO indicates an unanticipated problem.
- *
- * Note that it is permissible to omit this call entirely, as is
- * done in architectures that do no CPU-hotplug error checking.
- */
-int cpu_check_up_prepare(int cpu)
-{
- if (!IS_ENABLED(CONFIG_HOTPLUG_CPU)) {
- atomic_set(&per_cpu(cpu_hotplug_state, cpu), CPU_UP_PREPARE);
- return 0;
- }
-
- switch (atomic_read(&per_cpu(cpu_hotplug_state, cpu))) {
-
- case CPU_POST_DEAD:
-
- /* The CPU died properly, so just start it up again. */
- atomic_set(&per_cpu(cpu_hotplug_state, cpu), CPU_UP_PREPARE);
- return 0;
-
- case CPU_DEAD_FROZEN:
-
- /*
- * Timeout during CPU death, so let caller know.
- * The outgoing CPU completed its processing, but after
- * cpu_wait_death() timed out and reported the error. The
- * caller is free to proceed, in which case the state
- * will be reset properly by cpu_set_state_online().
- * Proceeding despite this -EBUSY return makes sense
- * for systems where the outgoing CPUs take themselves
- * offline, with no post-death manipulation required from
- * a surviving CPU.
- */
- return -EBUSY;
-
- case CPU_BROKEN:
-
- /*
- * The most likely reason we got here is that there was
- * a timeout during CPU death, and the outgoing CPU never
- * did complete its processing. This could happen on
- * a virtualized system if the outgoing VCPU gets preempted
- * for more than five seconds, and the user attempts to
- * immediately online that same CPU. Trying again later
- * might return -EBUSY above, hence -EAGAIN.
- */
- return -EAGAIN;
-
- case CPU_UP_PREPARE:
- /*
- * Timeout while waiting for the CPU to show up. Allow to try
- * again later.
- */
- return 0;
-
- default:
-
- /* Should not happen. Famous last words. */
- return -EIO;
- }
-}
-
-/*
- * Mark the specified CPU online.
- *
- * Note that it is permissible to omit this call entirely, as is
- * done in architectures that do no CPU-hotplug error checking.
- */
-void cpu_set_state_online(int cpu)
-{
- (void)atomic_xchg(&per_cpu(cpu_hotplug_state, cpu), CPU_ONLINE);
-}
-
#ifdef CONFIG_HOTPLUG_CPU
-
/*
* Wait for the specified CPU to exit the idle loop and die.
*/
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 20/37] cpu/hotplug: Remove cpu_report_state() and related unused cruft
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
No more users.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/cpu.h | 2 -
kernel/smpboot.c | 90 ----------------------------------------------------
2 files changed, 92 deletions(-)
--- a/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@ -184,8 +184,6 @@ void arch_cpu_idle_enter(void);
void arch_cpu_idle_exit(void);
void arch_cpu_idle_dead(void);
-int cpu_report_state(int cpu);
-int cpu_check_up_prepare(int cpu);
void cpu_set_state_online(int cpu);
void play_idle_precise(u64 duration_ns, u64 latency_ns);
--- a/kernel/smpboot.c
+++ b/kernel/smpboot.c
@@ -329,97 +329,7 @@ EXPORT_SYMBOL_GPL(smpboot_unregister_per
#ifndef CONFIG_HOTPLUG_CORE_SYNC
static DEFINE_PER_CPU(atomic_t, cpu_hotplug_state) = ATOMIC_INIT(CPU_POST_DEAD);
-/*
- * Called to poll specified CPU's state, for example, when waiting for
- * a CPU to come online.
- */
-int cpu_report_state(int cpu)
-{
- return atomic_read(&per_cpu(cpu_hotplug_state, cpu));
-}
-
-/*
- * If CPU has died properly, set its state to CPU_UP_PREPARE and
- * return success. Otherwise, return -EBUSY if the CPU died after
- * cpu_wait_death() timed out. And yet otherwise again, return -EAGAIN
- * if cpu_wait_death() timed out and the CPU still hasn't gotten around
- * to dying. In the latter two cases, the CPU might not be set up
- * properly, but it is up to the arch-specific code to decide.
- * Finally, -EIO indicates an unanticipated problem.
- *
- * Note that it is permissible to omit this call entirely, as is
- * done in architectures that do no CPU-hotplug error checking.
- */
-int cpu_check_up_prepare(int cpu)
-{
- if (!IS_ENABLED(CONFIG_HOTPLUG_CPU)) {
- atomic_set(&per_cpu(cpu_hotplug_state, cpu), CPU_UP_PREPARE);
- return 0;
- }
-
- switch (atomic_read(&per_cpu(cpu_hotplug_state, cpu))) {
-
- case CPU_POST_DEAD:
-
- /* The CPU died properly, so just start it up again. */
- atomic_set(&per_cpu(cpu_hotplug_state, cpu), CPU_UP_PREPARE);
- return 0;
-
- case CPU_DEAD_FROZEN:
-
- /*
- * Timeout during CPU death, so let caller know.
- * The outgoing CPU completed its processing, but after
- * cpu_wait_death() timed out and reported the error. The
- * caller is free to proceed, in which case the state
- * will be reset properly by cpu_set_state_online().
- * Proceeding despite this -EBUSY return makes sense
- * for systems where the outgoing CPUs take themselves
- * offline, with no post-death manipulation required from
- * a surviving CPU.
- */
- return -EBUSY;
-
- case CPU_BROKEN:
-
- /*
- * The most likely reason we got here is that there was
- * a timeout during CPU death, and the outgoing CPU never
- * did complete its processing. This could happen on
- * a virtualized system if the outgoing VCPU gets preempted
- * for more than five seconds, and the user attempts to
- * immediately online that same CPU. Trying again later
- * might return -EBUSY above, hence -EAGAIN.
- */
- return -EAGAIN;
-
- case CPU_UP_PREPARE:
- /*
- * Timeout while waiting for the CPU to show up. Allow to try
- * again later.
- */
- return 0;
-
- default:
-
- /* Should not happen. Famous last words. */
- return -EIO;
- }
-}
-
-/*
- * Mark the specified CPU online.
- *
- * Note that it is permissible to omit this call entirely, as is
- * done in architectures that do no CPU-hotplug error checking.
- */
-void cpu_set_state_online(int cpu)
-{
- (void)atomic_xchg(&per_cpu(cpu_hotplug_state, cpu), CPU_ONLINE);
-}
-
#ifdef CONFIG_HOTPLUG_CPU
-
/*
* Wait for the specified CPU to exit the idle loop and die.
*/
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 20/37] cpu/hotplug: Remove cpu_report_state() and related unused cruft
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
No more users.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/cpu.h | 2 -
kernel/smpboot.c | 90 ----------------------------------------------------
2 files changed, 92 deletions(-)
--- a/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@ -184,8 +184,6 @@ void arch_cpu_idle_enter(void);
void arch_cpu_idle_exit(void);
void arch_cpu_idle_dead(void);
-int cpu_report_state(int cpu);
-int cpu_check_up_prepare(int cpu);
void cpu_set_state_online(int cpu);
void play_idle_precise(u64 duration_ns, u64 latency_ns);
--- a/kernel/smpboot.c
+++ b/kernel/smpboot.c
@@ -329,97 +329,7 @@ EXPORT_SYMBOL_GPL(smpboot_unregister_per
#ifndef CONFIG_HOTPLUG_CORE_SYNC
static DEFINE_PER_CPU(atomic_t, cpu_hotplug_state) = ATOMIC_INIT(CPU_POST_DEAD);
-/*
- * Called to poll specified CPU's state, for example, when waiting for
- * a CPU to come online.
- */
-int cpu_report_state(int cpu)
-{
- return atomic_read(&per_cpu(cpu_hotplug_state, cpu));
-}
-
-/*
- * If CPU has died properly, set its state to CPU_UP_PREPARE and
- * return success. Otherwise, return -EBUSY if the CPU died after
- * cpu_wait_death() timed out. And yet otherwise again, return -EAGAIN
- * if cpu_wait_death() timed out and the CPU still hasn't gotten around
- * to dying. In the latter two cases, the CPU might not be set up
- * properly, but it is up to the arch-specific code to decide.
- * Finally, -EIO indicates an unanticipated problem.
- *
- * Note that it is permissible to omit this call entirely, as is
- * done in architectures that do no CPU-hotplug error checking.
- */
-int cpu_check_up_prepare(int cpu)
-{
- if (!IS_ENABLED(CONFIG_HOTPLUG_CPU)) {
- atomic_set(&per_cpu(cpu_hotplug_state, cpu), CPU_UP_PREPARE);
- return 0;
- }
-
- switch (atomic_read(&per_cpu(cpu_hotplug_state, cpu))) {
-
- case CPU_POST_DEAD:
-
- /* The CPU died properly, so just start it up again. */
- atomic_set(&per_cpu(cpu_hotplug_state, cpu), CPU_UP_PREPARE);
- return 0;
-
- case CPU_DEAD_FROZEN:
-
- /*
- * Timeout during CPU death, so let caller know.
- * The outgoing CPU completed its processing, but after
- * cpu_wait_death() timed out and reported the error. The
- * caller is free to proceed, in which case the state
- * will be reset properly by cpu_set_state_online().
- * Proceeding despite this -EBUSY return makes sense
- * for systems where the outgoing CPUs take themselves
- * offline, with no post-death manipulation required from
- * a surviving CPU.
- */
- return -EBUSY;
-
- case CPU_BROKEN:
-
- /*
- * The most likely reason we got here is that there was
- * a timeout during CPU death, and the outgoing CPU never
- * did complete its processing. This could happen on
- * a virtualized system if the outgoing VCPU gets preempted
- * for more than five seconds, and the user attempts to
- * immediately online that same CPU. Trying again later
- * might return -EBUSY above, hence -EAGAIN.
- */
- return -EAGAIN;
-
- case CPU_UP_PREPARE:
- /*
- * Timeout while waiting for the CPU to show up. Allow to try
- * again later.
- */
- return 0;
-
- default:
-
- /* Should not happen. Famous last words. */
- return -EIO;
- }
-}
-
-/*
- * Mark the specified CPU online.
- *
- * Note that it is permissible to omit this call entirely, as is
- * done in architectures that do no CPU-hotplug error checking.
- */
-void cpu_set_state_online(int cpu)
-{
- (void)atomic_xchg(&per_cpu(cpu_hotplug_state, cpu), CPU_ONLINE);
-}
-
#ifdef CONFIG_HOTPLUG_CPU
-
/*
* Wait for the specified CPU to exit the idle loop and die.
*/
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 21/37] ARM: smp: Switch to hotplug core state synchronization
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-14 23:44 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Russell King, Arnd Bergmann,
linux-arm-kernel, David Woodhouse, Usama Arif, Juergen Gross,
Boris Ostrovsky, xen-devel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Switch to the CPU hotplug core state tracking and synchronization
mechanim. No functional change intended.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: linux-arm-kernel@lists.infradead.org
---
arch/arm/Kconfig | 1 +
arch/arm/include/asm/smp.h | 2 +-
arch/arm/kernel/smp.c | 18 +++++++-----------
3 files changed, 9 insertions(+), 12 deletions(-)
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -124,6 +124,7 @@ config ARM
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_UID16
select HAVE_VIRT_CPU_ACCOUNTING_GEN
+ select HOTPLUG_CORE_SYNC_DEAD if HOTPLUG_CPU
select IRQ_FORCED_THREADING
select MODULES_USE_ELF_REL
select NEED_DMA_MAP_STATE
--- a/arch/arm/include/asm/smp.h
+++ b/arch/arm/include/asm/smp.h
@@ -64,7 +64,7 @@ extern void secondary_startup_arm(void);
extern int __cpu_disable(void);
-extern void __cpu_die(unsigned int cpu);
+static inline void __cpu_die(unsigned int cpu) { }
extern void arch_send_call_function_single_ipi(int cpu);
extern void arch_send_call_function_ipi_mask(const struct cpumask *mask);
--- a/arch/arm/kernel/smp.c
+++ b/arch/arm/kernel/smp.c
@@ -289,15 +289,11 @@ int __cpu_disable(void)
}
/*
- * called on the thread which is asking for a CPU to be shutdown -
- * waits until shutdown has completed, or it is timed out.
+ * called on the thread which is asking for a CPU to be shutdown after the
+ * shutdown completed.
*/
-void __cpu_die(unsigned int cpu)
+void arch_cpuhp_cleanup_dead_cpu(unsigned int cpu)
{
- if (!cpu_wait_death(cpu, 5)) {
- pr_err("CPU%u: cpu didn't die\n", cpu);
- return;
- }
pr_debug("CPU%u: shutdown\n", cpu);
clear_tasks_mm_cpumask(cpu);
@@ -337,11 +333,11 @@ void arch_cpu_idle_dead(void)
flush_cache_louis();
/*
- * Tell __cpu_die() that this CPU is now safe to dispose of. Once
- * this returns, power and/or clocks can be removed at any point
- * from this CPU and its cache by platform_cpu_kill().
+ * Tell cpuhp_bp_sync_dead() that this CPU is now safe to dispose
+ * of. Once this returns, power and/or clocks can be removed at
+ * any point from this CPU and its cache by platform_cpu_kill().
*/
- (void)cpu_report_death();
+ cpuhp_ap_report_dead();
/*
* Ensure that the cache lines associated with that completion are
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 21/37] ARM: smp: Switch to hotplug core state synchronization
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Russell King, Arnd Bergmann,
linux-arm-kernel, David Woodhouse, Usama Arif, Juergen Gross,
Boris Ostrovsky, xen-devel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Switch to the CPU hotplug core state tracking and synchronization
mechanim. No functional change intended.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: linux-arm-kernel@lists.infradead.org
---
arch/arm/Kconfig | 1 +
arch/arm/include/asm/smp.h | 2 +-
arch/arm/kernel/smp.c | 18 +++++++-----------
3 files changed, 9 insertions(+), 12 deletions(-)
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -124,6 +124,7 @@ config ARM
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_UID16
select HAVE_VIRT_CPU_ACCOUNTING_GEN
+ select HOTPLUG_CORE_SYNC_DEAD if HOTPLUG_CPU
select IRQ_FORCED_THREADING
select MODULES_USE_ELF_REL
select NEED_DMA_MAP_STATE
--- a/arch/arm/include/asm/smp.h
+++ b/arch/arm/include/asm/smp.h
@@ -64,7 +64,7 @@ extern void secondary_startup_arm(void);
extern int __cpu_disable(void);
-extern void __cpu_die(unsigned int cpu);
+static inline void __cpu_die(unsigned int cpu) { }
extern void arch_send_call_function_single_ipi(int cpu);
extern void arch_send_call_function_ipi_mask(const struct cpumask *mask);
--- a/arch/arm/kernel/smp.c
+++ b/arch/arm/kernel/smp.c
@@ -289,15 +289,11 @@ int __cpu_disable(void)
}
/*
- * called on the thread which is asking for a CPU to be shutdown -
- * waits until shutdown has completed, or it is timed out.
+ * called on the thread which is asking for a CPU to be shutdown after the
+ * shutdown completed.
*/
-void __cpu_die(unsigned int cpu)
+void arch_cpuhp_cleanup_dead_cpu(unsigned int cpu)
{
- if (!cpu_wait_death(cpu, 5)) {
- pr_err("CPU%u: cpu didn't die\n", cpu);
- return;
- }
pr_debug("CPU%u: shutdown\n", cpu);
clear_tasks_mm_cpumask(cpu);
@@ -337,11 +333,11 @@ void arch_cpu_idle_dead(void)
flush_cache_louis();
/*
- * Tell __cpu_die() that this CPU is now safe to dispose of. Once
- * this returns, power and/or clocks can be removed at any point
- * from this CPU and its cache by platform_cpu_kill().
+ * Tell cpuhp_bp_sync_dead() that this CPU is now safe to dispose
+ * of. Once this returns, power and/or clocks can be removed at
+ * any point from this CPU and its cache by platform_cpu_kill().
*/
- (void)cpu_report_death();
+ cpuhp_ap_report_dead();
/*
* Ensure that the cache lines associated with that completion are
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 21/37] ARM: smp: Switch to hotplug core state synchronization
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Russell King, Arnd Bergmann,
linux-arm-kernel, David Woodhouse, Usama Arif, Juergen Gross,
Boris Ostrovsky, xen-devel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Switch to the CPU hotplug core state tracking and synchronization
mechanim. No functional change intended.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: linux-arm-kernel@lists.infradead.org
---
arch/arm/Kconfig | 1 +
arch/arm/include/asm/smp.h | 2 +-
arch/arm/kernel/smp.c | 18 +++++++-----------
3 files changed, 9 insertions(+), 12 deletions(-)
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -124,6 +124,7 @@ config ARM
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_UID16
select HAVE_VIRT_CPU_ACCOUNTING_GEN
+ select HOTPLUG_CORE_SYNC_DEAD if HOTPLUG_CPU
select IRQ_FORCED_THREADING
select MODULES_USE_ELF_REL
select NEED_DMA_MAP_STATE
--- a/arch/arm/include/asm/smp.h
+++ b/arch/arm/include/asm/smp.h
@@ -64,7 +64,7 @@ extern void secondary_startup_arm(void);
extern int __cpu_disable(void);
-extern void __cpu_die(unsigned int cpu);
+static inline void __cpu_die(unsigned int cpu) { }
extern void arch_send_call_function_single_ipi(int cpu);
extern void arch_send_call_function_ipi_mask(const struct cpumask *mask);
--- a/arch/arm/kernel/smp.c
+++ b/arch/arm/kernel/smp.c
@@ -289,15 +289,11 @@ int __cpu_disable(void)
}
/*
- * called on the thread which is asking for a CPU to be shutdown -
- * waits until shutdown has completed, or it is timed out.
+ * called on the thread which is asking for a CPU to be shutdown after the
+ * shutdown completed.
*/
-void __cpu_die(unsigned int cpu)
+void arch_cpuhp_cleanup_dead_cpu(unsigned int cpu)
{
- if (!cpu_wait_death(cpu, 5)) {
- pr_err("CPU%u: cpu didn't die\n", cpu);
- return;
- }
pr_debug("CPU%u: shutdown\n", cpu);
clear_tasks_mm_cpumask(cpu);
@@ -337,11 +333,11 @@ void arch_cpu_idle_dead(void)
flush_cache_louis();
/*
- * Tell __cpu_die() that this CPU is now safe to dispose of. Once
- * this returns, power and/or clocks can be removed at any point
- * from this CPU and its cache by platform_cpu_kill().
+ * Tell cpuhp_bp_sync_dead() that this CPU is now safe to dispose
+ * of. Once this returns, power and/or clocks can be removed at
+ * any point from this CPU and its cache by platform_cpu_kill().
*/
- (void)cpu_report_death();
+ cpuhp_ap_report_dead();
/*
* Ensure that the cache lines associated with that completion are
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 22/37] arm64: smp: Switch to hotplug core state synchronization
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-14 23:44 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Catalin Marinas, Will Deacon,
linux-arm-kernel, David Woodhouse, Usama Arif, Juergen Gross,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Switch to the CPU hotplug core state tracking and synchronization
mechanim. No functional change intended.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will@kernel.org>
Cc: linux-arm-kernel@lists.infradead.org
---
arch/arm64/Kconfig | 1 +
arch/arm64/include/asm/smp.h | 2 +-
arch/arm64/kernel/smp.c | 14 +++++---------
3 files changed, 7 insertions(+), 10 deletions(-)
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -216,6 +216,7 @@ config ARM64
select HAVE_KPROBES
select HAVE_KRETPROBES
select HAVE_GENERIC_VDSO
+ select HOTPLUG_CORE_SYNC_DEAD if HOTPLUG_CPU
select IRQ_DOMAIN
select IRQ_FORCED_THREADING
select KASAN_VMALLOC if KASAN
--- a/arch/arm64/include/asm/smp.h
+++ b/arch/arm64/include/asm/smp.h
@@ -99,7 +99,7 @@ static inline void arch_send_wakeup_ipi_
extern int __cpu_disable(void);
-extern void __cpu_die(unsigned int cpu);
+static inline void __cpu_die(unsigned int cpu) { }
extern void cpu_die(void);
extern void cpu_die_early(void);
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -333,17 +333,13 @@ static int op_cpu_kill(unsigned int cpu)
}
/*
- * called on the thread which is asking for a CPU to be shutdown -
- * waits until shutdown has completed, or it is timed out.
+ * Called on the thread which is asking for a CPU to be shutdown after the
+ * shutdown completed.
*/
-void __cpu_die(unsigned int cpu)
+void arch_cpuhp_cleanup_dead_cpu(unsigned int cpu)
{
int err;
- if (!cpu_wait_death(cpu, 5)) {
- pr_crit("CPU%u: cpu didn't die\n", cpu);
- return;
- }
pr_debug("CPU%u: shutdown\n", cpu);
/*
@@ -370,8 +366,8 @@ void cpu_die(void)
local_daif_mask();
- /* Tell __cpu_die() that this CPU is now safe to dispose of */
- (void)cpu_report_death();
+ /* Tell cpuhp_bp_sync_dead() that this CPU is now safe to dispose of */
+ cpuhp_ap_report_dead();
/*
* Actually shutdown the CPU. This must never fail. The specific hotplug
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 22/37] arm64: smp: Switch to hotplug core state synchronization
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Catalin Marinas, Will Deacon,
linux-arm-kernel, David Woodhouse, Usama Arif, Juergen Gross,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Switch to the CPU hotplug core state tracking and synchronization
mechanim. No functional change intended.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will@kernel.org>
Cc: linux-arm-kernel@lists.infradead.org
---
arch/arm64/Kconfig | 1 +
arch/arm64/include/asm/smp.h | 2 +-
arch/arm64/kernel/smp.c | 14 +++++---------
3 files changed, 7 insertions(+), 10 deletions(-)
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -216,6 +216,7 @@ config ARM64
select HAVE_KPROBES
select HAVE_KRETPROBES
select HAVE_GENERIC_VDSO
+ select HOTPLUG_CORE_SYNC_DEAD if HOTPLUG_CPU
select IRQ_DOMAIN
select IRQ_FORCED_THREADING
select KASAN_VMALLOC if KASAN
--- a/arch/arm64/include/asm/smp.h
+++ b/arch/arm64/include/asm/smp.h
@@ -99,7 +99,7 @@ static inline void arch_send_wakeup_ipi_
extern int __cpu_disable(void);
-extern void __cpu_die(unsigned int cpu);
+static inline void __cpu_die(unsigned int cpu) { }
extern void cpu_die(void);
extern void cpu_die_early(void);
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -333,17 +333,13 @@ static int op_cpu_kill(unsigned int cpu)
}
/*
- * called on the thread which is asking for a CPU to be shutdown -
- * waits until shutdown has completed, or it is timed out.
+ * Called on the thread which is asking for a CPU to be shutdown after the
+ * shutdown completed.
*/
-void __cpu_die(unsigned int cpu)
+void arch_cpuhp_cleanup_dead_cpu(unsigned int cpu)
{
int err;
- if (!cpu_wait_death(cpu, 5)) {
- pr_crit("CPU%u: cpu didn't die\n", cpu);
- return;
- }
pr_debug("CPU%u: shutdown\n", cpu);
/*
@@ -370,8 +366,8 @@ void cpu_die(void)
local_daif_mask();
- /* Tell __cpu_die() that this CPU is now safe to dispose of */
- (void)cpu_report_death();
+ /* Tell cpuhp_bp_sync_dead() that this CPU is now safe to dispose of */
+ cpuhp_ap_report_dead();
/*
* Actually shutdown the CPU. This must never fail. The specific hotplug
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 22/37] arm64: smp: Switch to hotplug core state synchronization
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Catalin Marinas, Will Deacon,
linux-arm-kernel, David Woodhouse, Usama Arif, Juergen Gross,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Switch to the CPU hotplug core state tracking and synchronization
mechanim. No functional change intended.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will@kernel.org>
Cc: linux-arm-kernel@lists.infradead.org
---
arch/arm64/Kconfig | 1 +
arch/arm64/include/asm/smp.h | 2 +-
arch/arm64/kernel/smp.c | 14 +++++---------
3 files changed, 7 insertions(+), 10 deletions(-)
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -216,6 +216,7 @@ config ARM64
select HAVE_KPROBES
select HAVE_KRETPROBES
select HAVE_GENERIC_VDSO
+ select HOTPLUG_CORE_SYNC_DEAD if HOTPLUG_CPU
select IRQ_DOMAIN
select IRQ_FORCED_THREADING
select KASAN_VMALLOC if KASAN
--- a/arch/arm64/include/asm/smp.h
+++ b/arch/arm64/include/asm/smp.h
@@ -99,7 +99,7 @@ static inline void arch_send_wakeup_ipi_
extern int __cpu_disable(void);
-extern void __cpu_die(unsigned int cpu);
+static inline void __cpu_die(unsigned int cpu) { }
extern void cpu_die(void);
extern void cpu_die_early(void);
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -333,17 +333,13 @@ static int op_cpu_kill(unsigned int cpu)
}
/*
- * called on the thread which is asking for a CPU to be shutdown -
- * waits until shutdown has completed, or it is timed out.
+ * Called on the thread which is asking for a CPU to be shutdown after the
+ * shutdown completed.
*/
-void __cpu_die(unsigned int cpu)
+void arch_cpuhp_cleanup_dead_cpu(unsigned int cpu)
{
int err;
- if (!cpu_wait_death(cpu, 5)) {
- pr_crit("CPU%u: cpu didn't die\n", cpu);
- return;
- }
pr_debug("CPU%u: shutdown\n", cpu);
/*
@@ -370,8 +366,8 @@ void cpu_die(void)
local_daif_mask();
- /* Tell __cpu_die() that this CPU is now safe to dispose of */
- (void)cpu_report_death();
+ /* Tell cpuhp_bp_sync_dead() that this CPU is now safe to dispose of */
+ cpuhp_ap_report_dead();
/*
* Actually shutdown the CPU. This must never fail. The specific hotplug
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 23/37] csky/smp: Switch to hotplug core state synchronization
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-14 23:44 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Guo Ren, linux-csky,
David Woodhouse, Usama Arif, Juergen Gross, Boris Ostrovsky,
xen-devel, Russell King, Arnd Bergmann, linux-arm-kernel,
Catalin Marinas, Will Deacon, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Switch to the CPU hotplug core state tracking and synchronization
mechanim. No functional change intended.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Guo Ren <guoren@kernel.org>
Cc: linux-csky@vger.kernel.org
---
arch/csky/Kconfig | 1 +
arch/csky/include/asm/smp.h | 2 +-
arch/csky/kernel/smp.c | 8 ++------
3 files changed, 4 insertions(+), 7 deletions(-)
--- a/arch/csky/Kconfig
+++ b/arch/csky/Kconfig
@@ -96,6 +96,7 @@ config CSKY
select HAVE_REGS_AND_STACK_ACCESS_API
select HAVE_STACKPROTECTOR
select HAVE_SYSCALL_TRACEPOINTS
+ select HOTPLUG_CORE_SYNC_DEAD if HOTPLUG_CPU
select MAY_HAVE_SPARSE_IRQ
select MODULES_USE_ELF_RELA if MODULES
select OF
--- a/arch/csky/include/asm/smp.h
+++ b/arch/csky/include/asm/smp.h
@@ -23,7 +23,7 @@ void __init set_send_ipi(void (*func)(co
int __cpu_disable(void);
-void __cpu_die(unsigned int cpu);
+static inline void __cpu_die(unsigned int cpu) { }
#endif /* CONFIG_SMP */
--- a/arch/csky/kernel/smp.c
+++ b/arch/csky/kernel/smp.c
@@ -291,12 +291,8 @@ int __cpu_disable(void)
return 0;
}
-void __cpu_die(unsigned int cpu)
+void arch_cpuhp_cleanup_dead_cpu(unsigned int cpu)
{
- if (!cpu_wait_death(cpu, 5)) {
- pr_crit("CPU%u: shutdown failed\n", cpu);
- return;
- }
pr_notice("CPU%u: shutdown\n", cpu);
}
@@ -304,7 +300,7 @@ void arch_cpu_idle_dead(void)
{
idle_task_exit();
- cpu_report_death();
+ cpuhp_ap_report_dead();
while (!secondary_stack)
arch_cpu_idle();
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 23/37] csky/smp: Switch to hotplug core state synchronization
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Guo Ren, linux-csky,
David Woodhouse, Usama Arif, Juergen Gross, Boris Ostrovsky,
xen-devel, Russell King, Arnd Bergmann, linux-arm-kernel,
Catalin Marinas, Will Deacon, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Switch to the CPU hotplug core state tracking and synchronization
mechanim. No functional change intended.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Guo Ren <guoren@kernel.org>
Cc: linux-csky@vger.kernel.org
---
arch/csky/Kconfig | 1 +
arch/csky/include/asm/smp.h | 2 +-
arch/csky/kernel/smp.c | 8 ++------
3 files changed, 4 insertions(+), 7 deletions(-)
--- a/arch/csky/Kconfig
+++ b/arch/csky/Kconfig
@@ -96,6 +96,7 @@ config CSKY
select HAVE_REGS_AND_STACK_ACCESS_API
select HAVE_STACKPROTECTOR
select HAVE_SYSCALL_TRACEPOINTS
+ select HOTPLUG_CORE_SYNC_DEAD if HOTPLUG_CPU
select MAY_HAVE_SPARSE_IRQ
select MODULES_USE_ELF_RELA if MODULES
select OF
--- a/arch/csky/include/asm/smp.h
+++ b/arch/csky/include/asm/smp.h
@@ -23,7 +23,7 @@ void __init set_send_ipi(void (*func)(co
int __cpu_disable(void);
-void __cpu_die(unsigned int cpu);
+static inline void __cpu_die(unsigned int cpu) { }
#endif /* CONFIG_SMP */
--- a/arch/csky/kernel/smp.c
+++ b/arch/csky/kernel/smp.c
@@ -291,12 +291,8 @@ int __cpu_disable(void)
return 0;
}
-void __cpu_die(unsigned int cpu)
+void arch_cpuhp_cleanup_dead_cpu(unsigned int cpu)
{
- if (!cpu_wait_death(cpu, 5)) {
- pr_crit("CPU%u: shutdown failed\n", cpu);
- return;
- }
pr_notice("CPU%u: shutdown\n", cpu);
}
@@ -304,7 +300,7 @@ void arch_cpu_idle_dead(void)
{
idle_task_exit();
- cpu_report_death();
+ cpuhp_ap_report_dead();
while (!secondary_stack)
arch_cpu_idle();
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 23/37] csky/smp: Switch to hotplug core state synchronization
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Guo Ren, linux-csky,
David Woodhouse, Usama Arif, Juergen Gross, Boris Ostrovsky,
xen-devel, Russell King, Arnd Bergmann, linux-arm-kernel,
Catalin Marinas, Will Deacon, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Switch to the CPU hotplug core state tracking and synchronization
mechanim. No functional change intended.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Guo Ren <guoren@kernel.org>
Cc: linux-csky@vger.kernel.org
---
arch/csky/Kconfig | 1 +
arch/csky/include/asm/smp.h | 2 +-
arch/csky/kernel/smp.c | 8 ++------
3 files changed, 4 insertions(+), 7 deletions(-)
--- a/arch/csky/Kconfig
+++ b/arch/csky/Kconfig
@@ -96,6 +96,7 @@ config CSKY
select HAVE_REGS_AND_STACK_ACCESS_API
select HAVE_STACKPROTECTOR
select HAVE_SYSCALL_TRACEPOINTS
+ select HOTPLUG_CORE_SYNC_DEAD if HOTPLUG_CPU
select MAY_HAVE_SPARSE_IRQ
select MODULES_USE_ELF_RELA if MODULES
select OF
--- a/arch/csky/include/asm/smp.h
+++ b/arch/csky/include/asm/smp.h
@@ -23,7 +23,7 @@ void __init set_send_ipi(void (*func)(co
int __cpu_disable(void);
-void __cpu_die(unsigned int cpu);
+static inline void __cpu_die(unsigned int cpu) { }
#endif /* CONFIG_SMP */
--- a/arch/csky/kernel/smp.c
+++ b/arch/csky/kernel/smp.c
@@ -291,12 +291,8 @@ int __cpu_disable(void)
return 0;
}
-void __cpu_die(unsigned int cpu)
+void arch_cpuhp_cleanup_dead_cpu(unsigned int cpu)
{
- if (!cpu_wait_death(cpu, 5)) {
- pr_crit("CPU%u: shutdown failed\n", cpu);
- return;
- }
pr_notice("CPU%u: shutdown\n", cpu);
}
@@ -304,7 +300,7 @@ void arch_cpu_idle_dead(void)
{
idle_task_exit();
- cpu_report_death();
+ cpuhp_ap_report_dead();
while (!secondary_stack)
arch_cpu_idle();
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 24/37] MIPS: SMP_CPS: Switch to hotplug core state synchronization
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-14 23:44 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Thomas Bogendoerfer,
linux-mips, David Woodhouse, Usama Arif, Juergen Gross,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, James E.J. Bottomley, Helge Deller, linux-parisc,
Paul Walmsley, Palmer Dabbelt, linux-riscv, Mark Rutland,
Sabin Rapan
Switch to the CPU hotplug core state tracking and synchronization
mechanim. This unfortunately requires to add dead reporting to the non CPS
platforms as CPS is the only user, but it allows an overall consolidation
of this functionality.
No functional change intended.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Thomas Bogendoerfer <tsbogend@alpha.franken.de>
Cc: linux-mips@vger.kernel.org
---
arch/mips/Kconfig | 1 +
arch/mips/cavium-octeon/smp.c | 1 +
arch/mips/include/asm/smp-ops.h | 1 +
arch/mips/kernel/smp-bmips.c | 1 +
arch/mips/kernel/smp-cps.c | 14 +++++---------
arch/mips/kernel/smp.c | 8 ++++++++
arch/mips/loongson64/smp.c | 1 +
7 files changed, 18 insertions(+), 9 deletions(-)
--- a/arch/mips/Kconfig
+++ b/arch/mips/Kconfig
@@ -2356,6 +2356,7 @@ config MIPS_CPS
select MIPS_CM
select MIPS_CPS_PM if HOTPLUG_CPU
select SMP
+ select HOTPLUG_CORE_SYNC_DEAD if HOTPLUG_CPU
select SYNC_R4K if (CEVT_R4K || CSRC_R4K)
select SYS_SUPPORTS_HOTPLUG_CPU
select SYS_SUPPORTS_SCHED_SMT if CPU_MIPSR6
--- a/arch/mips/cavium-octeon/smp.c
+++ b/arch/mips/cavium-octeon/smp.c
@@ -344,6 +344,7 @@ void play_dead(void)
int cpu = cpu_number_map(cvmx_get_core_num());
idle_task_exit();
+ cpuhp_ap_report_dead();
octeon_processor_boot = 0xff;
per_cpu(cpu_state, cpu) = CPU_DEAD;
--- a/arch/mips/include/asm/smp-ops.h
+++ b/arch/mips/include/asm/smp-ops.h
@@ -33,6 +33,7 @@ struct plat_smp_ops {
#ifdef CONFIG_HOTPLUG_CPU
int (*cpu_disable)(void);
void (*cpu_die)(unsigned int cpu);
+ void (*cleanup_dead_cpu)(unsigned cpu);
#endif
#ifdef CONFIG_KEXEC
void (*kexec_nonboot_cpu)(void);
--- a/arch/mips/kernel/smp-bmips.c
+++ b/arch/mips/kernel/smp-bmips.c
@@ -390,6 +390,7 @@ static void bmips_cpu_die(unsigned int c
void __ref play_dead(void)
{
idle_task_exit();
+ cpuhp_ap_report_dead();
/* flush data cache */
_dma_cache_wback_inv(0, ~0);
--- a/arch/mips/kernel/smp-cps.c
+++ b/arch/mips/kernel/smp-cps.c
@@ -497,8 +497,7 @@ void play_dead(void)
}
}
- /* This CPU has chosen its way out */
- (void)cpu_report_death();
+ cpuhp_ap_report_dead();
cps_shutdown_this_cpu(cpu_death);
@@ -521,7 +520,9 @@ static void wait_for_sibling_halt(void *
} while (!(halted & TCHALT_H));
}
-static void cps_cpu_die(unsigned int cpu)
+static void cps_cpu_die(unsigned int cpu) { }
+
+static void cps_cleanup_dead_cpu(unsigned cpu)
{
unsigned core = cpu_core(&cpu_data[cpu]);
unsigned int vpe_id = cpu_vpe_id(&cpu_data[cpu]);
@@ -529,12 +530,6 @@ static void cps_cpu_die(unsigned int cpu
unsigned stat;
int err;
- /* Wait for the cpu to choose its way out */
- if (!cpu_wait_death(cpu, 5)) {
- pr_err("CPU%u: didn't offline\n", cpu);
- return;
- }
-
/*
* Now wait for the CPU to actually offline. Without doing this that
* offlining may race with one or more of:
@@ -618,6 +613,7 @@ static const struct plat_smp_ops cps_smp
#ifdef CONFIG_HOTPLUG_CPU
.cpu_disable = cps_cpu_disable,
.cpu_die = cps_cpu_die,
+ .cleanup_dead_cpu = cps_cleanup_dead_cpu,
#endif
#ifdef CONFIG_KEXEC
.kexec_nonboot_cpu = cps_kexec_nonboot_cpu,
--- a/arch/mips/kernel/smp.c
+++ b/arch/mips/kernel/smp.c
@@ -690,6 +690,14 @@ void flush_tlb_one(unsigned long vaddr)
EXPORT_SYMBOL(flush_tlb_page);
EXPORT_SYMBOL(flush_tlb_one);
+#ifdef CONFIG_HOTPLUG_CPU
+void arch_cpuhp_cleanup_dead_cpu(unsigned int cpu)
+{
+ if (mp_ops->cleanup_dead_cpu)
+ mp_ops->cleanup_dead_cpu(cpu);
+}
+#endif
+
#ifdef CONFIG_GENERIC_CLOCKEVENTS_BROADCAST
static void tick_broadcast_callee(void *info)
--- a/arch/mips/loongson64/smp.c
+++ b/arch/mips/loongson64/smp.c
@@ -788,6 +788,7 @@ void play_dead(void)
void (*play_dead_at_ckseg1)(int *);
idle_task_exit();
+ cpuhp_ap_report_dead();
prid_imp = read_c0_prid() & PRID_IMP_MASK;
prid_rev = read_c0_prid() & PRID_REV_MASK;
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 24/37] MIPS: SMP_CPS: Switch to hotplug core state synchronization
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Thomas Bogendoerfer,
linux-mips, David Woodhouse, Usama Arif, Juergen Gross,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, James E.J. Bottomley, Helge Deller, linux-parisc,
Paul Walmsley, Palmer Dabbelt, linux-riscv, Mark Rutland,
Sabin Rapan
Switch to the CPU hotplug core state tracking and synchronization
mechanim. This unfortunately requires to add dead reporting to the non CPS
platforms as CPS is the only user, but it allows an overall consolidation
of this functionality.
No functional change intended.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Thomas Bogendoerfer <tsbogend@alpha.franken.de>
Cc: linux-mips@vger.kernel.org
---
arch/mips/Kconfig | 1 +
arch/mips/cavium-octeon/smp.c | 1 +
arch/mips/include/asm/smp-ops.h | 1 +
arch/mips/kernel/smp-bmips.c | 1 +
arch/mips/kernel/smp-cps.c | 14 +++++---------
arch/mips/kernel/smp.c | 8 ++++++++
arch/mips/loongson64/smp.c | 1 +
7 files changed, 18 insertions(+), 9 deletions(-)
--- a/arch/mips/Kconfig
+++ b/arch/mips/Kconfig
@@ -2356,6 +2356,7 @@ config MIPS_CPS
select MIPS_CM
select MIPS_CPS_PM if HOTPLUG_CPU
select SMP
+ select HOTPLUG_CORE_SYNC_DEAD if HOTPLUG_CPU
select SYNC_R4K if (CEVT_R4K || CSRC_R4K)
select SYS_SUPPORTS_HOTPLUG_CPU
select SYS_SUPPORTS_SCHED_SMT if CPU_MIPSR6
--- a/arch/mips/cavium-octeon/smp.c
+++ b/arch/mips/cavium-octeon/smp.c
@@ -344,6 +344,7 @@ void play_dead(void)
int cpu = cpu_number_map(cvmx_get_core_num());
idle_task_exit();
+ cpuhp_ap_report_dead();
octeon_processor_boot = 0xff;
per_cpu(cpu_state, cpu) = CPU_DEAD;
--- a/arch/mips/include/asm/smp-ops.h
+++ b/arch/mips/include/asm/smp-ops.h
@@ -33,6 +33,7 @@ struct plat_smp_ops {
#ifdef CONFIG_HOTPLUG_CPU
int (*cpu_disable)(void);
void (*cpu_die)(unsigned int cpu);
+ void (*cleanup_dead_cpu)(unsigned cpu);
#endif
#ifdef CONFIG_KEXEC
void (*kexec_nonboot_cpu)(void);
--- a/arch/mips/kernel/smp-bmips.c
+++ b/arch/mips/kernel/smp-bmips.c
@@ -390,6 +390,7 @@ static void bmips_cpu_die(unsigned int c
void __ref play_dead(void)
{
idle_task_exit();
+ cpuhp_ap_report_dead();
/* flush data cache */
_dma_cache_wback_inv(0, ~0);
--- a/arch/mips/kernel/smp-cps.c
+++ b/arch/mips/kernel/smp-cps.c
@@ -497,8 +497,7 @@ void play_dead(void)
}
}
- /* This CPU has chosen its way out */
- (void)cpu_report_death();
+ cpuhp_ap_report_dead();
cps_shutdown_this_cpu(cpu_death);
@@ -521,7 +520,9 @@ static void wait_for_sibling_halt(void *
} while (!(halted & TCHALT_H));
}
-static void cps_cpu_die(unsigned int cpu)
+static void cps_cpu_die(unsigned int cpu) { }
+
+static void cps_cleanup_dead_cpu(unsigned cpu)
{
unsigned core = cpu_core(&cpu_data[cpu]);
unsigned int vpe_id = cpu_vpe_id(&cpu_data[cpu]);
@@ -529,12 +530,6 @@ static void cps_cpu_die(unsigned int cpu
unsigned stat;
int err;
- /* Wait for the cpu to choose its way out */
- if (!cpu_wait_death(cpu, 5)) {
- pr_err("CPU%u: didn't offline\n", cpu);
- return;
- }
-
/*
* Now wait for the CPU to actually offline. Without doing this that
* offlining may race with one or more of:
@@ -618,6 +613,7 @@ static const struct plat_smp_ops cps_smp
#ifdef CONFIG_HOTPLUG_CPU
.cpu_disable = cps_cpu_disable,
.cpu_die = cps_cpu_die,
+ .cleanup_dead_cpu = cps_cleanup_dead_cpu,
#endif
#ifdef CONFIG_KEXEC
.kexec_nonboot_cpu = cps_kexec_nonboot_cpu,
--- a/arch/mips/kernel/smp.c
+++ b/arch/mips/kernel/smp.c
@@ -690,6 +690,14 @@ void flush_tlb_one(unsigned long vaddr)
EXPORT_SYMBOL(flush_tlb_page);
EXPORT_SYMBOL(flush_tlb_one);
+#ifdef CONFIG_HOTPLUG_CPU
+void arch_cpuhp_cleanup_dead_cpu(unsigned int cpu)
+{
+ if (mp_ops->cleanup_dead_cpu)
+ mp_ops->cleanup_dead_cpu(cpu);
+}
+#endif
+
#ifdef CONFIG_GENERIC_CLOCKEVENTS_BROADCAST
static void tick_broadcast_callee(void *info)
--- a/arch/mips/loongson64/smp.c
+++ b/arch/mips/loongson64/smp.c
@@ -788,6 +788,7 @@ void play_dead(void)
void (*play_dead_at_ckseg1)(int *);
idle_task_exit();
+ cpuhp_ap_report_dead();
prid_imp = read_c0_prid() & PRID_IMP_MASK;
prid_rev = read_c0_prid() & PRID_REV_MASK;
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 24/37] MIPS: SMP_CPS: Switch to hotplug core state synchronization
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Thomas Bogendoerfer,
linux-mips, David Woodhouse, Usama Arif, Juergen Gross,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, James E.J. Bottomley, Helge Deller, linux-parisc,
Paul Walmsley, Palmer Dabbelt, linux-riscv, Mark Rutland,
Sabin Rapan
Switch to the CPU hotplug core state tracking and synchronization
mechanim. This unfortunately requires to add dead reporting to the non CPS
platforms as CPS is the only user, but it allows an overall consolidation
of this functionality.
No functional change intended.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Thomas Bogendoerfer <tsbogend@alpha.franken.de>
Cc: linux-mips@vger.kernel.org
---
arch/mips/Kconfig | 1 +
arch/mips/cavium-octeon/smp.c | 1 +
arch/mips/include/asm/smp-ops.h | 1 +
arch/mips/kernel/smp-bmips.c | 1 +
arch/mips/kernel/smp-cps.c | 14 +++++---------
arch/mips/kernel/smp.c | 8 ++++++++
arch/mips/loongson64/smp.c | 1 +
7 files changed, 18 insertions(+), 9 deletions(-)
--- a/arch/mips/Kconfig
+++ b/arch/mips/Kconfig
@@ -2356,6 +2356,7 @@ config MIPS_CPS
select MIPS_CM
select MIPS_CPS_PM if HOTPLUG_CPU
select SMP
+ select HOTPLUG_CORE_SYNC_DEAD if HOTPLUG_CPU
select SYNC_R4K if (CEVT_R4K || CSRC_R4K)
select SYS_SUPPORTS_HOTPLUG_CPU
select SYS_SUPPORTS_SCHED_SMT if CPU_MIPSR6
--- a/arch/mips/cavium-octeon/smp.c
+++ b/arch/mips/cavium-octeon/smp.c
@@ -344,6 +344,7 @@ void play_dead(void)
int cpu = cpu_number_map(cvmx_get_core_num());
idle_task_exit();
+ cpuhp_ap_report_dead();
octeon_processor_boot = 0xff;
per_cpu(cpu_state, cpu) = CPU_DEAD;
--- a/arch/mips/include/asm/smp-ops.h
+++ b/arch/mips/include/asm/smp-ops.h
@@ -33,6 +33,7 @@ struct plat_smp_ops {
#ifdef CONFIG_HOTPLUG_CPU
int (*cpu_disable)(void);
void (*cpu_die)(unsigned int cpu);
+ void (*cleanup_dead_cpu)(unsigned cpu);
#endif
#ifdef CONFIG_KEXEC
void (*kexec_nonboot_cpu)(void);
--- a/arch/mips/kernel/smp-bmips.c
+++ b/arch/mips/kernel/smp-bmips.c
@@ -390,6 +390,7 @@ static void bmips_cpu_die(unsigned int c
void __ref play_dead(void)
{
idle_task_exit();
+ cpuhp_ap_report_dead();
/* flush data cache */
_dma_cache_wback_inv(0, ~0);
--- a/arch/mips/kernel/smp-cps.c
+++ b/arch/mips/kernel/smp-cps.c
@@ -497,8 +497,7 @@ void play_dead(void)
}
}
- /* This CPU has chosen its way out */
- (void)cpu_report_death();
+ cpuhp_ap_report_dead();
cps_shutdown_this_cpu(cpu_death);
@@ -521,7 +520,9 @@ static void wait_for_sibling_halt(void *
} while (!(halted & TCHALT_H));
}
-static void cps_cpu_die(unsigned int cpu)
+static void cps_cpu_die(unsigned int cpu) { }
+
+static void cps_cleanup_dead_cpu(unsigned cpu)
{
unsigned core = cpu_core(&cpu_data[cpu]);
unsigned int vpe_id = cpu_vpe_id(&cpu_data[cpu]);
@@ -529,12 +530,6 @@ static void cps_cpu_die(unsigned int cpu
unsigned stat;
int err;
- /* Wait for the cpu to choose its way out */
- if (!cpu_wait_death(cpu, 5)) {
- pr_err("CPU%u: didn't offline\n", cpu);
- return;
- }
-
/*
* Now wait for the CPU to actually offline. Without doing this that
* offlining may race with one or more of:
@@ -618,6 +613,7 @@ static const struct plat_smp_ops cps_smp
#ifdef CONFIG_HOTPLUG_CPU
.cpu_disable = cps_cpu_disable,
.cpu_die = cps_cpu_die,
+ .cleanup_dead_cpu = cps_cleanup_dead_cpu,
#endif
#ifdef CONFIG_KEXEC
.kexec_nonboot_cpu = cps_kexec_nonboot_cpu,
--- a/arch/mips/kernel/smp.c
+++ b/arch/mips/kernel/smp.c
@@ -690,6 +690,14 @@ void flush_tlb_one(unsigned long vaddr)
EXPORT_SYMBOL(flush_tlb_page);
EXPORT_SYMBOL(flush_tlb_one);
+#ifdef CONFIG_HOTPLUG_CPU
+void arch_cpuhp_cleanup_dead_cpu(unsigned int cpu)
+{
+ if (mp_ops->cleanup_dead_cpu)
+ mp_ops->cleanup_dead_cpu(cpu);
+}
+#endif
+
#ifdef CONFIG_GENERIC_CLOCKEVENTS_BROADCAST
static void tick_broadcast_callee(void *info)
--- a/arch/mips/loongson64/smp.c
+++ b/arch/mips/loongson64/smp.c
@@ -788,6 +788,7 @@ void play_dead(void)
void (*play_dead_at_ckseg1)(int *);
idle_task_exit();
+ cpuhp_ap_report_dead();
prid_imp = read_c0_prid() & PRID_IMP_MASK;
prid_rev = read_c0_prid() & PRID_REV_MASK;
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 25/37] parisc: Switch to hotplug core state synchronization
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-14 23:44 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, James E.J. Bottomley,
Helge Deller, linux-parisc, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
Paul Walmsley, Palmer Dabbelt, linux-riscv, Mark Rutland,
Sabin Rapan
Switch to the CPU hotplug core state tracking and synchronization
mechanim. No functional change intended.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: "James E.J. Bottomley" <James.Bottomley@HansenPartnership.com>
Cc: Helge Deller <deller@gmx.de>
Cc: linux-parisc@vger.kernel.org
---
arch/parisc/Kconfig | 1 +
arch/parisc/kernel/process.c | 4 ++--
arch/parisc/kernel/smp.c | 7 +++----
3 files changed, 6 insertions(+), 6 deletions(-)
--- a/arch/parisc/Kconfig
+++ b/arch/parisc/Kconfig
@@ -56,6 +56,7 @@ config PARISC
select HAVE_ARCH_SECCOMP_FILTER
select HAVE_ARCH_TRACEHOOK
select HAVE_REGS_AND_STACK_ACCESS_API
+ select HOTPLUG_CORE_SYNC_DEAD if HOTPLUG_CPU
select GENERIC_SCHED_CLOCK
select GENERIC_IRQ_MIGRATION if SMP
select HAVE_UNSTABLE_SCHED_CLOCK if SMP
--- a/arch/parisc/kernel/process.c
+++ b/arch/parisc/kernel/process.c
@@ -166,8 +166,8 @@ void arch_cpu_idle_dead(void)
local_irq_disable();
- /* Tell __cpu_die() that this CPU is now safe to dispose of. */
- (void)cpu_report_death();
+ /* Tell the core that this CPU is now safe to dispose of. */
+ cpuhp_ap_report_dead();
/* Ensure that the cache lines are written out. */
flush_cache_all_local();
--- a/arch/parisc/kernel/smp.c
+++ b/arch/parisc/kernel/smp.c
@@ -500,11 +500,10 @@ int __cpu_disable(void)
void __cpu_die(unsigned int cpu)
{
pdc_cpu_rendezvous_lock();
+}
- if (!cpu_wait_death(cpu, 5)) {
- pr_crit("CPU%u: cpu didn't die\n", cpu);
- return;
- }
+void arch_cpuhp_cleanup_dead_cpu(unsigned int cpu)
+{
pr_info("CPU%u: is shutting down\n", cpu);
/* set task's state to interruptible sleep */
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 25/37] parisc: Switch to hotplug core state synchronization
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, James E.J. Bottomley,
Helge Deller, linux-parisc, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
Paul Walmsley, Palmer Dabbelt, linux-riscv, Mark Rutland,
Sabin Rapan
Switch to the CPU hotplug core state tracking and synchronization
mechanim. No functional change intended.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: "James E.J. Bottomley" <James.Bottomley@HansenPartnership.com>
Cc: Helge Deller <deller@gmx.de>
Cc: linux-parisc@vger.kernel.org
---
arch/parisc/Kconfig | 1 +
arch/parisc/kernel/process.c | 4 ++--
arch/parisc/kernel/smp.c | 7 +++----
3 files changed, 6 insertions(+), 6 deletions(-)
--- a/arch/parisc/Kconfig
+++ b/arch/parisc/Kconfig
@@ -56,6 +56,7 @@ config PARISC
select HAVE_ARCH_SECCOMP_FILTER
select HAVE_ARCH_TRACEHOOK
select HAVE_REGS_AND_STACK_ACCESS_API
+ select HOTPLUG_CORE_SYNC_DEAD if HOTPLUG_CPU
select GENERIC_SCHED_CLOCK
select GENERIC_IRQ_MIGRATION if SMP
select HAVE_UNSTABLE_SCHED_CLOCK if SMP
--- a/arch/parisc/kernel/process.c
+++ b/arch/parisc/kernel/process.c
@@ -166,8 +166,8 @@ void arch_cpu_idle_dead(void)
local_irq_disable();
- /* Tell __cpu_die() that this CPU is now safe to dispose of. */
- (void)cpu_report_death();
+ /* Tell the core that this CPU is now safe to dispose of. */
+ cpuhp_ap_report_dead();
/* Ensure that the cache lines are written out. */
flush_cache_all_local();
--- a/arch/parisc/kernel/smp.c
+++ b/arch/parisc/kernel/smp.c
@@ -500,11 +500,10 @@ int __cpu_disable(void)
void __cpu_die(unsigned int cpu)
{
pdc_cpu_rendezvous_lock();
+}
- if (!cpu_wait_death(cpu, 5)) {
- pr_crit("CPU%u: cpu didn't die\n", cpu);
- return;
- }
+void arch_cpuhp_cleanup_dead_cpu(unsigned int cpu)
+{
pr_info("CPU%u: is shutting down\n", cpu);
/* set task's state to interruptible sleep */
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 25/37] parisc: Switch to hotplug core state synchronization
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, James E.J. Bottomley,
Helge Deller, linux-parisc, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
Paul Walmsley, Palmer Dabbelt, linux-riscv, Mark Rutland,
Sabin Rapan
Switch to the CPU hotplug core state tracking and synchronization
mechanim. No functional change intended.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: "James E.J. Bottomley" <James.Bottomley@HansenPartnership.com>
Cc: Helge Deller <deller@gmx.de>
Cc: linux-parisc@vger.kernel.org
---
arch/parisc/Kconfig | 1 +
arch/parisc/kernel/process.c | 4 ++--
arch/parisc/kernel/smp.c | 7 +++----
3 files changed, 6 insertions(+), 6 deletions(-)
--- a/arch/parisc/Kconfig
+++ b/arch/parisc/Kconfig
@@ -56,6 +56,7 @@ config PARISC
select HAVE_ARCH_SECCOMP_FILTER
select HAVE_ARCH_TRACEHOOK
select HAVE_REGS_AND_STACK_ACCESS_API
+ select HOTPLUG_CORE_SYNC_DEAD if HOTPLUG_CPU
select GENERIC_SCHED_CLOCK
select GENERIC_IRQ_MIGRATION if SMP
select HAVE_UNSTABLE_SCHED_CLOCK if SMP
--- a/arch/parisc/kernel/process.c
+++ b/arch/parisc/kernel/process.c
@@ -166,8 +166,8 @@ void arch_cpu_idle_dead(void)
local_irq_disable();
- /* Tell __cpu_die() that this CPU is now safe to dispose of. */
- (void)cpu_report_death();
+ /* Tell the core that this CPU is now safe to dispose of. */
+ cpuhp_ap_report_dead();
/* Ensure that the cache lines are written out. */
flush_cache_all_local();
--- a/arch/parisc/kernel/smp.c
+++ b/arch/parisc/kernel/smp.c
@@ -500,11 +500,10 @@ int __cpu_disable(void)
void __cpu_die(unsigned int cpu)
{
pdc_cpu_rendezvous_lock();
+}
- if (!cpu_wait_death(cpu, 5)) {
- pr_crit("CPU%u: cpu didn't die\n", cpu);
- return;
- }
+void arch_cpuhp_cleanup_dead_cpu(unsigned int cpu)
+{
pr_info("CPU%u: is shutting down\n", cpu);
/* set task's state to interruptible sleep */
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 26/37] riscv: Switch to hotplug core state synchronization
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-14 23:44 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Paul Walmsley,
Palmer Dabbelt, linux-riscv, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Mark Rutland,
Sabin Rapan
Switch to the CPU hotplug core state tracking and synchronization
mechanim. No functional change intended.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Paul Walmsley <paul.walmsley@sifive.com>
Cc: Palmer Dabbelt <palmer@dabbelt.com>
Cc: linux-riscv@lists.infradead.org
---
arch/riscv/Kconfig | 1 +
arch/riscv/include/asm/smp.h | 2 +-
arch/riscv/kernel/cpu-hotplug.c | 14 +++++++-------
3 files changed, 9 insertions(+), 8 deletions(-)
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -116,6 +116,7 @@ config RISCV
select HAVE_RSEQ
select HAVE_STACKPROTECTOR
select HAVE_SYSCALL_TRACEPOINTS
+ select HOTPLUG_CORE_SYNC_DEAD if HOTPLUG_CPU
select IRQ_DOMAIN
select IRQ_FORCED_THREADING
select MODULES_USE_ELF_RELA if MODULES
--- a/arch/riscv/include/asm/smp.h
+++ b/arch/riscv/include/asm/smp.h
@@ -64,7 +64,7 @@ asmlinkage void smp_callin(void);
#if defined CONFIG_HOTPLUG_CPU
int __cpu_disable(void);
-void __cpu_die(unsigned int cpu);
+static inline void __cpu_die(unsigned int cpu) { }
#endif /* CONFIG_HOTPLUG_CPU */
#else
--- a/arch/riscv/kernel/cpu-hotplug.c
+++ b/arch/riscv/kernel/cpu-hotplug.c
@@ -8,6 +8,7 @@
#include <linux/sched.h>
#include <linux/err.h>
#include <linux/irq.h>
+#include <linux/cpuhotplug.h>
#include <linux/cpu.h>
#include <linux/sched/hotplug.h>
#include <asm/irq.h>
@@ -48,17 +49,15 @@ int __cpu_disable(void)
return ret;
}
+#ifdef CONFIG_HOTPLUG_CPU
/*
- * Called on the thread which is asking for a CPU to be shutdown.
+ * Called on the thread which is asking for a CPU to be shutdown, if the
+ * CPU reported dead to the hotplug core.
*/
-void __cpu_die(unsigned int cpu)
+void arch_cpuhp_cleanup_dead_cpu(unsigned int cpu)
{
int ret = 0;
- if (!cpu_wait_death(cpu, 5)) {
- pr_err("CPU %u: didn't die\n", cpu);
- return;
- }
pr_notice("CPU%u: off\n", cpu);
/* Verify from the firmware if the cpu is really stopped*/
@@ -75,9 +74,10 @@ void arch_cpu_idle_dead(void)
{
idle_task_exit();
- (void)cpu_report_death();
+ cpuhp_ap_report_dead();
cpu_ops[smp_processor_id()]->cpu_stop();
/* It should never reach here */
BUG();
}
+#endif
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 26/37] riscv: Switch to hotplug core state synchronization
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Paul Walmsley,
Palmer Dabbelt, linux-riscv, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Mark Rutland,
Sabin Rapan
Switch to the CPU hotplug core state tracking and synchronization
mechanim. No functional change intended.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Paul Walmsley <paul.walmsley@sifive.com>
Cc: Palmer Dabbelt <palmer@dabbelt.com>
Cc: linux-riscv@lists.infradead.org
---
arch/riscv/Kconfig | 1 +
arch/riscv/include/asm/smp.h | 2 +-
arch/riscv/kernel/cpu-hotplug.c | 14 +++++++-------
3 files changed, 9 insertions(+), 8 deletions(-)
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -116,6 +116,7 @@ config RISCV
select HAVE_RSEQ
select HAVE_STACKPROTECTOR
select HAVE_SYSCALL_TRACEPOINTS
+ select HOTPLUG_CORE_SYNC_DEAD if HOTPLUG_CPU
select IRQ_DOMAIN
select IRQ_FORCED_THREADING
select MODULES_USE_ELF_RELA if MODULES
--- a/arch/riscv/include/asm/smp.h
+++ b/arch/riscv/include/asm/smp.h
@@ -64,7 +64,7 @@ asmlinkage void smp_callin(void);
#if defined CONFIG_HOTPLUG_CPU
int __cpu_disable(void);
-void __cpu_die(unsigned int cpu);
+static inline void __cpu_die(unsigned int cpu) { }
#endif /* CONFIG_HOTPLUG_CPU */
#else
--- a/arch/riscv/kernel/cpu-hotplug.c
+++ b/arch/riscv/kernel/cpu-hotplug.c
@@ -8,6 +8,7 @@
#include <linux/sched.h>
#include <linux/err.h>
#include <linux/irq.h>
+#include <linux/cpuhotplug.h>
#include <linux/cpu.h>
#include <linux/sched/hotplug.h>
#include <asm/irq.h>
@@ -48,17 +49,15 @@ int __cpu_disable(void)
return ret;
}
+#ifdef CONFIG_HOTPLUG_CPU
/*
- * Called on the thread which is asking for a CPU to be shutdown.
+ * Called on the thread which is asking for a CPU to be shutdown, if the
+ * CPU reported dead to the hotplug core.
*/
-void __cpu_die(unsigned int cpu)
+void arch_cpuhp_cleanup_dead_cpu(unsigned int cpu)
{
int ret = 0;
- if (!cpu_wait_death(cpu, 5)) {
- pr_err("CPU %u: didn't die\n", cpu);
- return;
- }
pr_notice("CPU%u: off\n", cpu);
/* Verify from the firmware if the cpu is really stopped*/
@@ -75,9 +74,10 @@ void arch_cpu_idle_dead(void)
{
idle_task_exit();
- (void)cpu_report_death();
+ cpuhp_ap_report_dead();
cpu_ops[smp_processor_id()]->cpu_stop();
/* It should never reach here */
BUG();
}
+#endif
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 26/37] riscv: Switch to hotplug core state synchronization
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Paul Walmsley,
Palmer Dabbelt, linux-riscv, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Mark Rutland,
Sabin Rapan
Switch to the CPU hotplug core state tracking and synchronization
mechanim. No functional change intended.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Paul Walmsley <paul.walmsley@sifive.com>
Cc: Palmer Dabbelt <palmer@dabbelt.com>
Cc: linux-riscv@lists.infradead.org
---
arch/riscv/Kconfig | 1 +
arch/riscv/include/asm/smp.h | 2 +-
arch/riscv/kernel/cpu-hotplug.c | 14 +++++++-------
3 files changed, 9 insertions(+), 8 deletions(-)
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -116,6 +116,7 @@ config RISCV
select HAVE_RSEQ
select HAVE_STACKPROTECTOR
select HAVE_SYSCALL_TRACEPOINTS
+ select HOTPLUG_CORE_SYNC_DEAD if HOTPLUG_CPU
select IRQ_DOMAIN
select IRQ_FORCED_THREADING
select MODULES_USE_ELF_RELA if MODULES
--- a/arch/riscv/include/asm/smp.h
+++ b/arch/riscv/include/asm/smp.h
@@ -64,7 +64,7 @@ asmlinkage void smp_callin(void);
#if defined CONFIG_HOTPLUG_CPU
int __cpu_disable(void);
-void __cpu_die(unsigned int cpu);
+static inline void __cpu_die(unsigned int cpu) { }
#endif /* CONFIG_HOTPLUG_CPU */
#else
--- a/arch/riscv/kernel/cpu-hotplug.c
+++ b/arch/riscv/kernel/cpu-hotplug.c
@@ -8,6 +8,7 @@
#include <linux/sched.h>
#include <linux/err.h>
#include <linux/irq.h>
+#include <linux/cpuhotplug.h>
#include <linux/cpu.h>
#include <linux/sched/hotplug.h>
#include <asm/irq.h>
@@ -48,17 +49,15 @@ int __cpu_disable(void)
return ret;
}
+#ifdef CONFIG_HOTPLUG_CPU
/*
- * Called on the thread which is asking for a CPU to be shutdown.
+ * Called on the thread which is asking for a CPU to be shutdown, if the
+ * CPU reported dead to the hotplug core.
*/
-void __cpu_die(unsigned int cpu)
+void arch_cpuhp_cleanup_dead_cpu(unsigned int cpu)
{
int ret = 0;
- if (!cpu_wait_death(cpu, 5)) {
- pr_err("CPU %u: didn't die\n", cpu);
- return;
- }
pr_notice("CPU%u: off\n", cpu);
/* Verify from the firmware if the cpu is really stopped*/
@@ -75,9 +74,10 @@ void arch_cpu_idle_dead(void)
{
idle_task_exit();
- (void)cpu_report_death();
+ cpuhp_ap_report_dead();
cpu_ops[smp_processor_id()]->cpu_stop();
/* It should never reach here */
BUG();
}
+#endif
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 27/37] cpu/hotplug: Remove unused state functions
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-14 23:44 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
All users converted to the hotplug core mechanism.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/cpu.h | 2 -
kernel/smpboot.c | 75 ----------------------------------------------------
2 files changed, 77 deletions(-)
--- a/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@ -193,8 +193,6 @@ static inline void play_idle(unsigned lo
}
#ifdef CONFIG_HOTPLUG_CPU
-bool cpu_wait_death(unsigned int cpu, int seconds);
-bool cpu_report_death(void);
void cpuhp_report_idle_dead(void);
#else
static inline void cpuhp_report_idle_dead(void) { }
--- a/kernel/smpboot.c
+++ b/kernel/smpboot.c
@@ -325,78 +325,3 @@ void smpboot_unregister_percpu_thread(st
cpus_read_unlock();
}
EXPORT_SYMBOL_GPL(smpboot_unregister_percpu_thread);
-
-#ifndef CONFIG_HOTPLUG_CORE_SYNC
-static DEFINE_PER_CPU(atomic_t, cpu_hotplug_state) = ATOMIC_INIT(CPU_POST_DEAD);
-
-#ifdef CONFIG_HOTPLUG_CPU
-/*
- * Wait for the specified CPU to exit the idle loop and die.
- */
-bool cpu_wait_death(unsigned int cpu, int seconds)
-{
- int jf_left = seconds * HZ;
- int oldstate;
- bool ret = true;
- int sleep_jf = 1;
-
- might_sleep();
-
- /* The outgoing CPU will normally get done quite quickly. */
- if (atomic_read(&per_cpu(cpu_hotplug_state, cpu)) == CPU_DEAD)
- goto update_state_early;
- udelay(5);
-
- /* But if the outgoing CPU dawdles, wait increasingly long times. */
- while (atomic_read(&per_cpu(cpu_hotplug_state, cpu)) != CPU_DEAD) {
- schedule_timeout_uninterruptible(sleep_jf);
- jf_left -= sleep_jf;
- if (jf_left <= 0)
- break;
- sleep_jf = DIV_ROUND_UP(sleep_jf * 11, 10);
- }
-update_state_early:
- oldstate = atomic_read(&per_cpu(cpu_hotplug_state, cpu));
-update_state:
- if (oldstate == CPU_DEAD) {
- /* Outgoing CPU died normally, update state. */
- smp_mb(); /* atomic_read() before update. */
- atomic_set(&per_cpu(cpu_hotplug_state, cpu), CPU_POST_DEAD);
- } else {
- /* Outgoing CPU still hasn't died, set state accordingly. */
- if (!atomic_try_cmpxchg(&per_cpu(cpu_hotplug_state, cpu),
- &oldstate, CPU_BROKEN))
- goto update_state;
- ret = false;
- }
- return ret;
-}
-
-/*
- * Called by the outgoing CPU to report its successful death. Return
- * false if this report follows the surviving CPU's timing out.
- *
- * A separate "CPU_DEAD_FROZEN" is used when the surviving CPU
- * timed out. This approach allows architectures to omit calls to
- * cpu_check_up_prepare() and cpu_set_state_online() without defeating
- * the next cpu_wait_death()'s polling loop.
- */
-bool cpu_report_death(void)
-{
- int oldstate;
- int newstate;
- int cpu = smp_processor_id();
-
- oldstate = atomic_read(&per_cpu(cpu_hotplug_state, cpu));
- do {
- if (oldstate != CPU_BROKEN)
- newstate = CPU_DEAD;
- else
- newstate = CPU_DEAD_FROZEN;
- } while (!atomic_try_cmpxchg(&per_cpu(cpu_hotplug_state, cpu),
- &oldstate, newstate));
- return newstate == CPU_DEAD;
-}
-
-#endif /* #ifdef CONFIG_HOTPLUG_CPU */
-#endif /* !CONFIG_HOTPLUG_CORE_SYNC */
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 27/37] cpu/hotplug: Remove unused state functions
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
All users converted to the hotplug core mechanism.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/cpu.h | 2 -
kernel/smpboot.c | 75 ----------------------------------------------------
2 files changed, 77 deletions(-)
--- a/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@ -193,8 +193,6 @@ static inline void play_idle(unsigned lo
}
#ifdef CONFIG_HOTPLUG_CPU
-bool cpu_wait_death(unsigned int cpu, int seconds);
-bool cpu_report_death(void);
void cpuhp_report_idle_dead(void);
#else
static inline void cpuhp_report_idle_dead(void) { }
--- a/kernel/smpboot.c
+++ b/kernel/smpboot.c
@@ -325,78 +325,3 @@ void smpboot_unregister_percpu_thread(st
cpus_read_unlock();
}
EXPORT_SYMBOL_GPL(smpboot_unregister_percpu_thread);
-
-#ifndef CONFIG_HOTPLUG_CORE_SYNC
-static DEFINE_PER_CPU(atomic_t, cpu_hotplug_state) = ATOMIC_INIT(CPU_POST_DEAD);
-
-#ifdef CONFIG_HOTPLUG_CPU
-/*
- * Wait for the specified CPU to exit the idle loop and die.
- */
-bool cpu_wait_death(unsigned int cpu, int seconds)
-{
- int jf_left = seconds * HZ;
- int oldstate;
- bool ret = true;
- int sleep_jf = 1;
-
- might_sleep();
-
- /* The outgoing CPU will normally get done quite quickly. */
- if (atomic_read(&per_cpu(cpu_hotplug_state, cpu)) == CPU_DEAD)
- goto update_state_early;
- udelay(5);
-
- /* But if the outgoing CPU dawdles, wait increasingly long times. */
- while (atomic_read(&per_cpu(cpu_hotplug_state, cpu)) != CPU_DEAD) {
- schedule_timeout_uninterruptible(sleep_jf);
- jf_left -= sleep_jf;
- if (jf_left <= 0)
- break;
- sleep_jf = DIV_ROUND_UP(sleep_jf * 11, 10);
- }
-update_state_early:
- oldstate = atomic_read(&per_cpu(cpu_hotplug_state, cpu));
-update_state:
- if (oldstate == CPU_DEAD) {
- /* Outgoing CPU died normally, update state. */
- smp_mb(); /* atomic_read() before update. */
- atomic_set(&per_cpu(cpu_hotplug_state, cpu), CPU_POST_DEAD);
- } else {
- /* Outgoing CPU still hasn't died, set state accordingly. */
- if (!atomic_try_cmpxchg(&per_cpu(cpu_hotplug_state, cpu),
- &oldstate, CPU_BROKEN))
- goto update_state;
- ret = false;
- }
- return ret;
-}
-
-/*
- * Called by the outgoing CPU to report its successful death. Return
- * false if this report follows the surviving CPU's timing out.
- *
- * A separate "CPU_DEAD_FROZEN" is used when the surviving CPU
- * timed out. This approach allows architectures to omit calls to
- * cpu_check_up_prepare() and cpu_set_state_online() without defeating
- * the next cpu_wait_death()'s polling loop.
- */
-bool cpu_report_death(void)
-{
- int oldstate;
- int newstate;
- int cpu = smp_processor_id();
-
- oldstate = atomic_read(&per_cpu(cpu_hotplug_state, cpu));
- do {
- if (oldstate != CPU_BROKEN)
- newstate = CPU_DEAD;
- else
- newstate = CPU_DEAD_FROZEN;
- } while (!atomic_try_cmpxchg(&per_cpu(cpu_hotplug_state, cpu),
- &oldstate, newstate));
- return newstate == CPU_DEAD;
-}
-
-#endif /* #ifdef CONFIG_HOTPLUG_CPU */
-#endif /* !CONFIG_HOTPLUG_CORE_SYNC */
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 27/37] cpu/hotplug: Remove unused state functions
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
All users converted to the hotplug core mechanism.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
include/linux/cpu.h | 2 -
kernel/smpboot.c | 75 ----------------------------------------------------
2 files changed, 77 deletions(-)
--- a/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@ -193,8 +193,6 @@ static inline void play_idle(unsigned lo
}
#ifdef CONFIG_HOTPLUG_CPU
-bool cpu_wait_death(unsigned int cpu, int seconds);
-bool cpu_report_death(void);
void cpuhp_report_idle_dead(void);
#else
static inline void cpuhp_report_idle_dead(void) { }
--- a/kernel/smpboot.c
+++ b/kernel/smpboot.c
@@ -325,78 +325,3 @@ void smpboot_unregister_percpu_thread(st
cpus_read_unlock();
}
EXPORT_SYMBOL_GPL(smpboot_unregister_percpu_thread);
-
-#ifndef CONFIG_HOTPLUG_CORE_SYNC
-static DEFINE_PER_CPU(atomic_t, cpu_hotplug_state) = ATOMIC_INIT(CPU_POST_DEAD);
-
-#ifdef CONFIG_HOTPLUG_CPU
-/*
- * Wait for the specified CPU to exit the idle loop and die.
- */
-bool cpu_wait_death(unsigned int cpu, int seconds)
-{
- int jf_left = seconds * HZ;
- int oldstate;
- bool ret = true;
- int sleep_jf = 1;
-
- might_sleep();
-
- /* The outgoing CPU will normally get done quite quickly. */
- if (atomic_read(&per_cpu(cpu_hotplug_state, cpu)) == CPU_DEAD)
- goto update_state_early;
- udelay(5);
-
- /* But if the outgoing CPU dawdles, wait increasingly long times. */
- while (atomic_read(&per_cpu(cpu_hotplug_state, cpu)) != CPU_DEAD) {
- schedule_timeout_uninterruptible(sleep_jf);
- jf_left -= sleep_jf;
- if (jf_left <= 0)
- break;
- sleep_jf = DIV_ROUND_UP(sleep_jf * 11, 10);
- }
-update_state_early:
- oldstate = atomic_read(&per_cpu(cpu_hotplug_state, cpu));
-update_state:
- if (oldstate == CPU_DEAD) {
- /* Outgoing CPU died normally, update state. */
- smp_mb(); /* atomic_read() before update. */
- atomic_set(&per_cpu(cpu_hotplug_state, cpu), CPU_POST_DEAD);
- } else {
- /* Outgoing CPU still hasn't died, set state accordingly. */
- if (!atomic_try_cmpxchg(&per_cpu(cpu_hotplug_state, cpu),
- &oldstate, CPU_BROKEN))
- goto update_state;
- ret = false;
- }
- return ret;
-}
-
-/*
- * Called by the outgoing CPU to report its successful death. Return
- * false if this report follows the surviving CPU's timing out.
- *
- * A separate "CPU_DEAD_FROZEN" is used when the surviving CPU
- * timed out. This approach allows architectures to omit calls to
- * cpu_check_up_prepare() and cpu_set_state_online() without defeating
- * the next cpu_wait_death()'s polling loop.
- */
-bool cpu_report_death(void)
-{
- int oldstate;
- int newstate;
- int cpu = smp_processor_id();
-
- oldstate = atomic_read(&per_cpu(cpu_hotplug_state, cpu));
- do {
- if (oldstate != CPU_BROKEN)
- newstate = CPU_DEAD;
- else
- newstate = CPU_DEAD_FROZEN;
- } while (!atomic_try_cmpxchg(&per_cpu(cpu_hotplug_state, cpu),
- &oldstate, newstate));
- return newstate == CPU_DEAD;
-}
-
-#endif /* #ifdef CONFIG_HOTPLUG_CPU */
-#endif /* !CONFIG_HOTPLUG_CORE_SYNC */
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 28/37] cpu/hotplug: Reset task stack state in _cpu_up()
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-14 23:44 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse,
Mark Rutland, Usama Arif, Juergen Gross, Boris Ostrovsky,
xen-devel, Russell King, Arnd Bergmann, linux-arm-kernel,
Catalin Marinas, Will Deacon, Guo Ren, linux-csky,
Thomas Bogendoerfer, linux-mips, James E.J. Bottomley,
Helge Deller, linux-parisc, Paul Walmsley, Palmer Dabbelt,
linux-riscv, Sabin Rapan
From: David Woodhouse <dwmw@amazon.co.uk>
Commit dce1ca0525bf ("sched/scs: Reset task stack state in bringup_cpu()")
ensured that the shadow call stack and KASAN poisoning were removed from
a CPU's stack each time that CPU is brought up, not just once.
This is not incorrect. However, with parallel bringup the idle thread setup
will happen at a different step. As a consequence the cleanup in
bringup_cpu() would be too late.
Move the SCS/KASAN cleanup to the generic _cpu_up() function instead,
which already ensures that the new CPU's stack is available, purely to
allow for early failure. This occurs when the CPU to be brought up is
in the CPUHP_OFFLINE state, which should correctly do the cleanup any
time the CPU has been taken down to the point where such is needed.
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Mark Rutland <mark.rutland@arm.com>
---
kernel/cpu.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -769,12 +769,6 @@ static int bringup_cpu(unsigned int cpu)
return -EAGAIN;
/*
- * Reset stale stack state from the last time this CPU was online.
- */
- scs_task_reset(idle);
- kasan_unpoison_task_stack(idle);
-
- /*
* Some architectures have to walk the irq descriptors to
* setup the vector space for the cpu which comes online.
* Prevent irq alloc/free across the bringup.
@@ -1581,6 +1575,12 @@ static int _cpu_up(unsigned int cpu, int
ret = PTR_ERR(idle);
goto out;
}
+
+ /*
+ * Reset stale stack state from the last time this CPU was online.
+ */
+ scs_task_reset(idle);
+ kasan_unpoison_task_stack(idle);
}
cpuhp_tasks_frozen = tasks_frozen;
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 28/37] cpu/hotplug: Reset task stack state in _cpu_up()
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse,
Mark Rutland, Usama Arif, Juergen Gross, Boris Ostrovsky,
xen-devel, Russell King, Arnd Bergmann, linux-arm-kernel,
Catalin Marinas, Will Deacon, Guo Ren, linux-csky,
Thomas Bogendoerfer, linux-mips, James E.J. Bottomley,
Helge Deller, linux-parisc, Paul Walmsley, Palmer Dabbelt,
linux-riscv, Sabin Rapan
From: David Woodhouse <dwmw@amazon.co.uk>
Commit dce1ca0525bf ("sched/scs: Reset task stack state in bringup_cpu()")
ensured that the shadow call stack and KASAN poisoning were removed from
a CPU's stack each time that CPU is brought up, not just once.
This is not incorrect. However, with parallel bringup the idle thread setup
will happen at a different step. As a consequence the cleanup in
bringup_cpu() would be too late.
Move the SCS/KASAN cleanup to the generic _cpu_up() function instead,
which already ensures that the new CPU's stack is available, purely to
allow for early failure. This occurs when the CPU to be brought up is
in the CPUHP_OFFLINE state, which should correctly do the cleanup any
time the CPU has been taken down to the point where such is needed.
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Mark Rutland <mark.rutland@arm.com>
---
kernel/cpu.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -769,12 +769,6 @@ static int bringup_cpu(unsigned int cpu)
return -EAGAIN;
/*
- * Reset stale stack state from the last time this CPU was online.
- */
- scs_task_reset(idle);
- kasan_unpoison_task_stack(idle);
-
- /*
* Some architectures have to walk the irq descriptors to
* setup the vector space for the cpu which comes online.
* Prevent irq alloc/free across the bringup.
@@ -1581,6 +1575,12 @@ static int _cpu_up(unsigned int cpu, int
ret = PTR_ERR(idle);
goto out;
}
+
+ /*
+ * Reset stale stack state from the last time this CPU was online.
+ */
+ scs_task_reset(idle);
+ kasan_unpoison_task_stack(idle);
}
cpuhp_tasks_frozen = tasks_frozen;
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 28/37] cpu/hotplug: Reset task stack state in _cpu_up()
@ 2023-04-14 23:44 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:44 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse,
Mark Rutland, Usama Arif, Juergen Gross, Boris Ostrovsky,
xen-devel, Russell King, Arnd Bergmann, linux-arm-kernel,
Catalin Marinas, Will Deacon, Guo Ren, linux-csky,
Thomas Bogendoerfer, linux-mips, James E.J. Bottomley,
Helge Deller, linux-parisc, Paul Walmsley, Palmer Dabbelt,
linux-riscv, Sabin Rapan
From: David Woodhouse <dwmw@amazon.co.uk>
Commit dce1ca0525bf ("sched/scs: Reset task stack state in bringup_cpu()")
ensured that the shadow call stack and KASAN poisoning were removed from
a CPU's stack each time that CPU is brought up, not just once.
This is not incorrect. However, with parallel bringup the idle thread setup
will happen at a different step. As a consequence the cleanup in
bringup_cpu() would be too late.
Move the SCS/KASAN cleanup to the generic _cpu_up() function instead,
which already ensures that the new CPU's stack is available, purely to
allow for early failure. This occurs when the CPU to be brought up is
in the CPUHP_OFFLINE state, which should correctly do the cleanup any
time the CPU has been taken down to the point where such is needed.
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Tested-by: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Mark Rutland <mark.rutland@arm.com>
---
kernel/cpu.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -769,12 +769,6 @@ static int bringup_cpu(unsigned int cpu)
return -EAGAIN;
/*
- * Reset stale stack state from the last time this CPU was online.
- */
- scs_task_reset(idle);
- kasan_unpoison_task_stack(idle);
-
- /*
* Some architectures have to walk the irq descriptors to
* setup the vector space for the cpu which comes online.
* Prevent irq alloc/free across the bringup.
@@ -1581,6 +1575,12 @@ static int _cpu_up(unsigned int cpu, int
ret = PTR_ERR(idle);
goto out;
}
+
+ /*
+ * Reset stale stack state from the last time this CPU was online.
+ */
+ scs_task_reset(idle);
+ kasan_unpoison_task_stack(idle);
}
cpuhp_tasks_frozen = tasks_frozen;
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 29/37] cpu/hotplug: Provide a split up CPUHP_BRINGUP mechanism
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-14 23:45 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:45 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
The bring up logic of a to be onlined CPU consists of several parts, which
are considered to be a single hotplug state:
1) Control CPU issues the wake-up
2) To be onlined CPU starts up, does the minimal initialization,
reports to be alive and waits for release into the complete bring-up.
3) Control CPU waits for the alive report and releases the upcoming CPU
for the complete bring-up.
Allow to split this into two states:
1) Control CPU issues the wake-up
After that the to be onlined CPU starts up, does the minimal
initialization, reports to be alive and waits for release into the
full bring-up. As this can run after the control CPU dropped the
hotplug locks the code which is executed on the AP before it reports
alive has to be carefully audited to not violate any of the hotplug
constraints, especially not modifying any of the various cpumasks.
This is really only meant to avoid waiting for the AP to react on the
wake-up. Of course an architecture can move strict CPU related setup
functionality, e.g. microcode loading, with care before the
synchronization point to save further pointless waiting time.
2) Control CPU waits for the alive report and releases the upcoming CPU
for the complete bring-up.
This allows that the two states can be split up to run all to be onlined
CPUs up to state #1 on the control CPU and then at a later point run state
#2. This spares some of the latencies of the full serialized per CPU
bringup by avoiding the per CPU wakeup/wait serialization. The assumption
is that the first AP already waits when the last AP has been woken up. This
obvioulsy depends on the hardware latencies and depending on the timings
this might still not completely eliminate all wait scenarios.
This split is just a preparatory step for enabling the parallel bringup
later. The boot time bringup is still fully serialized. It has a separate
config switch so that architectures which want to support parallel bringup
can test the split of the CPUHP_BRINGUG step separately.
To enable this the architecture must support the CPU hotplug core sync
mechanism and has to be audited that there are no implicit hotplug state
dependencies which require a fully serialized bringup.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/Kconfig | 4 ++
include/linux/cpuhotplug.h | 4 ++
kernel/cpu.c | 70 +++++++++++++++++++++++++++++++++++++++++++--
3 files changed, 76 insertions(+), 2 deletions(-)
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -49,6 +49,10 @@ config HOTPLUG_CORE_SYNC_FULL
select HOTPLUG_CORE_SYNC_DEAD if HOTPLUG_CPU
select HOTPLUG_CORE_SYNC
+config HOTPLUG_SPLIT_STARTUP
+ bool
+ select HOTPLUG_CORE_SYNC_FULL
+
config GENERIC_ENTRY
bool
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -133,6 +133,7 @@ enum cpuhp_state {
CPUHP_MIPS_SOC_PREPARE,
CPUHP_BP_PREPARE_DYN,
CPUHP_BP_PREPARE_DYN_END = CPUHP_BP_PREPARE_DYN + 20,
+ CPUHP_BP_KICK_AP,
CPUHP_BRINGUP_CPU,
/*
@@ -519,9 +520,12 @@ void cpuhp_online_idle(enum cpuhp_state
static inline void cpuhp_online_idle(enum cpuhp_state state) { }
#endif
+struct task_struct;
+
void cpuhp_ap_sync_alive(void);
void arch_cpuhp_sync_state_poll(void);
void arch_cpuhp_cleanup_kick_cpu(unsigned int cpu);
+int arch_cpuhp_kick_ap_alive(unsigned int cpu, struct task_struct *tidle);
#ifdef CONFIG_HOTPLUG_CORE_SYNC_DEAD
void cpuhp_ap_report_dead(void);
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -759,6 +759,47 @@ static int bringup_wait_for_ap_online(un
return 0;
}
+#ifdef CONFIG_HOTPLUG_SPLIT_STARTUP
+static int cpuhp_kick_ap_alive(unsigned int cpu)
+{
+ if (!cpuhp_can_boot_ap(cpu))
+ return -EAGAIN;
+
+ return arch_cpuhp_kick_ap_alive(cpu, idle_thread_get(cpu));
+}
+
+static int cpuhp_bringup_ap(unsigned int cpu)
+{
+ struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+ int ret;
+
+ /*
+ * Some architectures have to walk the irq descriptors to
+ * setup the vector space for the cpu which comes online.
+ * Prevent irq alloc/free across the bringup.
+ */
+ irq_lock_sparse();
+
+ ret = cpuhp_bp_sync_alive(cpu);
+ if (ret)
+ goto out_unlock;
+
+ ret = bringup_wait_for_ap_online(cpu);
+ if (ret)
+ goto out_unlock;
+
+ irq_unlock_sparse();
+
+ if (st->target <= CPUHP_AP_ONLINE_IDLE)
+ return 0;
+
+ return cpuhp_kick_ap(cpu, st, st->target);
+
+out_unlock:
+ irq_unlock_sparse();
+ return ret;
+}
+#else
static int bringup_cpu(unsigned int cpu)
{
struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
@@ -775,7 +816,6 @@ static int bringup_cpu(unsigned int cpu)
*/
irq_lock_sparse();
- /* Arch-specific enabling code. */
ret = __cpu_up(cpu, idle);
if (ret)
goto out_unlock;
@@ -799,6 +839,7 @@ static int bringup_cpu(unsigned int cpu)
irq_unlock_sparse();
return ret;
}
+#endif
static int finish_cpu(unsigned int cpu)
{
@@ -1938,13 +1979,38 @@ static struct cpuhp_step cpuhp_hp_states
.startup.single = timers_prepare_cpu,
.teardown.single = timers_dead_cpu,
},
- /* Kicks the plugged cpu into life */
+
+#ifdef CONFIG_HOTPLUG_SPLIT_STARTUP
+ /*
+ * Kicks the AP alive. AP will wait in cpuhp_ap_sync_alive() until
+ * the next step will release it.
+ */
+ [CPUHP_BP_KICK_AP] = {
+ .name = "cpu:kick_ap",
+ .startup.single = cpuhp_kick_ap_alive,
+ },
+
+ /*
+ * Waits for the AP to reach cpuhp_ap_sync_alive() and then
+ * releases it for the complete bringup.
+ */
+ [CPUHP_BRINGUP_CPU] = {
+ .name = "cpu:bringup",
+ .startup.single = cpuhp_bringup_ap,
+ .teardown.single = finish_cpu,
+ .cant_stop = true,
+ },
+#else
+ /*
+ * All-in-one CPU bringup state which includes the kick alive.
+ */
[CPUHP_BRINGUP_CPU] = {
.name = "cpu:bringup",
.startup.single = bringup_cpu,
.teardown.single = finish_cpu,
.cant_stop = true,
},
+#endif
/* Final state before CPU kills itself */
[CPUHP_AP_IDLE_DEAD] = {
.name = "idle:dead",
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 29/37] cpu/hotplug: Provide a split up CPUHP_BRINGUP mechanism
@ 2023-04-14 23:45 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:45 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
The bring up logic of a to be onlined CPU consists of several parts, which
are considered to be a single hotplug state:
1) Control CPU issues the wake-up
2) To be onlined CPU starts up, does the minimal initialization,
reports to be alive and waits for release into the complete bring-up.
3) Control CPU waits for the alive report and releases the upcoming CPU
for the complete bring-up.
Allow to split this into two states:
1) Control CPU issues the wake-up
After that the to be onlined CPU starts up, does the minimal
initialization, reports to be alive and waits for release into the
full bring-up. As this can run after the control CPU dropped the
hotplug locks the code which is executed on the AP before it reports
alive has to be carefully audited to not violate any of the hotplug
constraints, especially not modifying any of the various cpumasks.
This is really only meant to avoid waiting for the AP to react on the
wake-up. Of course an architecture can move strict CPU related setup
functionality, e.g. microcode loading, with care before the
synchronization point to save further pointless waiting time.
2) Control CPU waits for the alive report and releases the upcoming CPU
for the complete bring-up.
This allows that the two states can be split up to run all to be onlined
CPUs up to state #1 on the control CPU and then at a later point run state
#2. This spares some of the latencies of the full serialized per CPU
bringup by avoiding the per CPU wakeup/wait serialization. The assumption
is that the first AP already waits when the last AP has been woken up. This
obvioulsy depends on the hardware latencies and depending on the timings
this might still not completely eliminate all wait scenarios.
This split is just a preparatory step for enabling the parallel bringup
later. The boot time bringup is still fully serialized. It has a separate
config switch so that architectures which want to support parallel bringup
can test the split of the CPUHP_BRINGUG step separately.
To enable this the architecture must support the CPU hotplug core sync
mechanism and has to be audited that there are no implicit hotplug state
dependencies which require a fully serialized bringup.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/Kconfig | 4 ++
include/linux/cpuhotplug.h | 4 ++
kernel/cpu.c | 70 +++++++++++++++++++++++++++++++++++++++++++--
3 files changed, 76 insertions(+), 2 deletions(-)
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -49,6 +49,10 @@ config HOTPLUG_CORE_SYNC_FULL
select HOTPLUG_CORE_SYNC_DEAD if HOTPLUG_CPU
select HOTPLUG_CORE_SYNC
+config HOTPLUG_SPLIT_STARTUP
+ bool
+ select HOTPLUG_CORE_SYNC_FULL
+
config GENERIC_ENTRY
bool
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -133,6 +133,7 @@ enum cpuhp_state {
CPUHP_MIPS_SOC_PREPARE,
CPUHP_BP_PREPARE_DYN,
CPUHP_BP_PREPARE_DYN_END = CPUHP_BP_PREPARE_DYN + 20,
+ CPUHP_BP_KICK_AP,
CPUHP_BRINGUP_CPU,
/*
@@ -519,9 +520,12 @@ void cpuhp_online_idle(enum cpuhp_state
static inline void cpuhp_online_idle(enum cpuhp_state state) { }
#endif
+struct task_struct;
+
void cpuhp_ap_sync_alive(void);
void arch_cpuhp_sync_state_poll(void);
void arch_cpuhp_cleanup_kick_cpu(unsigned int cpu);
+int arch_cpuhp_kick_ap_alive(unsigned int cpu, struct task_struct *tidle);
#ifdef CONFIG_HOTPLUG_CORE_SYNC_DEAD
void cpuhp_ap_report_dead(void);
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -759,6 +759,47 @@ static int bringup_wait_for_ap_online(un
return 0;
}
+#ifdef CONFIG_HOTPLUG_SPLIT_STARTUP
+static int cpuhp_kick_ap_alive(unsigned int cpu)
+{
+ if (!cpuhp_can_boot_ap(cpu))
+ return -EAGAIN;
+
+ return arch_cpuhp_kick_ap_alive(cpu, idle_thread_get(cpu));
+}
+
+static int cpuhp_bringup_ap(unsigned int cpu)
+{
+ struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+ int ret;
+
+ /*
+ * Some architectures have to walk the irq descriptors to
+ * setup the vector space for the cpu which comes online.
+ * Prevent irq alloc/free across the bringup.
+ */
+ irq_lock_sparse();
+
+ ret = cpuhp_bp_sync_alive(cpu);
+ if (ret)
+ goto out_unlock;
+
+ ret = bringup_wait_for_ap_online(cpu);
+ if (ret)
+ goto out_unlock;
+
+ irq_unlock_sparse();
+
+ if (st->target <= CPUHP_AP_ONLINE_IDLE)
+ return 0;
+
+ return cpuhp_kick_ap(cpu, st, st->target);
+
+out_unlock:
+ irq_unlock_sparse();
+ return ret;
+}
+#else
static int bringup_cpu(unsigned int cpu)
{
struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
@@ -775,7 +816,6 @@ static int bringup_cpu(unsigned int cpu)
*/
irq_lock_sparse();
- /* Arch-specific enabling code. */
ret = __cpu_up(cpu, idle);
if (ret)
goto out_unlock;
@@ -799,6 +839,7 @@ static int bringup_cpu(unsigned int cpu)
irq_unlock_sparse();
return ret;
}
+#endif
static int finish_cpu(unsigned int cpu)
{
@@ -1938,13 +1979,38 @@ static struct cpuhp_step cpuhp_hp_states
.startup.single = timers_prepare_cpu,
.teardown.single = timers_dead_cpu,
},
- /* Kicks the plugged cpu into life */
+
+#ifdef CONFIG_HOTPLUG_SPLIT_STARTUP
+ /*
+ * Kicks the AP alive. AP will wait in cpuhp_ap_sync_alive() until
+ * the next step will release it.
+ */
+ [CPUHP_BP_KICK_AP] = {
+ .name = "cpu:kick_ap",
+ .startup.single = cpuhp_kick_ap_alive,
+ },
+
+ /*
+ * Waits for the AP to reach cpuhp_ap_sync_alive() and then
+ * releases it for the complete bringup.
+ */
+ [CPUHP_BRINGUP_CPU] = {
+ .name = "cpu:bringup",
+ .startup.single = cpuhp_bringup_ap,
+ .teardown.single = finish_cpu,
+ .cant_stop = true,
+ },
+#else
+ /*
+ * All-in-one CPU bringup state which includes the kick alive.
+ */
[CPUHP_BRINGUP_CPU] = {
.name = "cpu:bringup",
.startup.single = bringup_cpu,
.teardown.single = finish_cpu,
.cant_stop = true,
},
+#endif
/* Final state before CPU kills itself */
[CPUHP_AP_IDLE_DEAD] = {
.name = "idle:dead",
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 29/37] cpu/hotplug: Provide a split up CPUHP_BRINGUP mechanism
@ 2023-04-14 23:45 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:45 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
The bring up logic of a to be onlined CPU consists of several parts, which
are considered to be a single hotplug state:
1) Control CPU issues the wake-up
2) To be onlined CPU starts up, does the minimal initialization,
reports to be alive and waits for release into the complete bring-up.
3) Control CPU waits for the alive report and releases the upcoming CPU
for the complete bring-up.
Allow to split this into two states:
1) Control CPU issues the wake-up
After that the to be onlined CPU starts up, does the minimal
initialization, reports to be alive and waits for release into the
full bring-up. As this can run after the control CPU dropped the
hotplug locks the code which is executed on the AP before it reports
alive has to be carefully audited to not violate any of the hotplug
constraints, especially not modifying any of the various cpumasks.
This is really only meant to avoid waiting for the AP to react on the
wake-up. Of course an architecture can move strict CPU related setup
functionality, e.g. microcode loading, with care before the
synchronization point to save further pointless waiting time.
2) Control CPU waits for the alive report and releases the upcoming CPU
for the complete bring-up.
This allows that the two states can be split up to run all to be onlined
CPUs up to state #1 on the control CPU and then at a later point run state
#2. This spares some of the latencies of the full serialized per CPU
bringup by avoiding the per CPU wakeup/wait serialization. The assumption
is that the first AP already waits when the last AP has been woken up. This
obvioulsy depends on the hardware latencies and depending on the timings
this might still not completely eliminate all wait scenarios.
This split is just a preparatory step for enabling the parallel bringup
later. The boot time bringup is still fully serialized. It has a separate
config switch so that architectures which want to support parallel bringup
can test the split of the CPUHP_BRINGUG step separately.
To enable this the architecture must support the CPU hotplug core sync
mechanism and has to be audited that there are no implicit hotplug state
dependencies which require a fully serialized bringup.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/Kconfig | 4 ++
include/linux/cpuhotplug.h | 4 ++
kernel/cpu.c | 70 +++++++++++++++++++++++++++++++++++++++++++--
3 files changed, 76 insertions(+), 2 deletions(-)
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -49,6 +49,10 @@ config HOTPLUG_CORE_SYNC_FULL
select HOTPLUG_CORE_SYNC_DEAD if HOTPLUG_CPU
select HOTPLUG_CORE_SYNC
+config HOTPLUG_SPLIT_STARTUP
+ bool
+ select HOTPLUG_CORE_SYNC_FULL
+
config GENERIC_ENTRY
bool
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -133,6 +133,7 @@ enum cpuhp_state {
CPUHP_MIPS_SOC_PREPARE,
CPUHP_BP_PREPARE_DYN,
CPUHP_BP_PREPARE_DYN_END = CPUHP_BP_PREPARE_DYN + 20,
+ CPUHP_BP_KICK_AP,
CPUHP_BRINGUP_CPU,
/*
@@ -519,9 +520,12 @@ void cpuhp_online_idle(enum cpuhp_state
static inline void cpuhp_online_idle(enum cpuhp_state state) { }
#endif
+struct task_struct;
+
void cpuhp_ap_sync_alive(void);
void arch_cpuhp_sync_state_poll(void);
void arch_cpuhp_cleanup_kick_cpu(unsigned int cpu);
+int arch_cpuhp_kick_ap_alive(unsigned int cpu, struct task_struct *tidle);
#ifdef CONFIG_HOTPLUG_CORE_SYNC_DEAD
void cpuhp_ap_report_dead(void);
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -759,6 +759,47 @@ static int bringup_wait_for_ap_online(un
return 0;
}
+#ifdef CONFIG_HOTPLUG_SPLIT_STARTUP
+static int cpuhp_kick_ap_alive(unsigned int cpu)
+{
+ if (!cpuhp_can_boot_ap(cpu))
+ return -EAGAIN;
+
+ return arch_cpuhp_kick_ap_alive(cpu, idle_thread_get(cpu));
+}
+
+static int cpuhp_bringup_ap(unsigned int cpu)
+{
+ struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+ int ret;
+
+ /*
+ * Some architectures have to walk the irq descriptors to
+ * setup the vector space for the cpu which comes online.
+ * Prevent irq alloc/free across the bringup.
+ */
+ irq_lock_sparse();
+
+ ret = cpuhp_bp_sync_alive(cpu);
+ if (ret)
+ goto out_unlock;
+
+ ret = bringup_wait_for_ap_online(cpu);
+ if (ret)
+ goto out_unlock;
+
+ irq_unlock_sparse();
+
+ if (st->target <= CPUHP_AP_ONLINE_IDLE)
+ return 0;
+
+ return cpuhp_kick_ap(cpu, st, st->target);
+
+out_unlock:
+ irq_unlock_sparse();
+ return ret;
+}
+#else
static int bringup_cpu(unsigned int cpu)
{
struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
@@ -775,7 +816,6 @@ static int bringup_cpu(unsigned int cpu)
*/
irq_lock_sparse();
- /* Arch-specific enabling code. */
ret = __cpu_up(cpu, idle);
if (ret)
goto out_unlock;
@@ -799,6 +839,7 @@ static int bringup_cpu(unsigned int cpu)
irq_unlock_sparse();
return ret;
}
+#endif
static int finish_cpu(unsigned int cpu)
{
@@ -1938,13 +1979,38 @@ static struct cpuhp_step cpuhp_hp_states
.startup.single = timers_prepare_cpu,
.teardown.single = timers_dead_cpu,
},
- /* Kicks the plugged cpu into life */
+
+#ifdef CONFIG_HOTPLUG_SPLIT_STARTUP
+ /*
+ * Kicks the AP alive. AP will wait in cpuhp_ap_sync_alive() until
+ * the next step will release it.
+ */
+ [CPUHP_BP_KICK_AP] = {
+ .name = "cpu:kick_ap",
+ .startup.single = cpuhp_kick_ap_alive,
+ },
+
+ /*
+ * Waits for the AP to reach cpuhp_ap_sync_alive() and then
+ * releases it for the complete bringup.
+ */
+ [CPUHP_BRINGUP_CPU] = {
+ .name = "cpu:bringup",
+ .startup.single = cpuhp_bringup_ap,
+ .teardown.single = finish_cpu,
+ .cant_stop = true,
+ },
+#else
+ /*
+ * All-in-one CPU bringup state which includes the kick alive.
+ */
[CPUHP_BRINGUP_CPU] = {
.name = "cpu:bringup",
.startup.single = bringup_cpu,
.teardown.single = finish_cpu,
.cant_stop = true,
},
+#endif
/* Final state before CPU kills itself */
[CPUHP_AP_IDLE_DEAD] = {
.name = "idle:dead",
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 30/37] x86/smpboot: Enable split CPU startup
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-14 23:45 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:45 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
The x86 CPU bringup state currently does AP wake-up, wait for AP to
respond and then release it for full bringup.
It is safe to be split into a wake-up and and a separate wait+release
state.
Provide the required functions and enable the split CPU bringup, which
prepares for parallel bringup, where the bringup of the non-boot CPUs takes
two iterations: One to prepare and wake all APs and the second to wait and
release them. Depending on timing this can eliminate the wait time
completely.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/Kconfig | 2 +-
arch/x86/include/asm/smp.h | 9 ++-------
arch/x86/kernel/smp.c | 2 +-
arch/x86/kernel/smpboot.c | 8 ++++----
arch/x86/xen/smp_pv.c | 4 ++--
5 files changed, 10 insertions(+), 15 deletions(-)
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -272,8 +272,8 @@ config X86
select HAVE_UNSTABLE_SCHED_CLOCK
select HAVE_USER_RETURN_NOTIFIER
select HAVE_GENERIC_VDSO
- select HOTPLUG_CORE_SYNC_FULL if SMP
select HOTPLUG_SMT if SMP
+ select HOTPLUG_SPLIT_STARTUP if SMP
select IRQ_FORCED_THREADING
select NEED_PER_CPU_EMBED_FIRST_CHUNK
select NEED_PER_CPU_PAGE_FIRST_CHUNK
--- a/arch/x86/include/asm/smp.h
+++ b/arch/x86/include/asm/smp.h
@@ -40,7 +40,7 @@ struct smp_ops {
void (*cleanup_dead_cpu)(unsigned cpu);
void (*poll_sync_state)(void);
- int (*cpu_up)(unsigned cpu, struct task_struct *tidle);
+ int (*kick_ap_alive)(unsigned cpu, struct task_struct *tidle);
int (*cpu_disable)(void);
void (*cpu_die)(unsigned int cpu);
void (*play_dead)(void);
@@ -80,11 +80,6 @@ static inline void smp_cpus_done(unsigne
smp_ops.smp_cpus_done(max_cpus);
}
-static inline int __cpu_up(unsigned int cpu, struct task_struct *tidle)
-{
- return smp_ops.cpu_up(cpu, tidle);
-}
-
static inline int __cpu_disable(void)
{
return smp_ops.cpu_disable();
@@ -123,7 +118,7 @@ void native_smp_prepare_cpus(unsigned in
void calculate_max_logical_packages(void);
void native_smp_cpus_done(unsigned int max_cpus);
int common_cpu_up(unsigned int cpunum, struct task_struct *tidle);
-int native_cpu_up(unsigned int cpunum, struct task_struct *tidle);
+int native_kick_ap(unsigned int cpu, struct task_struct *tidle);
int native_cpu_disable(void);
void hlt_play_dead(void);
void native_play_dead(void);
--- a/arch/x86/kernel/smp.c
+++ b/arch/x86/kernel/smp.c
@@ -268,7 +268,7 @@ struct smp_ops smp_ops = {
#endif
.smp_send_reschedule = native_smp_send_reschedule,
- .cpu_up = native_cpu_up,
+ .kick_ap_alive = native_kick_ap,
.cpu_disable = native_cpu_disable,
.play_dead = native_play_dead,
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -1070,7 +1070,7 @@ static int do_boot_cpu(int apicid, int c
return ret;
}
-static int native_kick_ap(unsigned int cpu, struct task_struct *tidle)
+int native_kick_ap(unsigned int cpu, struct task_struct *tidle)
{
int apicid = apic->cpu_present_to_apicid(cpu);
int err;
@@ -1106,15 +1106,15 @@ static int native_kick_ap(unsigned int c
return err;
}
-int native_cpu_up(unsigned int cpu, struct task_struct *tidle)
+int arch_cpuhp_kick_ap_alive(unsigned int cpu, struct task_struct *tidle)
{
- return native_kick_ap(cpu, tidle);
+ return smp_ops.kick_ap_alive(cpu, tidle);
}
void arch_cpuhp_cleanup_kick_cpu(unsigned int cpu)
{
/* Cleanup possible dangling ends... */
- if (smp_ops.cpu_up == native_cpu_up && x86_platform.legacy.warm_reset)
+ if (smp_ops.kick_ap_alive == native_kick_ap && x86_platform.legacy.warm_reset)
smpboot_restore_warm_reset_vector();
}
--- a/arch/x86/xen/smp_pv.c
+++ b/arch/x86/xen/smp_pv.c
@@ -314,7 +314,7 @@ cpu_initialize_context(unsigned int cpu,
return 0;
}
-static int xen_pv_cpu_up(unsigned int cpu, struct task_struct *idle)
+static int xen_pv_kick_ap(unsigned int cpu, struct task_struct *idle)
{
int rc;
@@ -438,7 +438,7 @@ static const struct smp_ops xen_smp_ops
.smp_prepare_cpus = xen_pv_smp_prepare_cpus,
.smp_cpus_done = xen_smp_cpus_done,
- .cpu_up = xen_pv_cpu_up,
+ .kick_ap_alive = xen_pv_kick_ap,
.cpu_die = xen_pv_cpu_die,
.cleanup_dead_cpu = xen_pv_cleanup_dead_cpu,
.poll_sync_state = xen_pv_poll_sync_state,
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 30/37] x86/smpboot: Enable split CPU startup
@ 2023-04-14 23:45 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:45 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
The x86 CPU bringup state currently does AP wake-up, wait for AP to
respond and then release it for full bringup.
It is safe to be split into a wake-up and and a separate wait+release
state.
Provide the required functions and enable the split CPU bringup, which
prepares for parallel bringup, where the bringup of the non-boot CPUs takes
two iterations: One to prepare and wake all APs and the second to wait and
release them. Depending on timing this can eliminate the wait time
completely.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/Kconfig | 2 +-
arch/x86/include/asm/smp.h | 9 ++-------
arch/x86/kernel/smp.c | 2 +-
arch/x86/kernel/smpboot.c | 8 ++++----
arch/x86/xen/smp_pv.c | 4 ++--
5 files changed, 10 insertions(+), 15 deletions(-)
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -272,8 +272,8 @@ config X86
select HAVE_UNSTABLE_SCHED_CLOCK
select HAVE_USER_RETURN_NOTIFIER
select HAVE_GENERIC_VDSO
- select HOTPLUG_CORE_SYNC_FULL if SMP
select HOTPLUG_SMT if SMP
+ select HOTPLUG_SPLIT_STARTUP if SMP
select IRQ_FORCED_THREADING
select NEED_PER_CPU_EMBED_FIRST_CHUNK
select NEED_PER_CPU_PAGE_FIRST_CHUNK
--- a/arch/x86/include/asm/smp.h
+++ b/arch/x86/include/asm/smp.h
@@ -40,7 +40,7 @@ struct smp_ops {
void (*cleanup_dead_cpu)(unsigned cpu);
void (*poll_sync_state)(void);
- int (*cpu_up)(unsigned cpu, struct task_struct *tidle);
+ int (*kick_ap_alive)(unsigned cpu, struct task_struct *tidle);
int (*cpu_disable)(void);
void (*cpu_die)(unsigned int cpu);
void (*play_dead)(void);
@@ -80,11 +80,6 @@ static inline void smp_cpus_done(unsigne
smp_ops.smp_cpus_done(max_cpus);
}
-static inline int __cpu_up(unsigned int cpu, struct task_struct *tidle)
-{
- return smp_ops.cpu_up(cpu, tidle);
-}
-
static inline int __cpu_disable(void)
{
return smp_ops.cpu_disable();
@@ -123,7 +118,7 @@ void native_smp_prepare_cpus(unsigned in
void calculate_max_logical_packages(void);
void native_smp_cpus_done(unsigned int max_cpus);
int common_cpu_up(unsigned int cpunum, struct task_struct *tidle);
-int native_cpu_up(unsigned int cpunum, struct task_struct *tidle);
+int native_kick_ap(unsigned int cpu, struct task_struct *tidle);
int native_cpu_disable(void);
void hlt_play_dead(void);
void native_play_dead(void);
--- a/arch/x86/kernel/smp.c
+++ b/arch/x86/kernel/smp.c
@@ -268,7 +268,7 @@ struct smp_ops smp_ops = {
#endif
.smp_send_reschedule = native_smp_send_reschedule,
- .cpu_up = native_cpu_up,
+ .kick_ap_alive = native_kick_ap,
.cpu_disable = native_cpu_disable,
.play_dead = native_play_dead,
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -1070,7 +1070,7 @@ static int do_boot_cpu(int apicid, int c
return ret;
}
-static int native_kick_ap(unsigned int cpu, struct task_struct *tidle)
+int native_kick_ap(unsigned int cpu, struct task_struct *tidle)
{
int apicid = apic->cpu_present_to_apicid(cpu);
int err;
@@ -1106,15 +1106,15 @@ static int native_kick_ap(unsigned int c
return err;
}
-int native_cpu_up(unsigned int cpu, struct task_struct *tidle)
+int arch_cpuhp_kick_ap_alive(unsigned int cpu, struct task_struct *tidle)
{
- return native_kick_ap(cpu, tidle);
+ return smp_ops.kick_ap_alive(cpu, tidle);
}
void arch_cpuhp_cleanup_kick_cpu(unsigned int cpu)
{
/* Cleanup possible dangling ends... */
- if (smp_ops.cpu_up == native_cpu_up && x86_platform.legacy.warm_reset)
+ if (smp_ops.kick_ap_alive == native_kick_ap && x86_platform.legacy.warm_reset)
smpboot_restore_warm_reset_vector();
}
--- a/arch/x86/xen/smp_pv.c
+++ b/arch/x86/xen/smp_pv.c
@@ -314,7 +314,7 @@ cpu_initialize_context(unsigned int cpu,
return 0;
}
-static int xen_pv_cpu_up(unsigned int cpu, struct task_struct *idle)
+static int xen_pv_kick_ap(unsigned int cpu, struct task_struct *idle)
{
int rc;
@@ -438,7 +438,7 @@ static const struct smp_ops xen_smp_ops
.smp_prepare_cpus = xen_pv_smp_prepare_cpus,
.smp_cpus_done = xen_smp_cpus_done,
- .cpu_up = xen_pv_cpu_up,
+ .kick_ap_alive = xen_pv_kick_ap,
.cpu_die = xen_pv_cpu_die,
.cleanup_dead_cpu = xen_pv_cleanup_dead_cpu,
.poll_sync_state = xen_pv_poll_sync_state,
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 30/37] x86/smpboot: Enable split CPU startup
@ 2023-04-14 23:45 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:45 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
The x86 CPU bringup state currently does AP wake-up, wait for AP to
respond and then release it for full bringup.
It is safe to be split into a wake-up and and a separate wait+release
state.
Provide the required functions and enable the split CPU bringup, which
prepares for parallel bringup, where the bringup of the non-boot CPUs takes
two iterations: One to prepare and wake all APs and the second to wait and
release them. Depending on timing this can eliminate the wait time
completely.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/Kconfig | 2 +-
arch/x86/include/asm/smp.h | 9 ++-------
arch/x86/kernel/smp.c | 2 +-
arch/x86/kernel/smpboot.c | 8 ++++----
arch/x86/xen/smp_pv.c | 4 ++--
5 files changed, 10 insertions(+), 15 deletions(-)
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -272,8 +272,8 @@ config X86
select HAVE_UNSTABLE_SCHED_CLOCK
select HAVE_USER_RETURN_NOTIFIER
select HAVE_GENERIC_VDSO
- select HOTPLUG_CORE_SYNC_FULL if SMP
select HOTPLUG_SMT if SMP
+ select HOTPLUG_SPLIT_STARTUP if SMP
select IRQ_FORCED_THREADING
select NEED_PER_CPU_EMBED_FIRST_CHUNK
select NEED_PER_CPU_PAGE_FIRST_CHUNK
--- a/arch/x86/include/asm/smp.h
+++ b/arch/x86/include/asm/smp.h
@@ -40,7 +40,7 @@ struct smp_ops {
void (*cleanup_dead_cpu)(unsigned cpu);
void (*poll_sync_state)(void);
- int (*cpu_up)(unsigned cpu, struct task_struct *tidle);
+ int (*kick_ap_alive)(unsigned cpu, struct task_struct *tidle);
int (*cpu_disable)(void);
void (*cpu_die)(unsigned int cpu);
void (*play_dead)(void);
@@ -80,11 +80,6 @@ static inline void smp_cpus_done(unsigne
smp_ops.smp_cpus_done(max_cpus);
}
-static inline int __cpu_up(unsigned int cpu, struct task_struct *tidle)
-{
- return smp_ops.cpu_up(cpu, tidle);
-}
-
static inline int __cpu_disable(void)
{
return smp_ops.cpu_disable();
@@ -123,7 +118,7 @@ void native_smp_prepare_cpus(unsigned in
void calculate_max_logical_packages(void);
void native_smp_cpus_done(unsigned int max_cpus);
int common_cpu_up(unsigned int cpunum, struct task_struct *tidle);
-int native_cpu_up(unsigned int cpunum, struct task_struct *tidle);
+int native_kick_ap(unsigned int cpu, struct task_struct *tidle);
int native_cpu_disable(void);
void hlt_play_dead(void);
void native_play_dead(void);
--- a/arch/x86/kernel/smp.c
+++ b/arch/x86/kernel/smp.c
@@ -268,7 +268,7 @@ struct smp_ops smp_ops = {
#endif
.smp_send_reschedule = native_smp_send_reschedule,
- .cpu_up = native_cpu_up,
+ .kick_ap_alive = native_kick_ap,
.cpu_disable = native_cpu_disable,
.play_dead = native_play_dead,
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -1070,7 +1070,7 @@ static int do_boot_cpu(int apicid, int c
return ret;
}
-static int native_kick_ap(unsigned int cpu, struct task_struct *tidle)
+int native_kick_ap(unsigned int cpu, struct task_struct *tidle)
{
int apicid = apic->cpu_present_to_apicid(cpu);
int err;
@@ -1106,15 +1106,15 @@ static int native_kick_ap(unsigned int c
return err;
}
-int native_cpu_up(unsigned int cpu, struct task_struct *tidle)
+int arch_cpuhp_kick_ap_alive(unsigned int cpu, struct task_struct *tidle)
{
- return native_kick_ap(cpu, tidle);
+ return smp_ops.kick_ap_alive(cpu, tidle);
}
void arch_cpuhp_cleanup_kick_cpu(unsigned int cpu)
{
/* Cleanup possible dangling ends... */
- if (smp_ops.cpu_up == native_cpu_up && x86_platform.legacy.warm_reset)
+ if (smp_ops.kick_ap_alive == native_kick_ap && x86_platform.legacy.warm_reset)
smpboot_restore_warm_reset_vector();
}
--- a/arch/x86/xen/smp_pv.c
+++ b/arch/x86/xen/smp_pv.c
@@ -314,7 +314,7 @@ cpu_initialize_context(unsigned int cpu,
return 0;
}
-static int xen_pv_cpu_up(unsigned int cpu, struct task_struct *idle)
+static int xen_pv_kick_ap(unsigned int cpu, struct task_struct *idle)
{
int rc;
@@ -438,7 +438,7 @@ static const struct smp_ops xen_smp_ops
.smp_prepare_cpus = xen_pv_smp_prepare_cpus,
.smp_cpus_done = xen_smp_cpus_done,
- .cpu_up = xen_pv_cpu_up,
+ .kick_ap_alive = xen_pv_kick_ap,
.cpu_die = xen_pv_cpu_die,
.cleanup_dead_cpu = xen_pv_cleanup_dead_cpu,
.poll_sync_state = xen_pv_poll_sync_state,
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 31/37] x86/apic: Provide cpu_primary_thread mask
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-14 23:45 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:45 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Make the primary thread tracking CPU mask based in preparation for simpler
handling of parallel bootup.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/include/asm/apic.h | 2 --
arch/x86/include/asm/topology.h | 19 +++++++++++++++----
arch/x86/kernel/apic/apic.c | 20 +++++++++-----------
arch/x86/kernel/smpboot.c | 12 +++---------
4 files changed, 27 insertions(+), 26 deletions(-)
--- a/arch/x86/include/asm/apic.h
+++ b/arch/x86/include/asm/apic.h
@@ -506,10 +506,8 @@ extern int default_check_phys_apicid_pre
#endif /* CONFIG_X86_LOCAL_APIC */
#ifdef CONFIG_SMP
-bool apic_id_is_primary_thread(unsigned int id);
void apic_smt_update(void);
#else
-static inline bool apic_id_is_primary_thread(unsigned int id) { return false; }
static inline void apic_smt_update(void) { }
#endif
--- a/arch/x86/include/asm/topology.h
+++ b/arch/x86/include/asm/topology.h
@@ -31,9 +31,9 @@
* CONFIG_NUMA.
*/
#include <linux/numa.h>
+#include <linux/cpumask.h>
#ifdef CONFIG_NUMA
-#include <linux/cpumask.h>
#include <asm/mpspec.h>
#include <asm/percpu.h>
@@ -139,9 +139,20 @@ static inline int topology_max_smt_threa
int topology_update_package_map(unsigned int apicid, unsigned int cpu);
int topology_update_die_map(unsigned int dieid, unsigned int cpu);
int topology_phys_to_logical_pkg(unsigned int pkg);
-bool topology_is_primary_thread(unsigned int cpu);
bool topology_smt_supported(void);
-#else
+
+extern struct cpumask __cpu_primary_thread_mask;
+#define cpu_primary_thread_mask ((const struct cpumask *)&__cpu_primary_thread_mask)
+
+/**
+ * topology_is_primary_thread - Check whether CPU is the primary SMT thread
+ * @cpu: CPU to check
+ */
+static inline bool topology_is_primary_thread(unsigned int cpu)
+{
+ return cpumask_test_cpu(cpu, cpu_primary_thread_mask);
+}
+#else /* CONFIG_SMP */
#define topology_max_packages() (1)
static inline int
topology_update_package_map(unsigned int apicid, unsigned int cpu) { return 0; }
@@ -152,7 +163,7 @@ static inline int topology_max_die_per_p
static inline int topology_max_smt_threads(void) { return 1; }
static inline bool topology_is_primary_thread(unsigned int cpu) { return true; }
static inline bool topology_smt_supported(void) { return false; }
-#endif
+#endif /* !CONFIG_SMP */
static inline void arch_fix_phys_package_id(int num, u32 slot)
{
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -2387,20 +2387,16 @@ bool arch_match_cpu_phys_id(int cpu, u64
}
#ifdef CONFIG_SMP
-/**
- * apic_id_is_primary_thread - Check whether APIC ID belongs to a primary thread
- * @apicid: APIC ID to check
- */
-bool apic_id_is_primary_thread(unsigned int apicid)
+static void cpu_mark_primary_thread(unsigned int cpu, unsigned int apicid)
{
- u32 mask;
-
- if (smp_num_siblings == 1)
- return true;
/* Isolate the SMT bit(s) in the APICID and check for 0 */
- mask = (1U << (fls(smp_num_siblings) - 1)) - 1;
- return !(apicid & mask);
+ u32 mask = (1U << (fls(smp_num_siblings) - 1)) - 1;
+
+ if (smp_num_siblings == 1 || !(apicid & mask))
+ cpumask_set_cpu(cpu, &__cpu_primary_thread_mask);
}
+#else
+static inline void cpu_mark_primary_thread(unsigned int cpu, unsigned int apicid) { }
#endif
/*
@@ -2545,6 +2541,8 @@ int generic_processor_info(int apicid, i
set_cpu_present(cpu, true);
num_processors++;
+ cpu_mark_primary_thread(cpu, apicid);
+
return cpu;
}
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -102,6 +102,9 @@ EXPORT_PER_CPU_SYMBOL(cpu_die_map);
DEFINE_PER_CPU_READ_MOSTLY(struct cpuinfo_x86, cpu_info);
EXPORT_PER_CPU_SYMBOL(cpu_info);
+/* CPUs which are the primary SMT threads */
+struct cpumask __cpu_primary_thread_mask __read_mostly;
+
/* Representing CPUs for which sibling maps can be computed */
static cpumask_var_t cpu_sibling_setup_mask;
@@ -294,15 +297,6 @@ static void notrace start_secondary(void
}
/**
- * topology_is_primary_thread - Check whether CPU is the primary SMT thread
- * @cpu: CPU to check
- */
-bool topology_is_primary_thread(unsigned int cpu)
-{
- return apic_id_is_primary_thread(per_cpu(x86_cpu_to_apicid, cpu));
-}
-
-/**
* topology_smt_supported - Check whether SMT is supported by the CPUs
*/
bool topology_smt_supported(void)
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 31/37] x86/apic: Provide cpu_primary_thread mask
@ 2023-04-14 23:45 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:45 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Make the primary thread tracking CPU mask based in preparation for simpler
handling of parallel bootup.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/include/asm/apic.h | 2 --
arch/x86/include/asm/topology.h | 19 +++++++++++++++----
arch/x86/kernel/apic/apic.c | 20 +++++++++-----------
arch/x86/kernel/smpboot.c | 12 +++---------
4 files changed, 27 insertions(+), 26 deletions(-)
--- a/arch/x86/include/asm/apic.h
+++ b/arch/x86/include/asm/apic.h
@@ -506,10 +506,8 @@ extern int default_check_phys_apicid_pre
#endif /* CONFIG_X86_LOCAL_APIC */
#ifdef CONFIG_SMP
-bool apic_id_is_primary_thread(unsigned int id);
void apic_smt_update(void);
#else
-static inline bool apic_id_is_primary_thread(unsigned int id) { return false; }
static inline void apic_smt_update(void) { }
#endif
--- a/arch/x86/include/asm/topology.h
+++ b/arch/x86/include/asm/topology.h
@@ -31,9 +31,9 @@
* CONFIG_NUMA.
*/
#include <linux/numa.h>
+#include <linux/cpumask.h>
#ifdef CONFIG_NUMA
-#include <linux/cpumask.h>
#include <asm/mpspec.h>
#include <asm/percpu.h>
@@ -139,9 +139,20 @@ static inline int topology_max_smt_threa
int topology_update_package_map(unsigned int apicid, unsigned int cpu);
int topology_update_die_map(unsigned int dieid, unsigned int cpu);
int topology_phys_to_logical_pkg(unsigned int pkg);
-bool topology_is_primary_thread(unsigned int cpu);
bool topology_smt_supported(void);
-#else
+
+extern struct cpumask __cpu_primary_thread_mask;
+#define cpu_primary_thread_mask ((const struct cpumask *)&__cpu_primary_thread_mask)
+
+/**
+ * topology_is_primary_thread - Check whether CPU is the primary SMT thread
+ * @cpu: CPU to check
+ */
+static inline bool topology_is_primary_thread(unsigned int cpu)
+{
+ return cpumask_test_cpu(cpu, cpu_primary_thread_mask);
+}
+#else /* CONFIG_SMP */
#define topology_max_packages() (1)
static inline int
topology_update_package_map(unsigned int apicid, unsigned int cpu) { return 0; }
@@ -152,7 +163,7 @@ static inline int topology_max_die_per_p
static inline int topology_max_smt_threads(void) { return 1; }
static inline bool topology_is_primary_thread(unsigned int cpu) { return true; }
static inline bool topology_smt_supported(void) { return false; }
-#endif
+#endif /* !CONFIG_SMP */
static inline void arch_fix_phys_package_id(int num, u32 slot)
{
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -2387,20 +2387,16 @@ bool arch_match_cpu_phys_id(int cpu, u64
}
#ifdef CONFIG_SMP
-/**
- * apic_id_is_primary_thread - Check whether APIC ID belongs to a primary thread
- * @apicid: APIC ID to check
- */
-bool apic_id_is_primary_thread(unsigned int apicid)
+static void cpu_mark_primary_thread(unsigned int cpu, unsigned int apicid)
{
- u32 mask;
-
- if (smp_num_siblings == 1)
- return true;
/* Isolate the SMT bit(s) in the APICID and check for 0 */
- mask = (1U << (fls(smp_num_siblings) - 1)) - 1;
- return !(apicid & mask);
+ u32 mask = (1U << (fls(smp_num_siblings) - 1)) - 1;
+
+ if (smp_num_siblings == 1 || !(apicid & mask))
+ cpumask_set_cpu(cpu, &__cpu_primary_thread_mask);
}
+#else
+static inline void cpu_mark_primary_thread(unsigned int cpu, unsigned int apicid) { }
#endif
/*
@@ -2545,6 +2541,8 @@ int generic_processor_info(int apicid, i
set_cpu_present(cpu, true);
num_processors++;
+ cpu_mark_primary_thread(cpu, apicid);
+
return cpu;
}
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -102,6 +102,9 @@ EXPORT_PER_CPU_SYMBOL(cpu_die_map);
DEFINE_PER_CPU_READ_MOSTLY(struct cpuinfo_x86, cpu_info);
EXPORT_PER_CPU_SYMBOL(cpu_info);
+/* CPUs which are the primary SMT threads */
+struct cpumask __cpu_primary_thread_mask __read_mostly;
+
/* Representing CPUs for which sibling maps can be computed */
static cpumask_var_t cpu_sibling_setup_mask;
@@ -294,15 +297,6 @@ static void notrace start_secondary(void
}
/**
- * topology_is_primary_thread - Check whether CPU is the primary SMT thread
- * @cpu: CPU to check
- */
-bool topology_is_primary_thread(unsigned int cpu)
-{
- return apic_id_is_primary_thread(per_cpu(x86_cpu_to_apicid, cpu));
-}
-
-/**
* topology_smt_supported - Check whether SMT is supported by the CPUs
*/
bool topology_smt_supported(void)
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 31/37] x86/apic: Provide cpu_primary_thread mask
@ 2023-04-14 23:45 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:45 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Make the primary thread tracking CPU mask based in preparation for simpler
handling of parallel bootup.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/include/asm/apic.h | 2 --
arch/x86/include/asm/topology.h | 19 +++++++++++++++----
arch/x86/kernel/apic/apic.c | 20 +++++++++-----------
arch/x86/kernel/smpboot.c | 12 +++---------
4 files changed, 27 insertions(+), 26 deletions(-)
--- a/arch/x86/include/asm/apic.h
+++ b/arch/x86/include/asm/apic.h
@@ -506,10 +506,8 @@ extern int default_check_phys_apicid_pre
#endif /* CONFIG_X86_LOCAL_APIC */
#ifdef CONFIG_SMP
-bool apic_id_is_primary_thread(unsigned int id);
void apic_smt_update(void);
#else
-static inline bool apic_id_is_primary_thread(unsigned int id) { return false; }
static inline void apic_smt_update(void) { }
#endif
--- a/arch/x86/include/asm/topology.h
+++ b/arch/x86/include/asm/topology.h
@@ -31,9 +31,9 @@
* CONFIG_NUMA.
*/
#include <linux/numa.h>
+#include <linux/cpumask.h>
#ifdef CONFIG_NUMA
-#include <linux/cpumask.h>
#include <asm/mpspec.h>
#include <asm/percpu.h>
@@ -139,9 +139,20 @@ static inline int topology_max_smt_threa
int topology_update_package_map(unsigned int apicid, unsigned int cpu);
int topology_update_die_map(unsigned int dieid, unsigned int cpu);
int topology_phys_to_logical_pkg(unsigned int pkg);
-bool topology_is_primary_thread(unsigned int cpu);
bool topology_smt_supported(void);
-#else
+
+extern struct cpumask __cpu_primary_thread_mask;
+#define cpu_primary_thread_mask ((const struct cpumask *)&__cpu_primary_thread_mask)
+
+/**
+ * topology_is_primary_thread - Check whether CPU is the primary SMT thread
+ * @cpu: CPU to check
+ */
+static inline bool topology_is_primary_thread(unsigned int cpu)
+{
+ return cpumask_test_cpu(cpu, cpu_primary_thread_mask);
+}
+#else /* CONFIG_SMP */
#define topology_max_packages() (1)
static inline int
topology_update_package_map(unsigned int apicid, unsigned int cpu) { return 0; }
@@ -152,7 +163,7 @@ static inline int topology_max_die_per_p
static inline int topology_max_smt_threads(void) { return 1; }
static inline bool topology_is_primary_thread(unsigned int cpu) { return true; }
static inline bool topology_smt_supported(void) { return false; }
-#endif
+#endif /* !CONFIG_SMP */
static inline void arch_fix_phys_package_id(int num, u32 slot)
{
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -2387,20 +2387,16 @@ bool arch_match_cpu_phys_id(int cpu, u64
}
#ifdef CONFIG_SMP
-/**
- * apic_id_is_primary_thread - Check whether APIC ID belongs to a primary thread
- * @apicid: APIC ID to check
- */
-bool apic_id_is_primary_thread(unsigned int apicid)
+static void cpu_mark_primary_thread(unsigned int cpu, unsigned int apicid)
{
- u32 mask;
-
- if (smp_num_siblings == 1)
- return true;
/* Isolate the SMT bit(s) in the APICID and check for 0 */
- mask = (1U << (fls(smp_num_siblings) - 1)) - 1;
- return !(apicid & mask);
+ u32 mask = (1U << (fls(smp_num_siblings) - 1)) - 1;
+
+ if (smp_num_siblings == 1 || !(apicid & mask))
+ cpumask_set_cpu(cpu, &__cpu_primary_thread_mask);
}
+#else
+static inline void cpu_mark_primary_thread(unsigned int cpu, unsigned int apicid) { }
#endif
/*
@@ -2545,6 +2541,8 @@ int generic_processor_info(int apicid, i
set_cpu_present(cpu, true);
num_processors++;
+ cpu_mark_primary_thread(cpu, apicid);
+
return cpu;
}
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -102,6 +102,9 @@ EXPORT_PER_CPU_SYMBOL(cpu_die_map);
DEFINE_PER_CPU_READ_MOSTLY(struct cpuinfo_x86, cpu_info);
EXPORT_PER_CPU_SYMBOL(cpu_info);
+/* CPUs which are the primary SMT threads */
+struct cpumask __cpu_primary_thread_mask __read_mostly;
+
/* Representing CPUs for which sibling maps can be computed */
static cpumask_var_t cpu_sibling_setup_mask;
@@ -294,15 +297,6 @@ static void notrace start_secondary(void
}
/**
- * topology_is_primary_thread - Check whether CPU is the primary SMT thread
- * @cpu: CPU to check
- */
-bool topology_is_primary_thread(unsigned int cpu)
-{
- return apic_id_is_primary_thread(per_cpu(x86_cpu_to_apicid, cpu));
-}
-
-/**
* topology_smt_supported - Check whether SMT is supported by the CPUs
*/
bool topology_smt_supported(void)
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 32/37] cpu/hotplug: Allow "parallel" bringup up to CPUHP_BP_KICK_AP_STATE
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-14 23:45 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:45 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
There is often significant latency in the early stages of CPU bringup, and
time is wasted by waking each CPU (e.g. with SIPI/INIT/INIT on x86) and
then waiting for it to respond before moving on to the next.
Allow a platform to enable parallel setup which brings all to be onlined
CPUs up to the CPUHP_BP_KICK_AP state. While this state advancement on the
control CPU (BP) is single-threaded the important part is the last state
CPUHP_BP_KICK_AP which wakes the to be onlined CPUs up.
This allows the CPUs to run up to the first sychronization point
cpuhp_ap_sync_alive() where they wait for the control CPU to release them
one by one for the full onlining procedure.
This parallelism depends on the CPU hotplug core sync mechanism which
ensures that the parallel brought up CPUs wait for release before touching
any state which would make the CPU visible to anything outside the hotplug
control mechanism.
To handle the SMT constraints of X86 correctly the bringup happens in two
iterations when CONFIG_HOTPLUG_SMT is enabled. The control CPU brings up
the primary SMT threads of each core first, which can load the microcode
without the need to rendevouz with the thread siblings. Once that's
completed it brings up the secondary SMT threads.
Co-developed-by: David Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
Documentation/admin-guide/kernel-parameters.txt | 6 +
arch/Kconfig | 4
include/linux/cpuhotplug.h | 1
kernel/cpu.c | 103 ++++++++++++++++++++++--
4 files changed, 109 insertions(+), 5 deletions(-)
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -815,6 +815,12 @@
on every CPU online, such as boot, and resume from suspend.
Default: 10000
+ cpuhp.parallel=
+ [SMP] Enable/disable parallel bringup of secondary CPUs
+ Format: <bool>
+ Default is enabled if CONFIG_HOTPLUG_PARALLEL=y. Otherwise
+ the parameter has no effect.
+
crash_kexec_post_notifiers
Run kdump after running panic-notifiers and dumping
kmsg. This only for the users who doubt kdump always
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -53,6 +53,10 @@ config HOTPLUG_SPLIT_STARTUP
bool
select HOTPLUG_CORE_SYNC_FULL
+config HOTPLUG_PARALLEL
+ bool
+ select HOTPLUG_SPLIT_STARTUP
+
config GENERIC_ENTRY
bool
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -526,6 +526,7 @@ void cpuhp_ap_sync_alive(void);
void arch_cpuhp_sync_state_poll(void);
void arch_cpuhp_cleanup_kick_cpu(unsigned int cpu);
int arch_cpuhp_kick_ap_alive(unsigned int cpu, struct task_struct *tidle);
+bool arch_cpuhp_init_parallel_bringup(void);
#ifdef CONFIG_HOTPLUG_CORE_SYNC_DEAD
void cpuhp_ap_report_dead(void);
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -649,8 +649,23 @@ bool cpu_smt_possible(void)
cpu_smt_control != CPU_SMT_NOT_SUPPORTED;
}
EXPORT_SYMBOL_GPL(cpu_smt_possible);
+
+static inline bool cpuhp_smt_aware(void)
+{
+ return topology_smt_supported();
+}
+
+static inline const struct cpumask *cpuhp_get_primary_thread_mask(void)
+{
+ return cpu_primary_thread_mask;
+}
#else
static inline bool cpu_smt_allowed(unsigned int cpu) { return true; }
+static inline bool cpuhp_smt_aware(void) { return false; }
+static inline const struct cpumask *cpuhp_get_primary_thread_mask(void)
+{
+ return cpu_present_mask;
+}
#endif
static inline enum cpuhp_state
@@ -1743,16 +1758,94 @@ int bringup_hibernate_cpu(unsigned int s
return 0;
}
-void __init bringup_nonboot_cpus(unsigned int setup_max_cpus)
+static void __init cpuhp_bringup_mask(const struct cpumask *mask, unsigned int ncpus,
+ enum cpuhp_state target)
{
unsigned int cpu;
- for_each_present_cpu(cpu) {
- if (num_online_cpus() >= setup_max_cpus)
+ for_each_cpu(cpu, mask) {
+ struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+
+ if (!--ncpus)
break;
- if (!cpu_online(cpu))
- cpu_up(cpu, CPUHP_ONLINE);
+
+ if (cpu_up(cpu, target) && can_rollback_cpu(st)) {
+ /*
+ * If this failed then cpu_up() might have only
+ * rolled back to CPUHP_BP_KICK_AP for the final
+ * online. Clean it up. NOOP if already rolled back.
+ */
+ WARN_ON(cpuhp_invoke_callback_range(false, cpu, st, CPUHP_OFFLINE));
+ }
+ }
+}
+
+#ifdef CONFIG_HOTPLUG_PARALLEL
+static bool __cpuhp_parallel_bringup __ro_after_init = true;
+
+static int __init parallel_bringup_parse_param(char *arg)
+{
+ return kstrtobool(arg, &__cpuhp_parallel_bringup);
+}
+early_param("cpuhp.parallel", parallel_bringup_parse_param);
+
+/*
+ * On architectures which have enabled parallel bringup this invokes all BP
+ * prepare states for each of the to be onlined APs first. The last state
+ * sends the startup IPI to the APs. The APs proceed through the low level
+ * bringup code in parallel and then wait for the control CPU to release
+ * them one by one for the final onlining procedure.
+ *
+ * This avoids waiting for each AP to respond to the startup IPI in
+ * CPUHP_BRINGUP_CPU.
+ */
+static bool __init cpuhp_bringup_cpus_parallel(unsigned int ncpus)
+{
+ const struct cpumask *mask = cpu_present_mask;
+
+ if (__cpuhp_parallel_bringup)
+ __cpuhp_parallel_bringup = arch_cpuhp_init_parallel_bringup();
+ if (!__cpuhp_parallel_bringup)
+ return false;
+
+ if (cpuhp_smt_aware()) {
+ const struct cpumask *pmask = cpuhp_get_primary_thread_mask();
+ static struct cpumask tmp_mask __initdata;
+
+ /*
+ * X86 requires to prevent that SMT siblings stopped while
+ * the primary thread does a microcode update for various
+ * reasons. Bring the primary threads up first.
+ */
+ cpumask_and(&tmp_mask, mask, pmask);
+ cpuhp_bringup_mask(&tmp_mask, ncpus, CPUHP_BP_KICK_AP);
+ cpuhp_bringup_mask(&tmp_mask, ncpus, CPUHP_ONLINE);
+ /* Account for the online CPUs */
+ ncpus -= num_online_cpus();
+ if (!ncpus)
+ return true;
+ /* Create the mask for secondary CPUs */
+ cpumask_andnot(&tmp_mask, mask, pmask);
+ mask = &tmp_mask;
}
+
+ /* Bring the not-yet started CPUs up */
+ cpuhp_bringup_mask(mask, ncpus, CPUHP_BP_KICK_AP);
+ cpuhp_bringup_mask(mask, ncpus, CPUHP_ONLINE);
+ return true;
+}
+#else
+static inline bool cpuhp_bringup_cpus_parallel(unsigned int ncpus) { return false; }
+#endif /* CONFIG_HOTPLUG_PARALLEL */
+
+void __init bringup_nonboot_cpus(unsigned int setup_max_cpus)
+{
+ /* Try parallel bringup optimization if enabled */
+ if (cpuhp_bringup_cpus_parallel(setup_max_cpus))
+ return;
+
+ /* Full per CPU serialized bringup */
+ cpuhp_bringup_mask(cpu_present_mask, setup_max_cpus, CPUHP_ONLINE);
}
#ifdef CONFIG_PM_SLEEP_SMP
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 32/37] cpu/hotplug: Allow "parallel" bringup up to CPUHP_BP_KICK_AP_STATE
@ 2023-04-14 23:45 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:45 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
There is often significant latency in the early stages of CPU bringup, and
time is wasted by waking each CPU (e.g. with SIPI/INIT/INIT on x86) and
then waiting for it to respond before moving on to the next.
Allow a platform to enable parallel setup which brings all to be onlined
CPUs up to the CPUHP_BP_KICK_AP state. While this state advancement on the
control CPU (BP) is single-threaded the important part is the last state
CPUHP_BP_KICK_AP which wakes the to be onlined CPUs up.
This allows the CPUs to run up to the first sychronization point
cpuhp_ap_sync_alive() where they wait for the control CPU to release them
one by one for the full onlining procedure.
This parallelism depends on the CPU hotplug core sync mechanism which
ensures that the parallel brought up CPUs wait for release before touching
any state which would make the CPU visible to anything outside the hotplug
control mechanism.
To handle the SMT constraints of X86 correctly the bringup happens in two
iterations when CONFIG_HOTPLUG_SMT is enabled. The control CPU brings up
the primary SMT threads of each core first, which can load the microcode
without the need to rendevouz with the thread siblings. Once that's
completed it brings up the secondary SMT threads.
Co-developed-by: David Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
Documentation/admin-guide/kernel-parameters.txt | 6 +
arch/Kconfig | 4
include/linux/cpuhotplug.h | 1
kernel/cpu.c | 103 ++++++++++++++++++++++--
4 files changed, 109 insertions(+), 5 deletions(-)
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -815,6 +815,12 @@
on every CPU online, such as boot, and resume from suspend.
Default: 10000
+ cpuhp.parallel=
+ [SMP] Enable/disable parallel bringup of secondary CPUs
+ Format: <bool>
+ Default is enabled if CONFIG_HOTPLUG_PARALLEL=y. Otherwise
+ the parameter has no effect.
+
crash_kexec_post_notifiers
Run kdump after running panic-notifiers and dumping
kmsg. This only for the users who doubt kdump always
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -53,6 +53,10 @@ config HOTPLUG_SPLIT_STARTUP
bool
select HOTPLUG_CORE_SYNC_FULL
+config HOTPLUG_PARALLEL
+ bool
+ select HOTPLUG_SPLIT_STARTUP
+
config GENERIC_ENTRY
bool
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -526,6 +526,7 @@ void cpuhp_ap_sync_alive(void);
void arch_cpuhp_sync_state_poll(void);
void arch_cpuhp_cleanup_kick_cpu(unsigned int cpu);
int arch_cpuhp_kick_ap_alive(unsigned int cpu, struct task_struct *tidle);
+bool arch_cpuhp_init_parallel_bringup(void);
#ifdef CONFIG_HOTPLUG_CORE_SYNC_DEAD
void cpuhp_ap_report_dead(void);
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -649,8 +649,23 @@ bool cpu_smt_possible(void)
cpu_smt_control != CPU_SMT_NOT_SUPPORTED;
}
EXPORT_SYMBOL_GPL(cpu_smt_possible);
+
+static inline bool cpuhp_smt_aware(void)
+{
+ return topology_smt_supported();
+}
+
+static inline const struct cpumask *cpuhp_get_primary_thread_mask(void)
+{
+ return cpu_primary_thread_mask;
+}
#else
static inline bool cpu_smt_allowed(unsigned int cpu) { return true; }
+static inline bool cpuhp_smt_aware(void) { return false; }
+static inline const struct cpumask *cpuhp_get_primary_thread_mask(void)
+{
+ return cpu_present_mask;
+}
#endif
static inline enum cpuhp_state
@@ -1743,16 +1758,94 @@ int bringup_hibernate_cpu(unsigned int s
return 0;
}
-void __init bringup_nonboot_cpus(unsigned int setup_max_cpus)
+static void __init cpuhp_bringup_mask(const struct cpumask *mask, unsigned int ncpus,
+ enum cpuhp_state target)
{
unsigned int cpu;
- for_each_present_cpu(cpu) {
- if (num_online_cpus() >= setup_max_cpus)
+ for_each_cpu(cpu, mask) {
+ struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+
+ if (!--ncpus)
break;
- if (!cpu_online(cpu))
- cpu_up(cpu, CPUHP_ONLINE);
+
+ if (cpu_up(cpu, target) && can_rollback_cpu(st)) {
+ /*
+ * If this failed then cpu_up() might have only
+ * rolled back to CPUHP_BP_KICK_AP for the final
+ * online. Clean it up. NOOP if already rolled back.
+ */
+ WARN_ON(cpuhp_invoke_callback_range(false, cpu, st, CPUHP_OFFLINE));
+ }
+ }
+}
+
+#ifdef CONFIG_HOTPLUG_PARALLEL
+static bool __cpuhp_parallel_bringup __ro_after_init = true;
+
+static int __init parallel_bringup_parse_param(char *arg)
+{
+ return kstrtobool(arg, &__cpuhp_parallel_bringup);
+}
+early_param("cpuhp.parallel", parallel_bringup_parse_param);
+
+/*
+ * On architectures which have enabled parallel bringup this invokes all BP
+ * prepare states for each of the to be onlined APs first. The last state
+ * sends the startup IPI to the APs. The APs proceed through the low level
+ * bringup code in parallel and then wait for the control CPU to release
+ * them one by one for the final onlining procedure.
+ *
+ * This avoids waiting for each AP to respond to the startup IPI in
+ * CPUHP_BRINGUP_CPU.
+ */
+static bool __init cpuhp_bringup_cpus_parallel(unsigned int ncpus)
+{
+ const struct cpumask *mask = cpu_present_mask;
+
+ if (__cpuhp_parallel_bringup)
+ __cpuhp_parallel_bringup = arch_cpuhp_init_parallel_bringup();
+ if (!__cpuhp_parallel_bringup)
+ return false;
+
+ if (cpuhp_smt_aware()) {
+ const struct cpumask *pmask = cpuhp_get_primary_thread_mask();
+ static struct cpumask tmp_mask __initdata;
+
+ /*
+ * X86 requires to prevent that SMT siblings stopped while
+ * the primary thread does a microcode update for various
+ * reasons. Bring the primary threads up first.
+ */
+ cpumask_and(&tmp_mask, mask, pmask);
+ cpuhp_bringup_mask(&tmp_mask, ncpus, CPUHP_BP_KICK_AP);
+ cpuhp_bringup_mask(&tmp_mask, ncpus, CPUHP_ONLINE);
+ /* Account for the online CPUs */
+ ncpus -= num_online_cpus();
+ if (!ncpus)
+ return true;
+ /* Create the mask for secondary CPUs */
+ cpumask_andnot(&tmp_mask, mask, pmask);
+ mask = &tmp_mask;
}
+
+ /* Bring the not-yet started CPUs up */
+ cpuhp_bringup_mask(mask, ncpus, CPUHP_BP_KICK_AP);
+ cpuhp_bringup_mask(mask, ncpus, CPUHP_ONLINE);
+ return true;
+}
+#else
+static inline bool cpuhp_bringup_cpus_parallel(unsigned int ncpus) { return false; }
+#endif /* CONFIG_HOTPLUG_PARALLEL */
+
+void __init bringup_nonboot_cpus(unsigned int setup_max_cpus)
+{
+ /* Try parallel bringup optimization if enabled */
+ if (cpuhp_bringup_cpus_parallel(setup_max_cpus))
+ return;
+
+ /* Full per CPU serialized bringup */
+ cpuhp_bringup_mask(cpu_present_mask, setup_max_cpus, CPUHP_ONLINE);
}
#ifdef CONFIG_PM_SLEEP_SMP
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 32/37] cpu/hotplug: Allow "parallel" bringup up to CPUHP_BP_KICK_AP_STATE
@ 2023-04-14 23:45 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:45 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
There is often significant latency in the early stages of CPU bringup, and
time is wasted by waking each CPU (e.g. with SIPI/INIT/INIT on x86) and
then waiting for it to respond before moving on to the next.
Allow a platform to enable parallel setup which brings all to be onlined
CPUs up to the CPUHP_BP_KICK_AP state. While this state advancement on the
control CPU (BP) is single-threaded the important part is the last state
CPUHP_BP_KICK_AP which wakes the to be onlined CPUs up.
This allows the CPUs to run up to the first sychronization point
cpuhp_ap_sync_alive() where they wait for the control CPU to release them
one by one for the full onlining procedure.
This parallelism depends on the CPU hotplug core sync mechanism which
ensures that the parallel brought up CPUs wait for release before touching
any state which would make the CPU visible to anything outside the hotplug
control mechanism.
To handle the SMT constraints of X86 correctly the bringup happens in two
iterations when CONFIG_HOTPLUG_SMT is enabled. The control CPU brings up
the primary SMT threads of each core first, which can load the microcode
without the need to rendevouz with the thread siblings. Once that's
completed it brings up the secondary SMT threads.
Co-developed-by: David Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
Documentation/admin-guide/kernel-parameters.txt | 6 +
arch/Kconfig | 4
include/linux/cpuhotplug.h | 1
kernel/cpu.c | 103 ++++++++++++++++++++++--
4 files changed, 109 insertions(+), 5 deletions(-)
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -815,6 +815,12 @@
on every CPU online, such as boot, and resume from suspend.
Default: 10000
+ cpuhp.parallel=
+ [SMP] Enable/disable parallel bringup of secondary CPUs
+ Format: <bool>
+ Default is enabled if CONFIG_HOTPLUG_PARALLEL=y. Otherwise
+ the parameter has no effect.
+
crash_kexec_post_notifiers
Run kdump after running panic-notifiers and dumping
kmsg. This only for the users who doubt kdump always
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -53,6 +53,10 @@ config HOTPLUG_SPLIT_STARTUP
bool
select HOTPLUG_CORE_SYNC_FULL
+config HOTPLUG_PARALLEL
+ bool
+ select HOTPLUG_SPLIT_STARTUP
+
config GENERIC_ENTRY
bool
--- a/include/linux/cpuhotplug.h
+++ b/include/linux/cpuhotplug.h
@@ -526,6 +526,7 @@ void cpuhp_ap_sync_alive(void);
void arch_cpuhp_sync_state_poll(void);
void arch_cpuhp_cleanup_kick_cpu(unsigned int cpu);
int arch_cpuhp_kick_ap_alive(unsigned int cpu, struct task_struct *tidle);
+bool arch_cpuhp_init_parallel_bringup(void);
#ifdef CONFIG_HOTPLUG_CORE_SYNC_DEAD
void cpuhp_ap_report_dead(void);
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -649,8 +649,23 @@ bool cpu_smt_possible(void)
cpu_smt_control != CPU_SMT_NOT_SUPPORTED;
}
EXPORT_SYMBOL_GPL(cpu_smt_possible);
+
+static inline bool cpuhp_smt_aware(void)
+{
+ return topology_smt_supported();
+}
+
+static inline const struct cpumask *cpuhp_get_primary_thread_mask(void)
+{
+ return cpu_primary_thread_mask;
+}
#else
static inline bool cpu_smt_allowed(unsigned int cpu) { return true; }
+static inline bool cpuhp_smt_aware(void) { return false; }
+static inline const struct cpumask *cpuhp_get_primary_thread_mask(void)
+{
+ return cpu_present_mask;
+}
#endif
static inline enum cpuhp_state
@@ -1743,16 +1758,94 @@ int bringup_hibernate_cpu(unsigned int s
return 0;
}
-void __init bringup_nonboot_cpus(unsigned int setup_max_cpus)
+static void __init cpuhp_bringup_mask(const struct cpumask *mask, unsigned int ncpus,
+ enum cpuhp_state target)
{
unsigned int cpu;
- for_each_present_cpu(cpu) {
- if (num_online_cpus() >= setup_max_cpus)
+ for_each_cpu(cpu, mask) {
+ struct cpuhp_cpu_state *st = per_cpu_ptr(&cpuhp_state, cpu);
+
+ if (!--ncpus)
break;
- if (!cpu_online(cpu))
- cpu_up(cpu, CPUHP_ONLINE);
+
+ if (cpu_up(cpu, target) && can_rollback_cpu(st)) {
+ /*
+ * If this failed then cpu_up() might have only
+ * rolled back to CPUHP_BP_KICK_AP for the final
+ * online. Clean it up. NOOP if already rolled back.
+ */
+ WARN_ON(cpuhp_invoke_callback_range(false, cpu, st, CPUHP_OFFLINE));
+ }
+ }
+}
+
+#ifdef CONFIG_HOTPLUG_PARALLEL
+static bool __cpuhp_parallel_bringup __ro_after_init = true;
+
+static int __init parallel_bringup_parse_param(char *arg)
+{
+ return kstrtobool(arg, &__cpuhp_parallel_bringup);
+}
+early_param("cpuhp.parallel", parallel_bringup_parse_param);
+
+/*
+ * On architectures which have enabled parallel bringup this invokes all BP
+ * prepare states for each of the to be onlined APs first. The last state
+ * sends the startup IPI to the APs. The APs proceed through the low level
+ * bringup code in parallel and then wait for the control CPU to release
+ * them one by one for the final onlining procedure.
+ *
+ * This avoids waiting for each AP to respond to the startup IPI in
+ * CPUHP_BRINGUP_CPU.
+ */
+static bool __init cpuhp_bringup_cpus_parallel(unsigned int ncpus)
+{
+ const struct cpumask *mask = cpu_present_mask;
+
+ if (__cpuhp_parallel_bringup)
+ __cpuhp_parallel_bringup = arch_cpuhp_init_parallel_bringup();
+ if (!__cpuhp_parallel_bringup)
+ return false;
+
+ if (cpuhp_smt_aware()) {
+ const struct cpumask *pmask = cpuhp_get_primary_thread_mask();
+ static struct cpumask tmp_mask __initdata;
+
+ /*
+ * X86 requires to prevent that SMT siblings stopped while
+ * the primary thread does a microcode update for various
+ * reasons. Bring the primary threads up first.
+ */
+ cpumask_and(&tmp_mask, mask, pmask);
+ cpuhp_bringup_mask(&tmp_mask, ncpus, CPUHP_BP_KICK_AP);
+ cpuhp_bringup_mask(&tmp_mask, ncpus, CPUHP_ONLINE);
+ /* Account for the online CPUs */
+ ncpus -= num_online_cpus();
+ if (!ncpus)
+ return true;
+ /* Create the mask for secondary CPUs */
+ cpumask_andnot(&tmp_mask, mask, pmask);
+ mask = &tmp_mask;
}
+
+ /* Bring the not-yet started CPUs up */
+ cpuhp_bringup_mask(mask, ncpus, CPUHP_BP_KICK_AP);
+ cpuhp_bringup_mask(mask, ncpus, CPUHP_ONLINE);
+ return true;
+}
+#else
+static inline bool cpuhp_bringup_cpus_parallel(unsigned int ncpus) { return false; }
+#endif /* CONFIG_HOTPLUG_PARALLEL */
+
+void __init bringup_nonboot_cpus(unsigned int setup_max_cpus)
+{
+ /* Try parallel bringup optimization if enabled */
+ if (cpuhp_bringup_cpus_parallel(setup_max_cpus))
+ return;
+
+ /* Full per CPU serialized bringup */
+ cpuhp_bringup_mask(cpu_present_mask, setup_max_cpus, CPUHP_ONLINE);
}
#ifdef CONFIG_PM_SLEEP_SMP
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 33/37] x86/topology: Store extended topology leaf information
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-14 23:45 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:45 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Save the extended topology leaf number if it exists and is valid in
preparation of parallel CPU bringup.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/include/asm/topology.h | 1 +
arch/x86/kernel/cpu/topology.c | 3 +++
2 files changed, 4 insertions(+)
--- a/arch/x86/include/asm/topology.h
+++ b/arch/x86/include/asm/topology.h
@@ -121,6 +121,7 @@ extern unsigned int __max_die_per_packag
#define topology_core_cpumask(cpu) (per_cpu(cpu_core_map, cpu))
#define topology_sibling_cpumask(cpu) (per_cpu(cpu_sibling_map, cpu))
+extern unsigned int topology_extended_leaf;
extern unsigned int __max_logical_packages;
#define topology_max_packages() (__max_logical_packages)
--- a/arch/x86/kernel/cpu/topology.c
+++ b/arch/x86/kernel/cpu/topology.c
@@ -29,6 +29,8 @@ unsigned int __max_die_per_package __rea
EXPORT_SYMBOL(__max_die_per_package);
#ifdef CONFIG_SMP
+unsigned int topology_extended_leaf __read_mostly;
+
/*
* Check if given CPUID extended topology "leaf" is implemented
*/
@@ -72,6 +74,7 @@ int detect_extended_topology_early(struc
if (leaf < 0)
return -1;
+ topology_extended_leaf = leaf;
set_cpu_cap(c, X86_FEATURE_XTOPOLOGY);
cpuid_count(leaf, SMT_LEVEL, &eax, &ebx, &ecx, &edx);
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 33/37] x86/topology: Store extended topology leaf information
@ 2023-04-14 23:45 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:45 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Save the extended topology leaf number if it exists and is valid in
preparation of parallel CPU bringup.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/include/asm/topology.h | 1 +
arch/x86/kernel/cpu/topology.c | 3 +++
2 files changed, 4 insertions(+)
--- a/arch/x86/include/asm/topology.h
+++ b/arch/x86/include/asm/topology.h
@@ -121,6 +121,7 @@ extern unsigned int __max_die_per_packag
#define topology_core_cpumask(cpu) (per_cpu(cpu_core_map, cpu))
#define topology_sibling_cpumask(cpu) (per_cpu(cpu_sibling_map, cpu))
+extern unsigned int topology_extended_leaf;
extern unsigned int __max_logical_packages;
#define topology_max_packages() (__max_logical_packages)
--- a/arch/x86/kernel/cpu/topology.c
+++ b/arch/x86/kernel/cpu/topology.c
@@ -29,6 +29,8 @@ unsigned int __max_die_per_package __rea
EXPORT_SYMBOL(__max_die_per_package);
#ifdef CONFIG_SMP
+unsigned int topology_extended_leaf __read_mostly;
+
/*
* Check if given CPUID extended topology "leaf" is implemented
*/
@@ -72,6 +74,7 @@ int detect_extended_topology_early(struc
if (leaf < 0)
return -1;
+ topology_extended_leaf = leaf;
set_cpu_cap(c, X86_FEATURE_XTOPOLOGY);
cpuid_count(leaf, SMT_LEVEL, &eax, &ebx, &ecx, &edx);
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 33/37] x86/topology: Store extended topology leaf information
@ 2023-04-14 23:45 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:45 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Save the extended topology leaf number if it exists and is valid in
preparation of parallel CPU bringup.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/include/asm/topology.h | 1 +
arch/x86/kernel/cpu/topology.c | 3 +++
2 files changed, 4 insertions(+)
--- a/arch/x86/include/asm/topology.h
+++ b/arch/x86/include/asm/topology.h
@@ -121,6 +121,7 @@ extern unsigned int __max_die_per_packag
#define topology_core_cpumask(cpu) (per_cpu(cpu_core_map, cpu))
#define topology_sibling_cpumask(cpu) (per_cpu(cpu_sibling_map, cpu))
+extern unsigned int topology_extended_leaf;
extern unsigned int __max_logical_packages;
#define topology_max_packages() (__max_logical_packages)
--- a/arch/x86/kernel/cpu/topology.c
+++ b/arch/x86/kernel/cpu/topology.c
@@ -29,6 +29,8 @@ unsigned int __max_die_per_package __rea
EXPORT_SYMBOL(__max_die_per_package);
#ifdef CONFIG_SMP
+unsigned int topology_extended_leaf __read_mostly;
+
/*
* Check if given CPUID extended topology "leaf" is implemented
*/
@@ -72,6 +74,7 @@ int detect_extended_topology_early(struc
if (leaf < 0)
return -1;
+ topology_extended_leaf = leaf;
set_cpu_cap(c, X86_FEATURE_XTOPOLOGY);
cpuid_count(leaf, SMT_LEVEL, &eax, &ebx, &ecx, &edx);
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 34/37] x86/cpu/amd; Invoke detect_extended_topology_early() on boot CPU
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-14 23:45 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:45 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
The early detection stores the extended topology leaf number which is
required for parallel hotplug.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/kernel/cpu/amd.c | 2 ++
1 file changed, 2 insertions(+)
--- a/arch/x86/kernel/cpu/amd.c
+++ b/arch/x86/kernel/cpu/amd.c
@@ -692,6 +692,8 @@ static void early_init_amd(struct cpuinf
}
}
+ detect_extended_topology_early(c);
+
if (cpu_has(c, X86_FEATURE_TOPOEXT))
smp_num_siblings = ((cpuid_ebx(0x8000001e) >> 8) & 0xff) + 1;
}
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 34/37] x86/cpu/amd; Invoke detect_extended_topology_early() on boot CPU
@ 2023-04-14 23:45 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:45 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
The early detection stores the extended topology leaf number which is
required for parallel hotplug.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/kernel/cpu/amd.c | 2 ++
1 file changed, 2 insertions(+)
--- a/arch/x86/kernel/cpu/amd.c
+++ b/arch/x86/kernel/cpu/amd.c
@@ -692,6 +692,8 @@ static void early_init_amd(struct cpuinf
}
}
+ detect_extended_topology_early(c);
+
if (cpu_has(c, X86_FEATURE_TOPOEXT))
smp_num_siblings = ((cpuid_ebx(0x8000001e) >> 8) & 0xff) + 1;
}
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 34/37] x86/cpu/amd; Invoke detect_extended_topology_early() on boot CPU
@ 2023-04-14 23:45 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:45 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
The early detection stores the extended topology leaf number which is
required for parallel hotplug.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/kernel/cpu/amd.c | 2 ++
1 file changed, 2 insertions(+)
--- a/arch/x86/kernel/cpu/amd.c
+++ b/arch/x86/kernel/cpu/amd.c
@@ -692,6 +692,8 @@ static void early_init_amd(struct cpuinf
}
}
+ detect_extended_topology_early(c);
+
if (cpu_has(c, X86_FEATURE_TOPOEXT))
smp_num_siblings = ((cpuid_ebx(0x8000001e) >> 8) & 0xff) + 1;
}
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 35/37] x86/smpboot: Support parallel startup of secondary CPUs
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-14 23:45 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:45 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
From: David Woodhouse <dwmw@amazon.co.uk>
Rework the real-mode startup code to allow for APs to be brought up in
parallel. This is in two parts:
1. Introduce a bit-spinlock to prevent them from all using the real
mode stack at the same time.
2. Avoid needing to use the global smpboot_control variable to pass
each AP its CPU number.
To achieve the latter, export the cpuid_to_apicid[] array so that each
AP can find its own CPU number by searching therein based on its APIC ID.
Introduce flags in the top bits of smpboot_control which indicate methods
by which an AP should find its CPU number. For a serialized bringup, the
CPU number is explicitly passed in the low bits of smpboot_control as
before. For parallel mode there are flags directing the AP to find its APIC
ID in CPUID leaf 0x0b or 1x1f (for X2APIC mode) or CPUID leaf 0x01 where 8
bits are sufficient, then perform the cpuid_to_apicid[] lookup with that.
Aside from the fact that APs will now look up their CPU number via the
newly-exported cpuid_to_apicid[] table, there is no behavioural change
intended, since the parallel bootup has not yet been enabled.
[ tglx: Initial proof of concept patch with bitlock and APIC ID lookup ]
[ dwmw2: Rework and testing, commit message, CPUID 0x1 and CPU0 support ]
[ seanc: Fix stray override of initial_gs in common_cpu_up() ]
[ Oleksandr Natalenko: reported suspend/resume issue fixed in
x86_acpi_suspend_lowlevel ]
Co-developed-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Co-developed-by: Brian Gerst <brgerst@gmail.com>
Signed-off-by: Brian Gerst <brgerst@gmail.com>
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: Usama Arif <usama.arif@bytedance.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/include/asm/apic.h | 2
arch/x86/include/asm/realmode.h | 3 +
arch/x86/include/asm/smp.h | 8 +++
arch/x86/kernel/acpi/sleep.c | 9 +++
arch/x86/kernel/apic/apic.c | 2
arch/x86/kernel/head_64.S | 79 ++++++++++++++++++++++++++++++++++-
arch/x86/kernel/smpboot.c | 5 --
arch/x86/realmode/init.c | 3 +
arch/x86/realmode/rm/trampoline_64.S | 27 +++++++++--
9 files changed, 125 insertions(+), 13 deletions(-)
--- a/arch/x86/include/asm/apic.h
+++ b/arch/x86/include/asm/apic.h
@@ -55,6 +55,8 @@ extern int local_apic_timer_c2_ok;
extern int disable_apic;
extern unsigned int lapic_timer_period;
+extern int cpuid_to_apicid[];
+
extern enum apic_intr_mode_id apic_intr_mode;
enum apic_intr_mode_id {
APIC_PIC,
--- a/arch/x86/include/asm/realmode.h
+++ b/arch/x86/include/asm/realmode.h
@@ -52,6 +52,7 @@ struct trampoline_header {
u64 efer;
u32 cr4;
u32 flags;
+ u32 lock;
#endif
};
@@ -64,6 +65,8 @@ extern unsigned long initial_stack;
extern unsigned long initial_vc_handler;
#endif
+extern u32 *trampoline_lock;
+
extern unsigned char real_mode_blob[];
extern unsigned char real_mode_relocs[];
--- a/arch/x86/include/asm/smp.h
+++ b/arch/x86/include/asm/smp.h
@@ -198,4 +198,12 @@ extern unsigned int smpboot_control;
#endif /* !__ASSEMBLY__ */
+/* Control bits for startup_64 */
+#define STARTUP_APICID_CPUID_1F 0x80000000
+#define STARTUP_APICID_CPUID_0B 0x40000000
+#define STARTUP_APICID_CPUID_01 0x20000000
+
+/* Top 8 bits are reserved for control */
+#define STARTUP_PARALLEL_MASK 0xFF000000
+
#endif /* _ASM_X86_SMP_H */
--- a/arch/x86/kernel/acpi/sleep.c
+++ b/arch/x86/kernel/acpi/sleep.c
@@ -16,6 +16,7 @@
#include <asm/cacheflush.h>
#include <asm/realmode.h>
#include <asm/hypervisor.h>
+#include <asm/smp.h>
#include <linux/ftrace.h>
#include "../../realmode/rm/wakeup.h"
@@ -127,7 +128,13 @@ int x86_acpi_suspend_lowlevel(void)
* value is in the actual %rsp register.
*/
current->thread.sp = (unsigned long)temp_stack + sizeof(temp_stack);
- smpboot_control = smp_processor_id();
+ /*
+ * Ensure the CPU knows which one it is when it comes back, if
+ * it isn't in parallel mode and expected to work that out for
+ * itself.
+ */
+ if (!(smpboot_control & STARTUP_PARALLEL_MASK))
+ smpboot_control = smp_processor_id();
#endif
initial_code = (unsigned long)wakeup_long64;
saved_magic = 0x123456789abcdef0L;
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -2377,7 +2377,7 @@ static int nr_logical_cpuids = 1;
/*
* Used to store mapping between logical CPU IDs and APIC IDs.
*/
-static int cpuid_to_apicid[] = {
+int cpuid_to_apicid[] = {
[0 ... NR_CPUS - 1] = -1,
};
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -25,6 +25,7 @@
#include <asm/export.h>
#include <asm/nospec-branch.h>
#include <asm/fixmap.h>
+#include <asm/smp.h>
/*
* We are not able to switch in one step to the final KERNEL ADDRESS SPACE
@@ -234,8 +235,70 @@ SYM_INNER_LABEL(secondary_startup_64_no_
ANNOTATE_NOENDBR // above
#ifdef CONFIG_SMP
+ /*
+ * For parallel boot, the APIC ID is retrieved from CPUID, and then
+ * used to look up the CPU number. For booting a single CPU, the
+ * CPU number is encoded in smpboot_control.
+ *
+ * Bit 31 STARTUP_APICID_CPUID_1F flag (use CPUID 0x1f)
+ * Bit 30 STARTUP_APICID_CPUID_0B flag (use CPUID 0x0b)
+ * Bit 29 STARTUP_APICID_CPUID_01 flag (use CPUID 0x01)
+ * Bit 0-23 CPU# if STARTUP_APICID_CPUID_xx flags are not set
+ */
movl smpboot_control(%rip), %ecx
+ testl $STARTUP_APICID_CPUID_1F, %ecx
+ jnz .Luse_cpuid_1f
+ testl $STARTUP_APICID_CPUID_0B, %ecx
+ jnz .Luse_cpuid_0b
+ testl $STARTUP_APICID_CPUID_01, %ecx
+ jnz .Luse_cpuid_01
+ andl $(~STARTUP_PARALLEL_MASK), %ecx
+ jmp .Lsetup_cpu
+
+.Luse_cpuid_01:
+ mov $0x01, %eax
+ cpuid
+ mov %ebx, %edx
+ shr $24, %edx
+ jmp .Lsetup_AP
+
+.Luse_cpuid_0b:
+ mov $0x0B, %eax
+ xorl %ecx, %ecx
+ cpuid
+ jmp .Lsetup_AP
+
+.Luse_cpuid_1f:
+ mov $0x1f, %eax
+ xorl %ecx, %ecx
+ cpuid
+.Lsetup_AP:
+ /* EDX contains the APIC ID of the current CPU */
+ xorq %rcx, %rcx
+ leaq cpuid_to_apicid(%rip), %rbx
+
+.Lfind_cpunr:
+ cmpl (%rbx,%rcx,4), %edx
+ jz .Lsetup_cpu
+ inc %ecx
+#ifdef CONFIG_FORCE_NR_CPUS
+ cmpl $NR_CPUS, %ecx
+#else
+ cmpl nr_cpu_ids(%rip), %ecx
+#endif
+ jb .Lfind_cpunr
+
+ /* APIC ID not found in the table. Drop the trampoline lock and bail. */
+ movq trampoline_lock(%rip), %rax
+ lock
+ btrl $0, (%rax)
+
+1: cli
+ hlt
+ jmp 1b
+
+.Lsetup_cpu:
/* Get the per cpu offset for the given CPU# which is in ECX */
movq __per_cpu_offset(,%rcx,8), %rdx
#else
@@ -248,10 +311,20 @@ SYM_INNER_LABEL(secondary_startup_64_no_
*
* RDX contains the per-cpu offset
*/
- movq pcpu_hot + X86_current_task(%rdx), %rax
- movq TASK_threadsp(%rax), %rsp
+ movq pcpu_hot + X86_top_of_stack(%rdx), %rsp
/*
+ * Now that this CPU is running on its own stack, drop the realmode
+ * protection. For the boot CPU the pointer is NULL!
+ */
+ movq trampoline_lock(%rip), %rax
+ testq %rax, %rax
+ jz .Lsetup_gdt
+ lock
+ btrl $0, (%rax)
+
+.Lsetup_gdt:
+ /*
* We must switch to a new descriptor in kernel space for the GDT
* because soon the kernel won't have access anymore to the userspace
* addresses where we're currently running on. We have to do that here
@@ -435,6 +508,8 @@ SYM_DATA(initial_code, .quad x86_64_star
#ifdef CONFIG_AMD_MEM_ENCRYPT
SYM_DATA(initial_vc_handler, .quad handle_vc_boot_ghcb)
#endif
+
+SYM_DATA(trampoline_lock, .quad 0);
__FINITDATA
__INIT
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -985,10 +985,7 @@ int common_cpu_up(unsigned int cpu, stru
if (ret)
return ret;
-#ifdef CONFIG_X86_32
- /* Stack for startup_32 can be just as for start_secondary onwards */
per_cpu(pcpu_hot.top_of_stack, cpu) = task_top_of_stack(idle);
-#endif
return 0;
}
@@ -1014,7 +1011,7 @@ static int do_boot_cpu(int apicid, int c
if (IS_ENABLED(CONFIG_X86_32)) {
early_gdt_descr.address = (unsigned long)get_cpu_gdt_rw(cpu);
initial_stack = idle->thread.sp;
- } else {
+ } else if (!(smpboot_control & STARTUP_PARALLEL_MASK)) {
smpboot_control = cpu;
}
--- a/arch/x86/realmode/init.c
+++ b/arch/x86/realmode/init.c
@@ -154,6 +154,9 @@ static void __init setup_real_mode(void)
trampoline_header->flags = 0;
+ trampoline_lock = &trampoline_header->lock;
+ *trampoline_lock = 0;
+
trampoline_pgd = (u64 *) __va(real_mode_header->trampoline_pgd);
/* Map the real mode stub as virtual == physical */
--- a/arch/x86/realmode/rm/trampoline_64.S
+++ b/arch/x86/realmode/rm/trampoline_64.S
@@ -37,6 +37,24 @@
.text
.code16
+.macro LOAD_REALMODE_ESP
+ /*
+ * Make sure only one CPU fiddles with the realmode stack
+ */
+.Llock_rm\@:
+ btl $0, tr_lock
+ jnc 2f
+ pause
+ jmp .Llock_rm\@
+2:
+ lock
+ btsl $0, tr_lock
+ jc .Llock_rm\@
+
+ # Setup stack
+ movl $rm_stack_end, %esp
+.endm
+
.balign PAGE_SIZE
SYM_CODE_START(trampoline_start)
cli # We should be safe anyway
@@ -49,8 +67,7 @@ SYM_CODE_START(trampoline_start)
mov %ax, %es
mov %ax, %ss
- # Setup stack
- movl $rm_stack_end, %esp
+ LOAD_REALMODE_ESP
call verify_cpu # Verify the cpu supports long mode
testl %eax, %eax # Check for return code
@@ -93,8 +110,7 @@ SYM_CODE_START(sev_es_trampoline_start)
mov %ax, %es
mov %ax, %ss
- # Setup stack
- movl $rm_stack_end, %esp
+ LOAD_REALMODE_ESP
jmp .Lswitch_to_protected
SYM_CODE_END(sev_es_trampoline_start)
@@ -177,7 +193,7 @@ SYM_CODE_START(pa_trampoline_compat)
* In compatibility mode. Prep ESP and DX for startup_32, then disable
* paging and complete the switch to legacy 32-bit mode.
*/
- movl $rm_stack_end, %esp
+ LOAD_REALMODE_ESP
movw $__KERNEL_DS, %dx
movl $(CR0_STATE & ~X86_CR0_PG), %eax
@@ -241,6 +257,7 @@ SYM_DATA_START(trampoline_header)
SYM_DATA(tr_efer, .space 8)
SYM_DATA(tr_cr4, .space 4)
SYM_DATA(tr_flags, .space 4)
+ SYM_DATA(tr_lock, .space 4)
SYM_DATA_END(trampoline_header)
#include "trampoline_common.S"
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 35/37] x86/smpboot: Support parallel startup of secondary CPUs
@ 2023-04-14 23:45 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:45 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
From: David Woodhouse <dwmw@amazon.co.uk>
Rework the real-mode startup code to allow for APs to be brought up in
parallel. This is in two parts:
1. Introduce a bit-spinlock to prevent them from all using the real
mode stack at the same time.
2. Avoid needing to use the global smpboot_control variable to pass
each AP its CPU number.
To achieve the latter, export the cpuid_to_apicid[] array so that each
AP can find its own CPU number by searching therein based on its APIC ID.
Introduce flags in the top bits of smpboot_control which indicate methods
by which an AP should find its CPU number. For a serialized bringup, the
CPU number is explicitly passed in the low bits of smpboot_control as
before. For parallel mode there are flags directing the AP to find its APIC
ID in CPUID leaf 0x0b or 1x1f (for X2APIC mode) or CPUID leaf 0x01 where 8
bits are sufficient, then perform the cpuid_to_apicid[] lookup with that.
Aside from the fact that APs will now look up their CPU number via the
newly-exported cpuid_to_apicid[] table, there is no behavioural change
intended, since the parallel bootup has not yet been enabled.
[ tglx: Initial proof of concept patch with bitlock and APIC ID lookup ]
[ dwmw2: Rework and testing, commit message, CPUID 0x1 and CPU0 support ]
[ seanc: Fix stray override of initial_gs in common_cpu_up() ]
[ Oleksandr Natalenko: reported suspend/resume issue fixed in
x86_acpi_suspend_lowlevel ]
Co-developed-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Co-developed-by: Brian Gerst <brgerst@gmail.com>
Signed-off-by: Brian Gerst <brgerst@gmail.com>
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: Usama Arif <usama.arif@bytedance.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/include/asm/apic.h | 2
arch/x86/include/asm/realmode.h | 3 +
arch/x86/include/asm/smp.h | 8 +++
arch/x86/kernel/acpi/sleep.c | 9 +++
arch/x86/kernel/apic/apic.c | 2
arch/x86/kernel/head_64.S | 79 ++++++++++++++++++++++++++++++++++-
arch/x86/kernel/smpboot.c | 5 --
arch/x86/realmode/init.c | 3 +
arch/x86/realmode/rm/trampoline_64.S | 27 +++++++++--
9 files changed, 125 insertions(+), 13 deletions(-)
--- a/arch/x86/include/asm/apic.h
+++ b/arch/x86/include/asm/apic.h
@@ -55,6 +55,8 @@ extern int local_apic_timer_c2_ok;
extern int disable_apic;
extern unsigned int lapic_timer_period;
+extern int cpuid_to_apicid[];
+
extern enum apic_intr_mode_id apic_intr_mode;
enum apic_intr_mode_id {
APIC_PIC,
--- a/arch/x86/include/asm/realmode.h
+++ b/arch/x86/include/asm/realmode.h
@@ -52,6 +52,7 @@ struct trampoline_header {
u64 efer;
u32 cr4;
u32 flags;
+ u32 lock;
#endif
};
@@ -64,6 +65,8 @@ extern unsigned long initial_stack;
extern unsigned long initial_vc_handler;
#endif
+extern u32 *trampoline_lock;
+
extern unsigned char real_mode_blob[];
extern unsigned char real_mode_relocs[];
--- a/arch/x86/include/asm/smp.h
+++ b/arch/x86/include/asm/smp.h
@@ -198,4 +198,12 @@ extern unsigned int smpboot_control;
#endif /* !__ASSEMBLY__ */
+/* Control bits for startup_64 */
+#define STARTUP_APICID_CPUID_1F 0x80000000
+#define STARTUP_APICID_CPUID_0B 0x40000000
+#define STARTUP_APICID_CPUID_01 0x20000000
+
+/* Top 8 bits are reserved for control */
+#define STARTUP_PARALLEL_MASK 0xFF000000
+
#endif /* _ASM_X86_SMP_H */
--- a/arch/x86/kernel/acpi/sleep.c
+++ b/arch/x86/kernel/acpi/sleep.c
@@ -16,6 +16,7 @@
#include <asm/cacheflush.h>
#include <asm/realmode.h>
#include <asm/hypervisor.h>
+#include <asm/smp.h>
#include <linux/ftrace.h>
#include "../../realmode/rm/wakeup.h"
@@ -127,7 +128,13 @@ int x86_acpi_suspend_lowlevel(void)
* value is in the actual %rsp register.
*/
current->thread.sp = (unsigned long)temp_stack + sizeof(temp_stack);
- smpboot_control = smp_processor_id();
+ /*
+ * Ensure the CPU knows which one it is when it comes back, if
+ * it isn't in parallel mode and expected to work that out for
+ * itself.
+ */
+ if (!(smpboot_control & STARTUP_PARALLEL_MASK))
+ smpboot_control = smp_processor_id();
#endif
initial_code = (unsigned long)wakeup_long64;
saved_magic = 0x123456789abcdef0L;
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -2377,7 +2377,7 @@ static int nr_logical_cpuids = 1;
/*
* Used to store mapping between logical CPU IDs and APIC IDs.
*/
-static int cpuid_to_apicid[] = {
+int cpuid_to_apicid[] = {
[0 ... NR_CPUS - 1] = -1,
};
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -25,6 +25,7 @@
#include <asm/export.h>
#include <asm/nospec-branch.h>
#include <asm/fixmap.h>
+#include <asm/smp.h>
/*
* We are not able to switch in one step to the final KERNEL ADDRESS SPACE
@@ -234,8 +235,70 @@ SYM_INNER_LABEL(secondary_startup_64_no_
ANNOTATE_NOENDBR // above
#ifdef CONFIG_SMP
+ /*
+ * For parallel boot, the APIC ID is retrieved from CPUID, and then
+ * used to look up the CPU number. For booting a single CPU, the
+ * CPU number is encoded in smpboot_control.
+ *
+ * Bit 31 STARTUP_APICID_CPUID_1F flag (use CPUID 0x1f)
+ * Bit 30 STARTUP_APICID_CPUID_0B flag (use CPUID 0x0b)
+ * Bit 29 STARTUP_APICID_CPUID_01 flag (use CPUID 0x01)
+ * Bit 0-23 CPU# if STARTUP_APICID_CPUID_xx flags are not set
+ */
movl smpboot_control(%rip), %ecx
+ testl $STARTUP_APICID_CPUID_1F, %ecx
+ jnz .Luse_cpuid_1f
+ testl $STARTUP_APICID_CPUID_0B, %ecx
+ jnz .Luse_cpuid_0b
+ testl $STARTUP_APICID_CPUID_01, %ecx
+ jnz .Luse_cpuid_01
+ andl $(~STARTUP_PARALLEL_MASK), %ecx
+ jmp .Lsetup_cpu
+
+.Luse_cpuid_01:
+ mov $0x01, %eax
+ cpuid
+ mov %ebx, %edx
+ shr $24, %edx
+ jmp .Lsetup_AP
+
+.Luse_cpuid_0b:
+ mov $0x0B, %eax
+ xorl %ecx, %ecx
+ cpuid
+ jmp .Lsetup_AP
+
+.Luse_cpuid_1f:
+ mov $0x1f, %eax
+ xorl %ecx, %ecx
+ cpuid
+.Lsetup_AP:
+ /* EDX contains the APIC ID of the current CPU */
+ xorq %rcx, %rcx
+ leaq cpuid_to_apicid(%rip), %rbx
+
+.Lfind_cpunr:
+ cmpl (%rbx,%rcx,4), %edx
+ jz .Lsetup_cpu
+ inc %ecx
+#ifdef CONFIG_FORCE_NR_CPUS
+ cmpl $NR_CPUS, %ecx
+#else
+ cmpl nr_cpu_ids(%rip), %ecx
+#endif
+ jb .Lfind_cpunr
+
+ /* APIC ID not found in the table. Drop the trampoline lock and bail. */
+ movq trampoline_lock(%rip), %rax
+ lock
+ btrl $0, (%rax)
+
+1: cli
+ hlt
+ jmp 1b
+
+.Lsetup_cpu:
/* Get the per cpu offset for the given CPU# which is in ECX */
movq __per_cpu_offset(,%rcx,8), %rdx
#else
@@ -248,10 +311,20 @@ SYM_INNER_LABEL(secondary_startup_64_no_
*
* RDX contains the per-cpu offset
*/
- movq pcpu_hot + X86_current_task(%rdx), %rax
- movq TASK_threadsp(%rax), %rsp
+ movq pcpu_hot + X86_top_of_stack(%rdx), %rsp
/*
+ * Now that this CPU is running on its own stack, drop the realmode
+ * protection. For the boot CPU the pointer is NULL!
+ */
+ movq trampoline_lock(%rip), %rax
+ testq %rax, %rax
+ jz .Lsetup_gdt
+ lock
+ btrl $0, (%rax)
+
+.Lsetup_gdt:
+ /*
* We must switch to a new descriptor in kernel space for the GDT
* because soon the kernel won't have access anymore to the userspace
* addresses where we're currently running on. We have to do that here
@@ -435,6 +508,8 @@ SYM_DATA(initial_code, .quad x86_64_star
#ifdef CONFIG_AMD_MEM_ENCRYPT
SYM_DATA(initial_vc_handler, .quad handle_vc_boot_ghcb)
#endif
+
+SYM_DATA(trampoline_lock, .quad 0);
__FINITDATA
__INIT
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -985,10 +985,7 @@ int common_cpu_up(unsigned int cpu, stru
if (ret)
return ret;
-#ifdef CONFIG_X86_32
- /* Stack for startup_32 can be just as for start_secondary onwards */
per_cpu(pcpu_hot.top_of_stack, cpu) = task_top_of_stack(idle);
-#endif
return 0;
}
@@ -1014,7 +1011,7 @@ static int do_boot_cpu(int apicid, int c
if (IS_ENABLED(CONFIG_X86_32)) {
early_gdt_descr.address = (unsigned long)get_cpu_gdt_rw(cpu);
initial_stack = idle->thread.sp;
- } else {
+ } else if (!(smpboot_control & STARTUP_PARALLEL_MASK)) {
smpboot_control = cpu;
}
--- a/arch/x86/realmode/init.c
+++ b/arch/x86/realmode/init.c
@@ -154,6 +154,9 @@ static void __init setup_real_mode(void)
trampoline_header->flags = 0;
+ trampoline_lock = &trampoline_header->lock;
+ *trampoline_lock = 0;
+
trampoline_pgd = (u64 *) __va(real_mode_header->trampoline_pgd);
/* Map the real mode stub as virtual == physical */
--- a/arch/x86/realmode/rm/trampoline_64.S
+++ b/arch/x86/realmode/rm/trampoline_64.S
@@ -37,6 +37,24 @@
.text
.code16
+.macro LOAD_REALMODE_ESP
+ /*
+ * Make sure only one CPU fiddles with the realmode stack
+ */
+.Llock_rm\@:
+ btl $0, tr_lock
+ jnc 2f
+ pause
+ jmp .Llock_rm\@
+2:
+ lock
+ btsl $0, tr_lock
+ jc .Llock_rm\@
+
+ # Setup stack
+ movl $rm_stack_end, %esp
+.endm
+
.balign PAGE_SIZE
SYM_CODE_START(trampoline_start)
cli # We should be safe anyway
@@ -49,8 +67,7 @@ SYM_CODE_START(trampoline_start)
mov %ax, %es
mov %ax, %ss
- # Setup stack
- movl $rm_stack_end, %esp
+ LOAD_REALMODE_ESP
call verify_cpu # Verify the cpu supports long mode
testl %eax, %eax # Check for return code
@@ -93,8 +110,7 @@ SYM_CODE_START(sev_es_trampoline_start)
mov %ax, %es
mov %ax, %ss
- # Setup stack
- movl $rm_stack_end, %esp
+ LOAD_REALMODE_ESP
jmp .Lswitch_to_protected
SYM_CODE_END(sev_es_trampoline_start)
@@ -177,7 +193,7 @@ SYM_CODE_START(pa_trampoline_compat)
* In compatibility mode. Prep ESP and DX for startup_32, then disable
* paging and complete the switch to legacy 32-bit mode.
*/
- movl $rm_stack_end, %esp
+ LOAD_REALMODE_ESP
movw $__KERNEL_DS, %dx
movl $(CR0_STATE & ~X86_CR0_PG), %eax
@@ -241,6 +257,7 @@ SYM_DATA_START(trampoline_header)
SYM_DATA(tr_efer, .space 8)
SYM_DATA(tr_cr4, .space 4)
SYM_DATA(tr_flags, .space 4)
+ SYM_DATA(tr_lock, .space 4)
SYM_DATA_END(trampoline_header)
#include "trampoline_common.S"
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 35/37] x86/smpboot: Support parallel startup of secondary CPUs
@ 2023-04-14 23:45 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:45 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
From: David Woodhouse <dwmw@amazon.co.uk>
Rework the real-mode startup code to allow for APs to be brought up in
parallel. This is in two parts:
1. Introduce a bit-spinlock to prevent them from all using the real
mode stack at the same time.
2. Avoid needing to use the global smpboot_control variable to pass
each AP its CPU number.
To achieve the latter, export the cpuid_to_apicid[] array so that each
AP can find its own CPU number by searching therein based on its APIC ID.
Introduce flags in the top bits of smpboot_control which indicate methods
by which an AP should find its CPU number. For a serialized bringup, the
CPU number is explicitly passed in the low bits of smpboot_control as
before. For parallel mode there are flags directing the AP to find its APIC
ID in CPUID leaf 0x0b or 1x1f (for X2APIC mode) or CPUID leaf 0x01 where 8
bits are sufficient, then perform the cpuid_to_apicid[] lookup with that.
Aside from the fact that APs will now look up their CPU number via the
newly-exported cpuid_to_apicid[] table, there is no behavioural change
intended, since the parallel bootup has not yet been enabled.
[ tglx: Initial proof of concept patch with bitlock and APIC ID lookup ]
[ dwmw2: Rework and testing, commit message, CPUID 0x1 and CPU0 support ]
[ seanc: Fix stray override of initial_gs in common_cpu_up() ]
[ Oleksandr Natalenko: reported suspend/resume issue fixed in
x86_acpi_suspend_lowlevel ]
Co-developed-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Co-developed-by: Brian Gerst <brgerst@gmail.com>
Signed-off-by: Brian Gerst <brgerst@gmail.com>
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: Usama Arif <usama.arif@bytedance.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/include/asm/apic.h | 2
arch/x86/include/asm/realmode.h | 3 +
arch/x86/include/asm/smp.h | 8 +++
arch/x86/kernel/acpi/sleep.c | 9 +++
arch/x86/kernel/apic/apic.c | 2
arch/x86/kernel/head_64.S | 79 ++++++++++++++++++++++++++++++++++-
arch/x86/kernel/smpboot.c | 5 --
arch/x86/realmode/init.c | 3 +
arch/x86/realmode/rm/trampoline_64.S | 27 +++++++++--
9 files changed, 125 insertions(+), 13 deletions(-)
--- a/arch/x86/include/asm/apic.h
+++ b/arch/x86/include/asm/apic.h
@@ -55,6 +55,8 @@ extern int local_apic_timer_c2_ok;
extern int disable_apic;
extern unsigned int lapic_timer_period;
+extern int cpuid_to_apicid[];
+
extern enum apic_intr_mode_id apic_intr_mode;
enum apic_intr_mode_id {
APIC_PIC,
--- a/arch/x86/include/asm/realmode.h
+++ b/arch/x86/include/asm/realmode.h
@@ -52,6 +52,7 @@ struct trampoline_header {
u64 efer;
u32 cr4;
u32 flags;
+ u32 lock;
#endif
};
@@ -64,6 +65,8 @@ extern unsigned long initial_stack;
extern unsigned long initial_vc_handler;
#endif
+extern u32 *trampoline_lock;
+
extern unsigned char real_mode_blob[];
extern unsigned char real_mode_relocs[];
--- a/arch/x86/include/asm/smp.h
+++ b/arch/x86/include/asm/smp.h
@@ -198,4 +198,12 @@ extern unsigned int smpboot_control;
#endif /* !__ASSEMBLY__ */
+/* Control bits for startup_64 */
+#define STARTUP_APICID_CPUID_1F 0x80000000
+#define STARTUP_APICID_CPUID_0B 0x40000000
+#define STARTUP_APICID_CPUID_01 0x20000000
+
+/* Top 8 bits are reserved for control */
+#define STARTUP_PARALLEL_MASK 0xFF000000
+
#endif /* _ASM_X86_SMP_H */
--- a/arch/x86/kernel/acpi/sleep.c
+++ b/arch/x86/kernel/acpi/sleep.c
@@ -16,6 +16,7 @@
#include <asm/cacheflush.h>
#include <asm/realmode.h>
#include <asm/hypervisor.h>
+#include <asm/smp.h>
#include <linux/ftrace.h>
#include "../../realmode/rm/wakeup.h"
@@ -127,7 +128,13 @@ int x86_acpi_suspend_lowlevel(void)
* value is in the actual %rsp register.
*/
current->thread.sp = (unsigned long)temp_stack + sizeof(temp_stack);
- smpboot_control = smp_processor_id();
+ /*
+ * Ensure the CPU knows which one it is when it comes back, if
+ * it isn't in parallel mode and expected to work that out for
+ * itself.
+ */
+ if (!(smpboot_control & STARTUP_PARALLEL_MASK))
+ smpboot_control = smp_processor_id();
#endif
initial_code = (unsigned long)wakeup_long64;
saved_magic = 0x123456789abcdef0L;
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -2377,7 +2377,7 @@ static int nr_logical_cpuids = 1;
/*
* Used to store mapping between logical CPU IDs and APIC IDs.
*/
-static int cpuid_to_apicid[] = {
+int cpuid_to_apicid[] = {
[0 ... NR_CPUS - 1] = -1,
};
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -25,6 +25,7 @@
#include <asm/export.h>
#include <asm/nospec-branch.h>
#include <asm/fixmap.h>
+#include <asm/smp.h>
/*
* We are not able to switch in one step to the final KERNEL ADDRESS SPACE
@@ -234,8 +235,70 @@ SYM_INNER_LABEL(secondary_startup_64_no_
ANNOTATE_NOENDBR // above
#ifdef CONFIG_SMP
+ /*
+ * For parallel boot, the APIC ID is retrieved from CPUID, and then
+ * used to look up the CPU number. For booting a single CPU, the
+ * CPU number is encoded in smpboot_control.
+ *
+ * Bit 31 STARTUP_APICID_CPUID_1F flag (use CPUID 0x1f)
+ * Bit 30 STARTUP_APICID_CPUID_0B flag (use CPUID 0x0b)
+ * Bit 29 STARTUP_APICID_CPUID_01 flag (use CPUID 0x01)
+ * Bit 0-23 CPU# if STARTUP_APICID_CPUID_xx flags are not set
+ */
movl smpboot_control(%rip), %ecx
+ testl $STARTUP_APICID_CPUID_1F, %ecx
+ jnz .Luse_cpuid_1f
+ testl $STARTUP_APICID_CPUID_0B, %ecx
+ jnz .Luse_cpuid_0b
+ testl $STARTUP_APICID_CPUID_01, %ecx
+ jnz .Luse_cpuid_01
+ andl $(~STARTUP_PARALLEL_MASK), %ecx
+ jmp .Lsetup_cpu
+
+.Luse_cpuid_01:
+ mov $0x01, %eax
+ cpuid
+ mov %ebx, %edx
+ shr $24, %edx
+ jmp .Lsetup_AP
+
+.Luse_cpuid_0b:
+ mov $0x0B, %eax
+ xorl %ecx, %ecx
+ cpuid
+ jmp .Lsetup_AP
+
+.Luse_cpuid_1f:
+ mov $0x1f, %eax
+ xorl %ecx, %ecx
+ cpuid
+.Lsetup_AP:
+ /* EDX contains the APIC ID of the current CPU */
+ xorq %rcx, %rcx
+ leaq cpuid_to_apicid(%rip), %rbx
+
+.Lfind_cpunr:
+ cmpl (%rbx,%rcx,4), %edx
+ jz .Lsetup_cpu
+ inc %ecx
+#ifdef CONFIG_FORCE_NR_CPUS
+ cmpl $NR_CPUS, %ecx
+#else
+ cmpl nr_cpu_ids(%rip), %ecx
+#endif
+ jb .Lfind_cpunr
+
+ /* APIC ID not found in the table. Drop the trampoline lock and bail. */
+ movq trampoline_lock(%rip), %rax
+ lock
+ btrl $0, (%rax)
+
+1: cli
+ hlt
+ jmp 1b
+
+.Lsetup_cpu:
/* Get the per cpu offset for the given CPU# which is in ECX */
movq __per_cpu_offset(,%rcx,8), %rdx
#else
@@ -248,10 +311,20 @@ SYM_INNER_LABEL(secondary_startup_64_no_
*
* RDX contains the per-cpu offset
*/
- movq pcpu_hot + X86_current_task(%rdx), %rax
- movq TASK_threadsp(%rax), %rsp
+ movq pcpu_hot + X86_top_of_stack(%rdx), %rsp
/*
+ * Now that this CPU is running on its own stack, drop the realmode
+ * protection. For the boot CPU the pointer is NULL!
+ */
+ movq trampoline_lock(%rip), %rax
+ testq %rax, %rax
+ jz .Lsetup_gdt
+ lock
+ btrl $0, (%rax)
+
+.Lsetup_gdt:
+ /*
* We must switch to a new descriptor in kernel space for the GDT
* because soon the kernel won't have access anymore to the userspace
* addresses where we're currently running on. We have to do that here
@@ -435,6 +508,8 @@ SYM_DATA(initial_code, .quad x86_64_star
#ifdef CONFIG_AMD_MEM_ENCRYPT
SYM_DATA(initial_vc_handler, .quad handle_vc_boot_ghcb)
#endif
+
+SYM_DATA(trampoline_lock, .quad 0);
__FINITDATA
__INIT
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -985,10 +985,7 @@ int common_cpu_up(unsigned int cpu, stru
if (ret)
return ret;
-#ifdef CONFIG_X86_32
- /* Stack for startup_32 can be just as for start_secondary onwards */
per_cpu(pcpu_hot.top_of_stack, cpu) = task_top_of_stack(idle);
-#endif
return 0;
}
@@ -1014,7 +1011,7 @@ static int do_boot_cpu(int apicid, int c
if (IS_ENABLED(CONFIG_X86_32)) {
early_gdt_descr.address = (unsigned long)get_cpu_gdt_rw(cpu);
initial_stack = idle->thread.sp;
- } else {
+ } else if (!(smpboot_control & STARTUP_PARALLEL_MASK)) {
smpboot_control = cpu;
}
--- a/arch/x86/realmode/init.c
+++ b/arch/x86/realmode/init.c
@@ -154,6 +154,9 @@ static void __init setup_real_mode(void)
trampoline_header->flags = 0;
+ trampoline_lock = &trampoline_header->lock;
+ *trampoline_lock = 0;
+
trampoline_pgd = (u64 *) __va(real_mode_header->trampoline_pgd);
/* Map the real mode stub as virtual == physical */
--- a/arch/x86/realmode/rm/trampoline_64.S
+++ b/arch/x86/realmode/rm/trampoline_64.S
@@ -37,6 +37,24 @@
.text
.code16
+.macro LOAD_REALMODE_ESP
+ /*
+ * Make sure only one CPU fiddles with the realmode stack
+ */
+.Llock_rm\@:
+ btl $0, tr_lock
+ jnc 2f
+ pause
+ jmp .Llock_rm\@
+2:
+ lock
+ btsl $0, tr_lock
+ jc .Llock_rm\@
+
+ # Setup stack
+ movl $rm_stack_end, %esp
+.endm
+
.balign PAGE_SIZE
SYM_CODE_START(trampoline_start)
cli # We should be safe anyway
@@ -49,8 +67,7 @@ SYM_CODE_START(trampoline_start)
mov %ax, %es
mov %ax, %ss
- # Setup stack
- movl $rm_stack_end, %esp
+ LOAD_REALMODE_ESP
call verify_cpu # Verify the cpu supports long mode
testl %eax, %eax # Check for return code
@@ -93,8 +110,7 @@ SYM_CODE_START(sev_es_trampoline_start)
mov %ax, %es
mov %ax, %ss
- # Setup stack
- movl $rm_stack_end, %esp
+ LOAD_REALMODE_ESP
jmp .Lswitch_to_protected
SYM_CODE_END(sev_es_trampoline_start)
@@ -177,7 +193,7 @@ SYM_CODE_START(pa_trampoline_compat)
* In compatibility mode. Prep ESP and DX for startup_32, then disable
* paging and complete the switch to legacy 32-bit mode.
*/
- movl $rm_stack_end, %esp
+ LOAD_REALMODE_ESP
movw $__KERNEL_DS, %dx
movl $(CR0_STATE & ~X86_CR0_PG), %eax
@@ -241,6 +257,7 @@ SYM_DATA_START(trampoline_header)
SYM_DATA(tr_efer, .space 8)
SYM_DATA(tr_cr4, .space 4)
SYM_DATA(tr_flags, .space 4)
+ SYM_DATA(tr_lock, .space 4)
SYM_DATA_END(trampoline_header)
#include "trampoline_common.S"
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 36/37] x86/smpboot/64: Implement arch_cpuhp_init_parallel_bringup() and enable it
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-14 23:45 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:45 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Implement the validation function which tells the core code whether
parallel bringup is possible:
1) Valid CPUID leaf for APIC ID retrieval. For non x2APIC systmms leaf
0x1 is sufficient, otherwise leaf 0xb or 0x1f must be available.
2) Prevent parallel bringup on encrypted guests as this requires a
different handling of the CPUID leaf retrieval via a call into the
trusted firmware module. This is what the #VC trap handler does later
on, which is not available during the very early startup.
Originally-by: David Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/Kconfig | 3 +-
arch/x86/kernel/cpu/common.c | 6 -----
arch/x86/kernel/smpboot.c | 49 +++++++++++++++++++++++++++++++++++++++++--
3 files changed, 50 insertions(+), 8 deletions(-)
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -272,8 +272,9 @@ config X86
select HAVE_UNSTABLE_SCHED_CLOCK
select HAVE_USER_RETURN_NOTIFIER
select HAVE_GENERIC_VDSO
+ select HOTPLUG_PARALLEL if SMP && X86_64
select HOTPLUG_SMT if SMP
- select HOTPLUG_SPLIT_STARTUP if SMP
+ select HOTPLUG_SPLIT_STARTUP if SMP && X86_32
select IRQ_FORCED_THREADING
select NEED_PER_CPU_EMBED_FIRST_CHUNK
select NEED_PER_CPU_PAGE_FIRST_CHUNK
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -2127,11 +2127,7 @@ static inline void setup_getcpu(int cpu)
}
#ifdef CONFIG_X86_64
-static inline void ucode_cpu_init(int cpu)
-{
- if (cpu)
- load_ucode_ap();
-}
+static inline void ucode_cpu_init(int cpu) { }
static inline void tss_setup_ist(struct tss_struct *tss)
{
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -58,6 +58,7 @@
#include <linux/overflow.h>
#include <linux/stackprotector.h>
#include <linux/cpuhotplug.h>
+#include <linux/mc146818rtc.h>
#include <asm/acpi.h>
#include <asm/cacheinfo.h>
@@ -75,7 +76,7 @@
#include <asm/fpu/api.h>
#include <asm/setup.h>
#include <asm/uv/uv.h>
-#include <linux/mc146818rtc.h>
+#include <asm/microcode.h>
#include <asm/i8259.h>
#include <asm/misc.h>
#include <asm/qspinlock.h>
@@ -128,7 +129,6 @@ int arch_update_cpu_topology(void)
return retval;
}
-
static unsigned int smpboot_warm_reset_vector_count;
static inline void smpboot_setup_warm_reset_vector(unsigned long start_eip)
@@ -247,6 +247,8 @@ static void notrace start_secondary(void
#endif
cpu_init_exception_handling();
+ load_ucode_ap();
+
/*
* Sync point with the hotplug core. Sets the sync state to ALIVE
* and waits for the control CPU to release it.
@@ -1251,6 +1253,49 @@ void __init smp_prepare_cpus_common(void
set_cpu_sibling_map(0);
}
+#ifdef CONFIG_X86_64
+/* Establish whether parallel bringup can be supported. */
+bool __init arch_cpuhp_init_parallel_bringup(void)
+{
+ unsigned int ctrl;
+
+ if (boot_cpu_data.cpuid_level < 0x01) {
+ pr_info("Parallel CPU startup disabled due to lack of CPUID\n");
+ return false;
+ }
+
+ /* Encrypted guests require special CPUID handling. */
+ if (cc_platform_has(CC_ATTR_GUEST_STATE_ENCRYPT)) {
+ pr_info("Parallel CPU startup disabled due to guest state encryption\n");
+ return false;
+ }
+
+ switch (topology_extended_leaf) {
+ case 0x0b:
+ ctrl = STARTUP_APICID_CPUID_0B;
+ break;
+ case 0x1f:
+ ctrl = STARTUP_APICID_CPUID_1F;
+ break;
+ case 0x00:
+ /* For !x2APIC mode 8 bits from leaf 0x01 are sufficient. */
+ if (!x2apic_mode) {
+ ctrl = STARTUP_APICID_CPUID_01;
+ break;
+ }
+ fallthrough;
+ default:
+ pr_info("Parallel CPU startup disabled. Unsupported topology leaf %u\n",
+ topology_extended_leaf);
+ return false;
+ }
+
+ pr_debug("Parallel CPU startup enabled: 0x%08x\n", ctrl);
+ smpboot_control = ctrl;
+ return true;
+}
+#endif
+
/*
* Prepare for SMP bootup.
* @max_cpus: configured maximum number of CPUs, It is a legacy parameter
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 36/37] x86/smpboot/64: Implement arch_cpuhp_init_parallel_bringup() and enable it
@ 2023-04-14 23:45 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:45 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Implement the validation function which tells the core code whether
parallel bringup is possible:
1) Valid CPUID leaf for APIC ID retrieval. For non x2APIC systmms leaf
0x1 is sufficient, otherwise leaf 0xb or 0x1f must be available.
2) Prevent parallel bringup on encrypted guests as this requires a
different handling of the CPUID leaf retrieval via a call into the
trusted firmware module. This is what the #VC trap handler does later
on, which is not available during the very early startup.
Originally-by: David Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/Kconfig | 3 +-
arch/x86/kernel/cpu/common.c | 6 -----
arch/x86/kernel/smpboot.c | 49 +++++++++++++++++++++++++++++++++++++++++--
3 files changed, 50 insertions(+), 8 deletions(-)
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -272,8 +272,9 @@ config X86
select HAVE_UNSTABLE_SCHED_CLOCK
select HAVE_USER_RETURN_NOTIFIER
select HAVE_GENERIC_VDSO
+ select HOTPLUG_PARALLEL if SMP && X86_64
select HOTPLUG_SMT if SMP
- select HOTPLUG_SPLIT_STARTUP if SMP
+ select HOTPLUG_SPLIT_STARTUP if SMP && X86_32
select IRQ_FORCED_THREADING
select NEED_PER_CPU_EMBED_FIRST_CHUNK
select NEED_PER_CPU_PAGE_FIRST_CHUNK
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -2127,11 +2127,7 @@ static inline void setup_getcpu(int cpu)
}
#ifdef CONFIG_X86_64
-static inline void ucode_cpu_init(int cpu)
-{
- if (cpu)
- load_ucode_ap();
-}
+static inline void ucode_cpu_init(int cpu) { }
static inline void tss_setup_ist(struct tss_struct *tss)
{
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -58,6 +58,7 @@
#include <linux/overflow.h>
#include <linux/stackprotector.h>
#include <linux/cpuhotplug.h>
+#include <linux/mc146818rtc.h>
#include <asm/acpi.h>
#include <asm/cacheinfo.h>
@@ -75,7 +76,7 @@
#include <asm/fpu/api.h>
#include <asm/setup.h>
#include <asm/uv/uv.h>
-#include <linux/mc146818rtc.h>
+#include <asm/microcode.h>
#include <asm/i8259.h>
#include <asm/misc.h>
#include <asm/qspinlock.h>
@@ -128,7 +129,6 @@ int arch_update_cpu_topology(void)
return retval;
}
-
static unsigned int smpboot_warm_reset_vector_count;
static inline void smpboot_setup_warm_reset_vector(unsigned long start_eip)
@@ -247,6 +247,8 @@ static void notrace start_secondary(void
#endif
cpu_init_exception_handling();
+ load_ucode_ap();
+
/*
* Sync point with the hotplug core. Sets the sync state to ALIVE
* and waits for the control CPU to release it.
@@ -1251,6 +1253,49 @@ void __init smp_prepare_cpus_common(void
set_cpu_sibling_map(0);
}
+#ifdef CONFIG_X86_64
+/* Establish whether parallel bringup can be supported. */
+bool __init arch_cpuhp_init_parallel_bringup(void)
+{
+ unsigned int ctrl;
+
+ if (boot_cpu_data.cpuid_level < 0x01) {
+ pr_info("Parallel CPU startup disabled due to lack of CPUID\n");
+ return false;
+ }
+
+ /* Encrypted guests require special CPUID handling. */
+ if (cc_platform_has(CC_ATTR_GUEST_STATE_ENCRYPT)) {
+ pr_info("Parallel CPU startup disabled due to guest state encryption\n");
+ return false;
+ }
+
+ switch (topology_extended_leaf) {
+ case 0x0b:
+ ctrl = STARTUP_APICID_CPUID_0B;
+ break;
+ case 0x1f:
+ ctrl = STARTUP_APICID_CPUID_1F;
+ break;
+ case 0x00:
+ /* For !x2APIC mode 8 bits from leaf 0x01 are sufficient. */
+ if (!x2apic_mode) {
+ ctrl = STARTUP_APICID_CPUID_01;
+ break;
+ }
+ fallthrough;
+ default:
+ pr_info("Parallel CPU startup disabled. Unsupported topology leaf %u\n",
+ topology_extended_leaf);
+ return false;
+ }
+
+ pr_debug("Parallel CPU startup enabled: 0x%08x\n", ctrl);
+ smpboot_control = ctrl;
+ return true;
+}
+#endif
+
/*
* Prepare for SMP bootup.
* @max_cpus: configured maximum number of CPUs, It is a legacy parameter
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 36/37] x86/smpboot/64: Implement arch_cpuhp_init_parallel_bringup() and enable it
@ 2023-04-14 23:45 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:45 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Implement the validation function which tells the core code whether
parallel bringup is possible:
1) Valid CPUID leaf for APIC ID retrieval. For non x2APIC systmms leaf
0x1 is sufficient, otherwise leaf 0xb or 0x1f must be available.
2) Prevent parallel bringup on encrypted guests as this requires a
different handling of the CPUID leaf retrieval via a call into the
trusted firmware module. This is what the #VC trap handler does later
on, which is not available during the very early startup.
Originally-by: David Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
---
arch/x86/Kconfig | 3 +-
arch/x86/kernel/cpu/common.c | 6 -----
arch/x86/kernel/smpboot.c | 49 +++++++++++++++++++++++++++++++++++++++++--
3 files changed, 50 insertions(+), 8 deletions(-)
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -272,8 +272,9 @@ config X86
select HAVE_UNSTABLE_SCHED_CLOCK
select HAVE_USER_RETURN_NOTIFIER
select HAVE_GENERIC_VDSO
+ select HOTPLUG_PARALLEL if SMP && X86_64
select HOTPLUG_SMT if SMP
- select HOTPLUG_SPLIT_STARTUP if SMP
+ select HOTPLUG_SPLIT_STARTUP if SMP && X86_32
select IRQ_FORCED_THREADING
select NEED_PER_CPU_EMBED_FIRST_CHUNK
select NEED_PER_CPU_PAGE_FIRST_CHUNK
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -2127,11 +2127,7 @@ static inline void setup_getcpu(int cpu)
}
#ifdef CONFIG_X86_64
-static inline void ucode_cpu_init(int cpu)
-{
- if (cpu)
- load_ucode_ap();
-}
+static inline void ucode_cpu_init(int cpu) { }
static inline void tss_setup_ist(struct tss_struct *tss)
{
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -58,6 +58,7 @@
#include <linux/overflow.h>
#include <linux/stackprotector.h>
#include <linux/cpuhotplug.h>
+#include <linux/mc146818rtc.h>
#include <asm/acpi.h>
#include <asm/cacheinfo.h>
@@ -75,7 +76,7 @@
#include <asm/fpu/api.h>
#include <asm/setup.h>
#include <asm/uv/uv.h>
-#include <linux/mc146818rtc.h>
+#include <asm/microcode.h>
#include <asm/i8259.h>
#include <asm/misc.h>
#include <asm/qspinlock.h>
@@ -128,7 +129,6 @@ int arch_update_cpu_topology(void)
return retval;
}
-
static unsigned int smpboot_warm_reset_vector_count;
static inline void smpboot_setup_warm_reset_vector(unsigned long start_eip)
@@ -247,6 +247,8 @@ static void notrace start_secondary(void
#endif
cpu_init_exception_handling();
+ load_ucode_ap();
+
/*
* Sync point with the hotplug core. Sets the sync state to ALIVE
* and waits for the control CPU to release it.
@@ -1251,6 +1253,49 @@ void __init smp_prepare_cpus_common(void
set_cpu_sibling_map(0);
}
+#ifdef CONFIG_X86_64
+/* Establish whether parallel bringup can be supported. */
+bool __init arch_cpuhp_init_parallel_bringup(void)
+{
+ unsigned int ctrl;
+
+ if (boot_cpu_data.cpuid_level < 0x01) {
+ pr_info("Parallel CPU startup disabled due to lack of CPUID\n");
+ return false;
+ }
+
+ /* Encrypted guests require special CPUID handling. */
+ if (cc_platform_has(CC_ATTR_GUEST_STATE_ENCRYPT)) {
+ pr_info("Parallel CPU startup disabled due to guest state encryption\n");
+ return false;
+ }
+
+ switch (topology_extended_leaf) {
+ case 0x0b:
+ ctrl = STARTUP_APICID_CPUID_0B;
+ break;
+ case 0x1f:
+ ctrl = STARTUP_APICID_CPUID_1F;
+ break;
+ case 0x00:
+ /* For !x2APIC mode 8 bits from leaf 0x01 are sufficient. */
+ if (!x2apic_mode) {
+ ctrl = STARTUP_APICID_CPUID_01;
+ break;
+ }
+ fallthrough;
+ default:
+ pr_info("Parallel CPU startup disabled. Unsupported topology leaf %u\n",
+ topology_extended_leaf);
+ return false;
+ }
+
+ pr_debug("Parallel CPU startup enabled: 0x%08x\n", ctrl);
+ smpboot_control = ctrl;
+ return true;
+}
+#endif
+
/*
* Prepare for SMP bootup.
* @max_cpus: configured maximum number of CPUs, It is a legacy parameter
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 37/37] x86/smpboot: Allow parallel bringup for SEV-ES
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-14 23:45 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:45 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Sabin Rapan, David Woodhouse,
Usama Arif, Juergen Gross, Boris Ostrovsky, xen-devel,
Russell King, Arnd Bergmann, linux-arm-kernel, Catalin Marinas,
Will Deacon, Guo Ren, linux-csky, Thomas Bogendoerfer,
linux-mips, James E.J. Bottomley, Helge Deller, linux-parisc,
Paul Walmsley, Palmer Dabbelt, linux-riscv, Mark Rutland
From: David Woodhouse <dwmw@amazon.co.uk>
Enable parallel bringup for SEV-ES guests. The APs can't actually execute
the CPUID instruction directly during early startup, but they can make the
GHCB call directly instead, just as the #VC trap handler would do.
Thanks to Sabin for talking me through the way this works.
Suggested-by: Sabin Rapan <sabrapan@amazon.com>
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: Usama Arif <usama.arif@bytedance.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
---
arch/x86/include/asm/sev-common.h | 3 +++
arch/x86/include/asm/smp.h | 1 +
arch/x86/kernel/head_64.S | 30 ++++++++++++++++++++++++++++++
arch/x86/kernel/smpboot.c | 14 ++++++++++++--
4 files changed, 46 insertions(+), 2 deletions(-)
--- a/arch/x86/include/asm/sev-common.h
+++ b/arch/x86/include/asm/sev-common.h
@@ -70,6 +70,7 @@
/* GHCBData[63:12] */ \
(((u64)(v) & GENMASK_ULL(63, 12)) >> 12)
+#ifndef __ASSEMBLY__
/*
* SNP Page State Change Operation
*
@@ -161,6 +162,8 @@ struct snp_psc_desc {
#define GHCB_RESP_CODE(v) ((v) & GHCB_MSR_INFO_MASK)
+#endif /* __ASSEMBLY__ */
+
/*
* Error codes related to GHCB input that can be communicated back to the guest
* by setting the lower 32-bits of the GHCB SW_EXITINFO1 field to 2.
--- a/arch/x86/include/asm/smp.h
+++ b/arch/x86/include/asm/smp.h
@@ -202,6 +202,7 @@ extern unsigned int smpboot_control;
#define STARTUP_APICID_CPUID_1F 0x80000000
#define STARTUP_APICID_CPUID_0B 0x40000000
#define STARTUP_APICID_CPUID_01 0x20000000
+#define STARTUP_APICID_SEV_ES 0x10000000
/* Top 8 bits are reserved for control */
#define STARTUP_PARALLEL_MASK 0xFF000000
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -26,6 +26,7 @@
#include <asm/nospec-branch.h>
#include <asm/fixmap.h>
#include <asm/smp.h>
+#include <asm/sev-common.h>
/*
* We are not able to switch in one step to the final KERNEL ADDRESS SPACE
@@ -243,9 +244,14 @@ SYM_INNER_LABEL(secondary_startup_64_no_
* Bit 31 STARTUP_APICID_CPUID_1F flag (use CPUID 0x1f)
* Bit 30 STARTUP_APICID_CPUID_0B flag (use CPUID 0x0b)
* Bit 29 STARTUP_APICID_CPUID_01 flag (use CPUID 0x01)
+ * Bit 28 STARTUP_APICID_SEV_ES flag (CPUID 0x0b via GHCB MSR)
* Bit 0-23 CPU# if STARTUP_APICID_CPUID_xx flags are not set
*/
movl smpboot_control(%rip), %ecx
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+ testl $STARTUP_APICID_SEV_ES, %ecx
+ jnz .Luse_sev_cpuid_0b
+#endif
testl $STARTUP_APICID_CPUID_1F, %ecx
jnz .Luse_cpuid_1f
testl $STARTUP_APICID_CPUID_0B, %ecx
@@ -262,6 +268,30 @@ SYM_INNER_LABEL(secondary_startup_64_no_
shr $24, %edx
jmp .Lsetup_AP
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+.Luse_sev_cpuid_0b:
+ /* Set the GHCB MSR to request CPUID 0x0B_EDX */
+ movl $MSR_AMD64_SEV_ES_GHCB, %ecx
+ movl $(GHCB_CPUID_REQ_EDX << 30) | GHCB_MSR_CPUID_REQ, %eax
+ movl $0x0b, %edx
+ wrmsr
+
+ /* Perform GHCB MSR protocol */
+ rep; vmmcall /* vmgexit */
+
+ /*
+ * Get the result. After the RDMSR:
+ * EAX should be 0xc0000005
+ * EDX should have the CPUID register value and since EDX
+ * is the target register, no need to move the result.
+ */
+ rdmsr
+ andl $GHCB_MSR_INFO_MASK, %eax
+ cmpl $GHCB_MSR_CPUID_RESP, %eax
+ jne 1f
+ jmp .Lsetup_AP
+#endif
+
.Luse_cpuid_0b:
mov $0x0B, %eax
xorl %ecx, %ecx
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -86,6 +86,7 @@
#include <asm/hw_irq.h>
#include <asm/stackprotector.h>
#include <asm/sev.h>
+#include <asm/coco.h>
/* representing HT siblings of each logical CPU */
DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_sibling_map);
@@ -1266,8 +1267,16 @@ bool __init arch_cpuhp_init_parallel_bri
/* Encrypted guests require special CPUID handling. */
if (cc_platform_has(CC_ATTR_GUEST_STATE_ENCRYPT)) {
- pr_info("Parallel CPU startup disabled due to guest state encryption\n");
- return false;
+ switch (cc_get_vendor()) {
+ case CC_VENDOR_AMD:
+ ctrl = STARTUP_APICID_SEV_ES;
+ if (topology_extended_leaf == 0x0b)
+ goto setup;
+ fallthrough;
+ default:
+ pr_info("Parallel CPU startup disabled due to guest state encryption\n");
+ return false;
+ }
}
switch (topology_extended_leaf) {
@@ -1290,6 +1299,7 @@ bool __init arch_cpuhp_init_parallel_bri
return false;
}
+setup:
pr_debug("Parallel CPU startup enabled: 0x%08x\n", ctrl);
smpboot_control = ctrl;
return true;
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 37/37] x86/smpboot: Allow parallel bringup for SEV-ES
@ 2023-04-14 23:45 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:45 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Sabin Rapan, David Woodhouse,
Usama Arif, Juergen Gross, Boris Ostrovsky, xen-devel,
Russell King, Arnd Bergmann, linux-arm-kernel, Catalin Marinas,
Will Deacon, Guo Ren, linux-csky, Thomas Bogendoerfer,
linux-mips, James E.J. Bottomley, Helge Deller, linux-parisc,
Paul Walmsley, Palmer Dabbelt, linux-riscv, Mark Rutland
From: David Woodhouse <dwmw@amazon.co.uk>
Enable parallel bringup for SEV-ES guests. The APs can't actually execute
the CPUID instruction directly during early startup, but they can make the
GHCB call directly instead, just as the #VC trap handler would do.
Thanks to Sabin for talking me through the way this works.
Suggested-by: Sabin Rapan <sabrapan@amazon.com>
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: Usama Arif <usama.arif@bytedance.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
---
arch/x86/include/asm/sev-common.h | 3 +++
arch/x86/include/asm/smp.h | 1 +
arch/x86/kernel/head_64.S | 30 ++++++++++++++++++++++++++++++
arch/x86/kernel/smpboot.c | 14 ++++++++++++--
4 files changed, 46 insertions(+), 2 deletions(-)
--- a/arch/x86/include/asm/sev-common.h
+++ b/arch/x86/include/asm/sev-common.h
@@ -70,6 +70,7 @@
/* GHCBData[63:12] */ \
(((u64)(v) & GENMASK_ULL(63, 12)) >> 12)
+#ifndef __ASSEMBLY__
/*
* SNP Page State Change Operation
*
@@ -161,6 +162,8 @@ struct snp_psc_desc {
#define GHCB_RESP_CODE(v) ((v) & GHCB_MSR_INFO_MASK)
+#endif /* __ASSEMBLY__ */
+
/*
* Error codes related to GHCB input that can be communicated back to the guest
* by setting the lower 32-bits of the GHCB SW_EXITINFO1 field to 2.
--- a/arch/x86/include/asm/smp.h
+++ b/arch/x86/include/asm/smp.h
@@ -202,6 +202,7 @@ extern unsigned int smpboot_control;
#define STARTUP_APICID_CPUID_1F 0x80000000
#define STARTUP_APICID_CPUID_0B 0x40000000
#define STARTUP_APICID_CPUID_01 0x20000000
+#define STARTUP_APICID_SEV_ES 0x10000000
/* Top 8 bits are reserved for control */
#define STARTUP_PARALLEL_MASK 0xFF000000
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -26,6 +26,7 @@
#include <asm/nospec-branch.h>
#include <asm/fixmap.h>
#include <asm/smp.h>
+#include <asm/sev-common.h>
/*
* We are not able to switch in one step to the final KERNEL ADDRESS SPACE
@@ -243,9 +244,14 @@ SYM_INNER_LABEL(secondary_startup_64_no_
* Bit 31 STARTUP_APICID_CPUID_1F flag (use CPUID 0x1f)
* Bit 30 STARTUP_APICID_CPUID_0B flag (use CPUID 0x0b)
* Bit 29 STARTUP_APICID_CPUID_01 flag (use CPUID 0x01)
+ * Bit 28 STARTUP_APICID_SEV_ES flag (CPUID 0x0b via GHCB MSR)
* Bit 0-23 CPU# if STARTUP_APICID_CPUID_xx flags are not set
*/
movl smpboot_control(%rip), %ecx
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+ testl $STARTUP_APICID_SEV_ES, %ecx
+ jnz .Luse_sev_cpuid_0b
+#endif
testl $STARTUP_APICID_CPUID_1F, %ecx
jnz .Luse_cpuid_1f
testl $STARTUP_APICID_CPUID_0B, %ecx
@@ -262,6 +268,30 @@ SYM_INNER_LABEL(secondary_startup_64_no_
shr $24, %edx
jmp .Lsetup_AP
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+.Luse_sev_cpuid_0b:
+ /* Set the GHCB MSR to request CPUID 0x0B_EDX */
+ movl $MSR_AMD64_SEV_ES_GHCB, %ecx
+ movl $(GHCB_CPUID_REQ_EDX << 30) | GHCB_MSR_CPUID_REQ, %eax
+ movl $0x0b, %edx
+ wrmsr
+
+ /* Perform GHCB MSR protocol */
+ rep; vmmcall /* vmgexit */
+
+ /*
+ * Get the result. After the RDMSR:
+ * EAX should be 0xc0000005
+ * EDX should have the CPUID register value and since EDX
+ * is the target register, no need to move the result.
+ */
+ rdmsr
+ andl $GHCB_MSR_INFO_MASK, %eax
+ cmpl $GHCB_MSR_CPUID_RESP, %eax
+ jne 1f
+ jmp .Lsetup_AP
+#endif
+
.Luse_cpuid_0b:
mov $0x0B, %eax
xorl %ecx, %ecx
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -86,6 +86,7 @@
#include <asm/hw_irq.h>
#include <asm/stackprotector.h>
#include <asm/sev.h>
+#include <asm/coco.h>
/* representing HT siblings of each logical CPU */
DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_sibling_map);
@@ -1266,8 +1267,16 @@ bool __init arch_cpuhp_init_parallel_bri
/* Encrypted guests require special CPUID handling. */
if (cc_platform_has(CC_ATTR_GUEST_STATE_ENCRYPT)) {
- pr_info("Parallel CPU startup disabled due to guest state encryption\n");
- return false;
+ switch (cc_get_vendor()) {
+ case CC_VENDOR_AMD:
+ ctrl = STARTUP_APICID_SEV_ES;
+ if (topology_extended_leaf == 0x0b)
+ goto setup;
+ fallthrough;
+ default:
+ pr_info("Parallel CPU startup disabled due to guest state encryption\n");
+ return false;
+ }
}
switch (topology_extended_leaf) {
@@ -1290,6 +1299,7 @@ bool __init arch_cpuhp_init_parallel_bri
return false;
}
+setup:
pr_debug("Parallel CPU startup enabled: 0x%08x\n", ctrl);
smpboot_control = ctrl;
return true;
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* [patch 37/37] x86/smpboot: Allow parallel bringup for SEV-ES
@ 2023-04-14 23:45 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-14 23:45 UTC (permalink / raw)
To: LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Sabin Rapan, David Woodhouse,
Usama Arif, Juergen Gross, Boris Ostrovsky, xen-devel,
Russell King, Arnd Bergmann, linux-arm-kernel, Catalin Marinas,
Will Deacon, Guo Ren, linux-csky, Thomas Bogendoerfer,
linux-mips, James E.J. Bottomley, Helge Deller, linux-parisc,
Paul Walmsley, Palmer Dabbelt, linux-riscv, Mark Rutland
From: David Woodhouse <dwmw@amazon.co.uk>
Enable parallel bringup for SEV-ES guests. The APs can't actually execute
the CPUID instruction directly during early startup, but they can make the
GHCB call directly instead, just as the #VC trap handler would do.
Thanks to Sabin for talking me through the way this works.
Suggested-by: Sabin Rapan <sabrapan@amazon.com>
Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
Signed-off-by: Usama Arif <usama.arif@bytedance.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
---
arch/x86/include/asm/sev-common.h | 3 +++
arch/x86/include/asm/smp.h | 1 +
arch/x86/kernel/head_64.S | 30 ++++++++++++++++++++++++++++++
arch/x86/kernel/smpboot.c | 14 ++++++++++++--
4 files changed, 46 insertions(+), 2 deletions(-)
--- a/arch/x86/include/asm/sev-common.h
+++ b/arch/x86/include/asm/sev-common.h
@@ -70,6 +70,7 @@
/* GHCBData[63:12] */ \
(((u64)(v) & GENMASK_ULL(63, 12)) >> 12)
+#ifndef __ASSEMBLY__
/*
* SNP Page State Change Operation
*
@@ -161,6 +162,8 @@ struct snp_psc_desc {
#define GHCB_RESP_CODE(v) ((v) & GHCB_MSR_INFO_MASK)
+#endif /* __ASSEMBLY__ */
+
/*
* Error codes related to GHCB input that can be communicated back to the guest
* by setting the lower 32-bits of the GHCB SW_EXITINFO1 field to 2.
--- a/arch/x86/include/asm/smp.h
+++ b/arch/x86/include/asm/smp.h
@@ -202,6 +202,7 @@ extern unsigned int smpboot_control;
#define STARTUP_APICID_CPUID_1F 0x80000000
#define STARTUP_APICID_CPUID_0B 0x40000000
#define STARTUP_APICID_CPUID_01 0x20000000
+#define STARTUP_APICID_SEV_ES 0x10000000
/* Top 8 bits are reserved for control */
#define STARTUP_PARALLEL_MASK 0xFF000000
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -26,6 +26,7 @@
#include <asm/nospec-branch.h>
#include <asm/fixmap.h>
#include <asm/smp.h>
+#include <asm/sev-common.h>
/*
* We are not able to switch in one step to the final KERNEL ADDRESS SPACE
@@ -243,9 +244,14 @@ SYM_INNER_LABEL(secondary_startup_64_no_
* Bit 31 STARTUP_APICID_CPUID_1F flag (use CPUID 0x1f)
* Bit 30 STARTUP_APICID_CPUID_0B flag (use CPUID 0x0b)
* Bit 29 STARTUP_APICID_CPUID_01 flag (use CPUID 0x01)
+ * Bit 28 STARTUP_APICID_SEV_ES flag (CPUID 0x0b via GHCB MSR)
* Bit 0-23 CPU# if STARTUP_APICID_CPUID_xx flags are not set
*/
movl smpboot_control(%rip), %ecx
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+ testl $STARTUP_APICID_SEV_ES, %ecx
+ jnz .Luse_sev_cpuid_0b
+#endif
testl $STARTUP_APICID_CPUID_1F, %ecx
jnz .Luse_cpuid_1f
testl $STARTUP_APICID_CPUID_0B, %ecx
@@ -262,6 +268,30 @@ SYM_INNER_LABEL(secondary_startup_64_no_
shr $24, %edx
jmp .Lsetup_AP
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+.Luse_sev_cpuid_0b:
+ /* Set the GHCB MSR to request CPUID 0x0B_EDX */
+ movl $MSR_AMD64_SEV_ES_GHCB, %ecx
+ movl $(GHCB_CPUID_REQ_EDX << 30) | GHCB_MSR_CPUID_REQ, %eax
+ movl $0x0b, %edx
+ wrmsr
+
+ /* Perform GHCB MSR protocol */
+ rep; vmmcall /* vmgexit */
+
+ /*
+ * Get the result. After the RDMSR:
+ * EAX should be 0xc0000005
+ * EDX should have the CPUID register value and since EDX
+ * is the target register, no need to move the result.
+ */
+ rdmsr
+ andl $GHCB_MSR_INFO_MASK, %eax
+ cmpl $GHCB_MSR_CPUID_RESP, %eax
+ jne 1f
+ jmp .Lsetup_AP
+#endif
+
.Luse_cpuid_0b:
mov $0x0B, %eax
xorl %ecx, %ecx
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -86,6 +86,7 @@
#include <asm/hw_irq.h>
#include <asm/stackprotector.h>
#include <asm/sev.h>
+#include <asm/coco.h>
/* representing HT siblings of each logical CPU */
DEFINE_PER_CPU_READ_MOSTLY(cpumask_var_t, cpu_sibling_map);
@@ -1266,8 +1267,16 @@ bool __init arch_cpuhp_init_parallel_bri
/* Encrypted guests require special CPUID handling. */
if (cc_platform_has(CC_ATTR_GUEST_STATE_ENCRYPT)) {
- pr_info("Parallel CPU startup disabled due to guest state encryption\n");
- return false;
+ switch (cc_get_vendor()) {
+ case CC_VENDOR_AMD:
+ ctrl = STARTUP_APICID_SEV_ES;
+ if (topology_extended_leaf == 0x0b)
+ goto setup;
+ fallthrough;
+ default:
+ pr_info("Parallel CPU startup disabled due to guest state encryption\n");
+ return false;
+ }
}
switch (topology_extended_leaf) {
@@ -1290,6 +1299,7 @@ bool __init arch_cpuhp_init_parallel_bri
return false;
}
+setup:
pr_debug("Parallel CPU startup enabled: 0x%08x\n", ctrl);
smpboot_control = ctrl;
return true;
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 19/37] x86/smpboot: Switch to hotplug core state synchronization
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-15 12:58 ` Brian Gerst
-1 siblings, 0 replies; 236+ messages in thread
From: Brian Gerst @ 2023-04-15 12:58 UTC (permalink / raw)
To: Thomas Gleixner
Cc: LKML, x86, David Woodhouse, Andrew Cooper, Arjan van de Veen,
Paolo Bonzini, Paul McKenney, Tom Lendacky, Sean Christopherson,
Oleksandr Natalenko, Paul Menzel, Guilherme G. Piccoli,
Piotr Gorski, Juergen Gross, Boris Ostrovsky, xen-devel,
David Woodhouse, Usama Arif, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Fri, Apr 14, 2023 at 7:44 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>
> The new AP state tracking and synchronization mechanism in the CPU hotplug
> core code allows to remove quite some x86 specific code:
>
> 1) The AP alive synchronization based on cpumasks
>
> 2) The decision whether an AP can be brought up again
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Juergen Gross <jgross@suse.com>
> Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
> Cc: xen-devel@lists.xenproject.org
> ---
> arch/x86/Kconfig | 1
> arch/x86/include/asm/smp.h | 7 +
> arch/x86/kernel/smp.c | 1
> arch/x86/kernel/smpboot.c | 159 ++++++++++-----------------------------------
> arch/x86/xen/smp_hvm.c | 16 +---
> arch/x86/xen/smp_pv.c | 39 ++++++-----
> 6 files changed, 72 insertions(+), 151 deletions(-)
>
> --- a/arch/x86/Kconfig
> +++ b/arch/x86/Kconfig
> @@ -272,6 +272,7 @@ config X86
> select HAVE_UNSTABLE_SCHED_CLOCK
> select HAVE_USER_RETURN_NOTIFIER
> select HAVE_GENERIC_VDSO
> + select HOTPLUG_CORE_SYNC_FULL if SMP
> select HOTPLUG_SMT if SMP
> select IRQ_FORCED_THREADING
> select NEED_PER_CPU_EMBED_FIRST_CHUNK
> --- a/arch/x86/include/asm/smp.h
> +++ b/arch/x86/include/asm/smp.h
> @@ -38,6 +38,8 @@ struct smp_ops {
> void (*crash_stop_other_cpus)(void);
> void (*smp_send_reschedule)(int cpu);
>
> + void (*cleanup_dead_cpu)(unsigned cpu);
> + void (*poll_sync_state)(void);
> int (*cpu_up)(unsigned cpu, struct task_struct *tidle);
> int (*cpu_disable)(void);
> void (*cpu_die)(unsigned int cpu);
> @@ -90,7 +92,8 @@ static inline int __cpu_disable(void)
>
> static inline void __cpu_die(unsigned int cpu)
> {
> - smp_ops.cpu_die(cpu);
> + if (smp_ops.cpu_die)
> + smp_ops.cpu_die(cpu);
> }
>
> static inline void play_dead(void)
> @@ -122,8 +125,6 @@ void native_smp_cpus_done(unsigned int m
> int common_cpu_up(unsigned int cpunum, struct task_struct *tidle);
> int native_cpu_up(unsigned int cpunum, struct task_struct *tidle);
> int native_cpu_disable(void);
> -int common_cpu_die(unsigned int cpu);
> -void native_cpu_die(unsigned int cpu);
> void hlt_play_dead(void);
> void native_play_dead(void);
> void play_dead_common(void);
> --- a/arch/x86/kernel/smp.c
> +++ b/arch/x86/kernel/smp.c
> @@ -269,7 +269,6 @@ struct smp_ops smp_ops = {
> .smp_send_reschedule = native_smp_send_reschedule,
>
> .cpu_up = native_cpu_up,
> - .cpu_die = native_cpu_die,
> .cpu_disable = native_cpu_disable,
> .play_dead = native_play_dead,
>
> --- a/arch/x86/kernel/smpboot.c
> +++ b/arch/x86/kernel/smpboot.c
> @@ -57,6 +57,7 @@
> #include <linux/pgtable.h>
> #include <linux/overflow.h>
> #include <linux/stackprotector.h>
> +#include <linux/cpuhotplug.h>
>
> #include <asm/acpi.h>
> #include <asm/cacheinfo.h>
> @@ -101,9 +102,6 @@ EXPORT_PER_CPU_SYMBOL(cpu_die_map);
> DEFINE_PER_CPU_READ_MOSTLY(struct cpuinfo_x86, cpu_info);
> EXPORT_PER_CPU_SYMBOL(cpu_info);
>
> -/* All of these masks are initialized in setup_cpu_local_masks() */
> -static cpumask_var_t cpu_initialized_mask;
> -static cpumask_var_t cpu_callout_mask;
> /* Representing CPUs for which sibling maps can be computed */
> static cpumask_var_t cpu_sibling_setup_mask;
>
> @@ -169,8 +167,8 @@ static void smp_callin(void)
> int cpuid = smp_processor_id();
>
> /*
> - * If waken up by an INIT in an 82489DX configuration
> - * cpu_callout_mask guarantees we don't get here before an
> + * If waken up by an INIT in an 82489DX configuration the alive
> + * synchronization guarantees we don't get here before an
> * INIT_deassert IPI reaches our local APIC, so it is now safe to
> * touch our local APIC.
> *
> @@ -216,17 +214,6 @@ static void ap_calibrate_delay(void)
> cpu_data(smp_processor_id()).loops_per_jiffy = loops_per_jiffy;
> }
>
> -static void wait_for_master_cpu(int cpu)
> -{
> - /*
> - * Wait for release by control CPU before continuing with AP
> - * initialization.
> - */
> - WARN_ON(cpumask_test_and_set_cpu(cpu, cpu_initialized_mask));
> - while (!cpumask_test_cpu(cpu, cpu_callout_mask))
> - cpu_relax();
> -}
> -
> /*
> * Activate a secondary processor.
> */
> @@ -247,11 +234,10 @@ static void notrace start_secondary(void
> cpu_init_exception_handling();
>
> /*
> - * Sync point with wait_cpu_initialized(). Sets AP in
> - * cpu_initialized_mask and then waits for the control CPU
> - * to release it.
> + * Sync point with the hotplug core. Sets the sync state to ALIVE
> + * and waits for the control CPU to release it.
> */
> - wait_for_master_cpu(raw_smp_processor_id());
> + cpuhp_ap_sync_alive();
>
> cpu_init();
> rcu_cpu_starting(raw_smp_processor_id());
> @@ -285,7 +271,6 @@ static void notrace start_secondary(void
> set_cpu_online(smp_processor_id(), true);
> lapic_online();
> unlock_vector_lock();
> - cpu_set_state_online(smp_processor_id());
> x86_platform.nmi_init();
>
> /* enable local interrupts */
> @@ -736,9 +721,10 @@ static void impress_friends(void)
> * Allow the user to impress friends.
> */
> pr_debug("Before bogomips\n");
> - for_each_possible_cpu(cpu)
> - if (cpumask_test_cpu(cpu, cpu_callout_mask))
> + for_each_possible_cpu(cpu) {
> + if (cpumask_test_cpu(cpu, cpu_online_mask))
> bogosum += cpu_data(cpu).loops_per_jiffy;
This should be the same as for_each_online_cpu().
--
Brian Gerst
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 19/37] x86/smpboot: Switch to hotplug core state synchronization
@ 2023-04-15 12:58 ` Brian Gerst
0 siblings, 0 replies; 236+ messages in thread
From: Brian Gerst @ 2023-04-15 12:58 UTC (permalink / raw)
To: Thomas Gleixner
Cc: LKML, x86, David Woodhouse, Andrew Cooper, Arjan van de Veen,
Paolo Bonzini, Paul McKenney, Tom Lendacky, Sean Christopherson,
Oleksandr Natalenko, Paul Menzel, Guilherme G. Piccoli,
Piotr Gorski, Juergen Gross, Boris Ostrovsky, xen-devel,
David Woodhouse, Usama Arif, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Fri, Apr 14, 2023 at 7:44 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>
> The new AP state tracking and synchronization mechanism in the CPU hotplug
> core code allows to remove quite some x86 specific code:
>
> 1) The AP alive synchronization based on cpumasks
>
> 2) The decision whether an AP can be brought up again
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Juergen Gross <jgross@suse.com>
> Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
> Cc: xen-devel@lists.xenproject.org
> ---
> arch/x86/Kconfig | 1
> arch/x86/include/asm/smp.h | 7 +
> arch/x86/kernel/smp.c | 1
> arch/x86/kernel/smpboot.c | 159 ++++++++++-----------------------------------
> arch/x86/xen/smp_hvm.c | 16 +---
> arch/x86/xen/smp_pv.c | 39 ++++++-----
> 6 files changed, 72 insertions(+), 151 deletions(-)
>
> --- a/arch/x86/Kconfig
> +++ b/arch/x86/Kconfig
> @@ -272,6 +272,7 @@ config X86
> select HAVE_UNSTABLE_SCHED_CLOCK
> select HAVE_USER_RETURN_NOTIFIER
> select HAVE_GENERIC_VDSO
> + select HOTPLUG_CORE_SYNC_FULL if SMP
> select HOTPLUG_SMT if SMP
> select IRQ_FORCED_THREADING
> select NEED_PER_CPU_EMBED_FIRST_CHUNK
> --- a/arch/x86/include/asm/smp.h
> +++ b/arch/x86/include/asm/smp.h
> @@ -38,6 +38,8 @@ struct smp_ops {
> void (*crash_stop_other_cpus)(void);
> void (*smp_send_reschedule)(int cpu);
>
> + void (*cleanup_dead_cpu)(unsigned cpu);
> + void (*poll_sync_state)(void);
> int (*cpu_up)(unsigned cpu, struct task_struct *tidle);
> int (*cpu_disable)(void);
> void (*cpu_die)(unsigned int cpu);
> @@ -90,7 +92,8 @@ static inline int __cpu_disable(void)
>
> static inline void __cpu_die(unsigned int cpu)
> {
> - smp_ops.cpu_die(cpu);
> + if (smp_ops.cpu_die)
> + smp_ops.cpu_die(cpu);
> }
>
> static inline void play_dead(void)
> @@ -122,8 +125,6 @@ void native_smp_cpus_done(unsigned int m
> int common_cpu_up(unsigned int cpunum, struct task_struct *tidle);
> int native_cpu_up(unsigned int cpunum, struct task_struct *tidle);
> int native_cpu_disable(void);
> -int common_cpu_die(unsigned int cpu);
> -void native_cpu_die(unsigned int cpu);
> void hlt_play_dead(void);
> void native_play_dead(void);
> void play_dead_common(void);
> --- a/arch/x86/kernel/smp.c
> +++ b/arch/x86/kernel/smp.c
> @@ -269,7 +269,6 @@ struct smp_ops smp_ops = {
> .smp_send_reschedule = native_smp_send_reschedule,
>
> .cpu_up = native_cpu_up,
> - .cpu_die = native_cpu_die,
> .cpu_disable = native_cpu_disable,
> .play_dead = native_play_dead,
>
> --- a/arch/x86/kernel/smpboot.c
> +++ b/arch/x86/kernel/smpboot.c
> @@ -57,6 +57,7 @@
> #include <linux/pgtable.h>
> #include <linux/overflow.h>
> #include <linux/stackprotector.h>
> +#include <linux/cpuhotplug.h>
>
> #include <asm/acpi.h>
> #include <asm/cacheinfo.h>
> @@ -101,9 +102,6 @@ EXPORT_PER_CPU_SYMBOL(cpu_die_map);
> DEFINE_PER_CPU_READ_MOSTLY(struct cpuinfo_x86, cpu_info);
> EXPORT_PER_CPU_SYMBOL(cpu_info);
>
> -/* All of these masks are initialized in setup_cpu_local_masks() */
> -static cpumask_var_t cpu_initialized_mask;
> -static cpumask_var_t cpu_callout_mask;
> /* Representing CPUs for which sibling maps can be computed */
> static cpumask_var_t cpu_sibling_setup_mask;
>
> @@ -169,8 +167,8 @@ static void smp_callin(void)
> int cpuid = smp_processor_id();
>
> /*
> - * If waken up by an INIT in an 82489DX configuration
> - * cpu_callout_mask guarantees we don't get here before an
> + * If waken up by an INIT in an 82489DX configuration the alive
> + * synchronization guarantees we don't get here before an
> * INIT_deassert IPI reaches our local APIC, so it is now safe to
> * touch our local APIC.
> *
> @@ -216,17 +214,6 @@ static void ap_calibrate_delay(void)
> cpu_data(smp_processor_id()).loops_per_jiffy = loops_per_jiffy;
> }
>
> -static void wait_for_master_cpu(int cpu)
> -{
> - /*
> - * Wait for release by control CPU before continuing with AP
> - * initialization.
> - */
> - WARN_ON(cpumask_test_and_set_cpu(cpu, cpu_initialized_mask));
> - while (!cpumask_test_cpu(cpu, cpu_callout_mask))
> - cpu_relax();
> -}
> -
> /*
> * Activate a secondary processor.
> */
> @@ -247,11 +234,10 @@ static void notrace start_secondary(void
> cpu_init_exception_handling();
>
> /*
> - * Sync point with wait_cpu_initialized(). Sets AP in
> - * cpu_initialized_mask and then waits for the control CPU
> - * to release it.
> + * Sync point with the hotplug core. Sets the sync state to ALIVE
> + * and waits for the control CPU to release it.
> */
> - wait_for_master_cpu(raw_smp_processor_id());
> + cpuhp_ap_sync_alive();
>
> cpu_init();
> rcu_cpu_starting(raw_smp_processor_id());
> @@ -285,7 +271,6 @@ static void notrace start_secondary(void
> set_cpu_online(smp_processor_id(), true);
> lapic_online();
> unlock_vector_lock();
> - cpu_set_state_online(smp_processor_id());
> x86_platform.nmi_init();
>
> /* enable local interrupts */
> @@ -736,9 +721,10 @@ static void impress_friends(void)
> * Allow the user to impress friends.
> */
> pr_debug("Before bogomips\n");
> - for_each_possible_cpu(cpu)
> - if (cpumask_test_cpu(cpu, cpu_callout_mask))
> + for_each_possible_cpu(cpu) {
> + if (cpumask_test_cpu(cpu, cpu_online_mask))
> bogosum += cpu_data(cpu).loops_per_jiffy;
This should be the same as for_each_online_cpu().
--
Brian Gerst
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 19/37] x86/smpboot: Switch to hotplug core state synchronization
@ 2023-04-15 12:58 ` Brian Gerst
0 siblings, 0 replies; 236+ messages in thread
From: Brian Gerst @ 2023-04-15 12:58 UTC (permalink / raw)
To: Thomas Gleixner
Cc: LKML, x86, David Woodhouse, Andrew Cooper, Arjan van de Veen,
Paolo Bonzini, Paul McKenney, Tom Lendacky, Sean Christopherson,
Oleksandr Natalenko, Paul Menzel, Guilherme G. Piccoli,
Piotr Gorski, Juergen Gross, Boris Ostrovsky, xen-devel,
David Woodhouse, Usama Arif, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Fri, Apr 14, 2023 at 7:44 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>
> The new AP state tracking and synchronization mechanism in the CPU hotplug
> core code allows to remove quite some x86 specific code:
>
> 1) The AP alive synchronization based on cpumasks
>
> 2) The decision whether an AP can be brought up again
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Juergen Gross <jgross@suse.com>
> Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
> Cc: xen-devel@lists.xenproject.org
> ---
> arch/x86/Kconfig | 1
> arch/x86/include/asm/smp.h | 7 +
> arch/x86/kernel/smp.c | 1
> arch/x86/kernel/smpboot.c | 159 ++++++++++-----------------------------------
> arch/x86/xen/smp_hvm.c | 16 +---
> arch/x86/xen/smp_pv.c | 39 ++++++-----
> 6 files changed, 72 insertions(+), 151 deletions(-)
>
> --- a/arch/x86/Kconfig
> +++ b/arch/x86/Kconfig
> @@ -272,6 +272,7 @@ config X86
> select HAVE_UNSTABLE_SCHED_CLOCK
> select HAVE_USER_RETURN_NOTIFIER
> select HAVE_GENERIC_VDSO
> + select HOTPLUG_CORE_SYNC_FULL if SMP
> select HOTPLUG_SMT if SMP
> select IRQ_FORCED_THREADING
> select NEED_PER_CPU_EMBED_FIRST_CHUNK
> --- a/arch/x86/include/asm/smp.h
> +++ b/arch/x86/include/asm/smp.h
> @@ -38,6 +38,8 @@ struct smp_ops {
> void (*crash_stop_other_cpus)(void);
> void (*smp_send_reschedule)(int cpu);
>
> + void (*cleanup_dead_cpu)(unsigned cpu);
> + void (*poll_sync_state)(void);
> int (*cpu_up)(unsigned cpu, struct task_struct *tidle);
> int (*cpu_disable)(void);
> void (*cpu_die)(unsigned int cpu);
> @@ -90,7 +92,8 @@ static inline int __cpu_disable(void)
>
> static inline void __cpu_die(unsigned int cpu)
> {
> - smp_ops.cpu_die(cpu);
> + if (smp_ops.cpu_die)
> + smp_ops.cpu_die(cpu);
> }
>
> static inline void play_dead(void)
> @@ -122,8 +125,6 @@ void native_smp_cpus_done(unsigned int m
> int common_cpu_up(unsigned int cpunum, struct task_struct *tidle);
> int native_cpu_up(unsigned int cpunum, struct task_struct *tidle);
> int native_cpu_disable(void);
> -int common_cpu_die(unsigned int cpu);
> -void native_cpu_die(unsigned int cpu);
> void hlt_play_dead(void);
> void native_play_dead(void);
> void play_dead_common(void);
> --- a/arch/x86/kernel/smp.c
> +++ b/arch/x86/kernel/smp.c
> @@ -269,7 +269,6 @@ struct smp_ops smp_ops = {
> .smp_send_reschedule = native_smp_send_reschedule,
>
> .cpu_up = native_cpu_up,
> - .cpu_die = native_cpu_die,
> .cpu_disable = native_cpu_disable,
> .play_dead = native_play_dead,
>
> --- a/arch/x86/kernel/smpboot.c
> +++ b/arch/x86/kernel/smpboot.c
> @@ -57,6 +57,7 @@
> #include <linux/pgtable.h>
> #include <linux/overflow.h>
> #include <linux/stackprotector.h>
> +#include <linux/cpuhotplug.h>
>
> #include <asm/acpi.h>
> #include <asm/cacheinfo.h>
> @@ -101,9 +102,6 @@ EXPORT_PER_CPU_SYMBOL(cpu_die_map);
> DEFINE_PER_CPU_READ_MOSTLY(struct cpuinfo_x86, cpu_info);
> EXPORT_PER_CPU_SYMBOL(cpu_info);
>
> -/* All of these masks are initialized in setup_cpu_local_masks() */
> -static cpumask_var_t cpu_initialized_mask;
> -static cpumask_var_t cpu_callout_mask;
> /* Representing CPUs for which sibling maps can be computed */
> static cpumask_var_t cpu_sibling_setup_mask;
>
> @@ -169,8 +167,8 @@ static void smp_callin(void)
> int cpuid = smp_processor_id();
>
> /*
> - * If waken up by an INIT in an 82489DX configuration
> - * cpu_callout_mask guarantees we don't get here before an
> + * If waken up by an INIT in an 82489DX configuration the alive
> + * synchronization guarantees we don't get here before an
> * INIT_deassert IPI reaches our local APIC, so it is now safe to
> * touch our local APIC.
> *
> @@ -216,17 +214,6 @@ static void ap_calibrate_delay(void)
> cpu_data(smp_processor_id()).loops_per_jiffy = loops_per_jiffy;
> }
>
> -static void wait_for_master_cpu(int cpu)
> -{
> - /*
> - * Wait for release by control CPU before continuing with AP
> - * initialization.
> - */
> - WARN_ON(cpumask_test_and_set_cpu(cpu, cpu_initialized_mask));
> - while (!cpumask_test_cpu(cpu, cpu_callout_mask))
> - cpu_relax();
> -}
> -
> /*
> * Activate a secondary processor.
> */
> @@ -247,11 +234,10 @@ static void notrace start_secondary(void
> cpu_init_exception_handling();
>
> /*
> - * Sync point with wait_cpu_initialized(). Sets AP in
> - * cpu_initialized_mask and then waits for the control CPU
> - * to release it.
> + * Sync point with the hotplug core. Sets the sync state to ALIVE
> + * and waits for the control CPU to release it.
> */
> - wait_for_master_cpu(raw_smp_processor_id());
> + cpuhp_ap_sync_alive();
>
> cpu_init();
> rcu_cpu_starting(raw_smp_processor_id());
> @@ -285,7 +271,6 @@ static void notrace start_secondary(void
> set_cpu_online(smp_processor_id(), true);
> lapic_online();
> unlock_vector_lock();
> - cpu_set_state_online(smp_processor_id());
> x86_platform.nmi_init();
>
> /* enable local interrupts */
> @@ -736,9 +721,10 @@ static void impress_friends(void)
> * Allow the user to impress friends.
> */
> pr_debug("Before bogomips\n");
> - for_each_possible_cpu(cpu)
> - if (cpumask_test_cpu(cpu, cpu_callout_mask))
> + for_each_possible_cpu(cpu) {
> + if (cpumask_test_cpu(cpu, cpu_online_mask))
> bogosum += cpu_data(cpu).loops_per_jiffy;
This should be the same as for_each_online_cpu().
--
Brian Gerst
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 35/37] x86/smpboot: Support parallel startup of secondary CPUs
2023-04-14 23:45 ` Thomas Gleixner
(?)
@ 2023-04-15 13:22 ` Brian Gerst
-1 siblings, 0 replies; 236+ messages in thread
From: Brian Gerst @ 2023-04-15 13:22 UTC (permalink / raw)
To: Thomas Gleixner
Cc: LKML, x86, David Woodhouse, Andrew Cooper, Arjan van de Veen,
Paolo Bonzini, Paul McKenney, Tom Lendacky, Sean Christopherson,
Oleksandr Natalenko, Paul Menzel, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Juergen Gross,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Fri, Apr 14, 2023 at 7:45 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>
> From: David Woodhouse <dwmw@amazon.co.uk>
>
> Rework the real-mode startup code to allow for APs to be brought up in
> parallel. This is in two parts:
>
> 1. Introduce a bit-spinlock to prevent them from all using the real
> mode stack at the same time.
>
> 2. Avoid needing to use the global smpboot_control variable to pass
> each AP its CPU number.
>
> To achieve the latter, export the cpuid_to_apicid[] array so that each
> AP can find its own CPU number by searching therein based on its APIC ID.
>
> Introduce flags in the top bits of smpboot_control which indicate methods
> by which an AP should find its CPU number. For a serialized bringup, the
> CPU number is explicitly passed in the low bits of smpboot_control as
> before. For parallel mode there are flags directing the AP to find its APIC
> ID in CPUID leaf 0x0b or 1x1f (for X2APIC mode) or CPUID leaf 0x01 where 8
> bits are sufficient, then perform the cpuid_to_apicid[] lookup with that.
>
> Aside from the fact that APs will now look up their CPU number via the
> newly-exported cpuid_to_apicid[] table, there is no behavioural change
> intended, since the parallel bootup has not yet been enabled.
>
> [ tglx: Initial proof of concept patch with bitlock and APIC ID lookup ]
> [ dwmw2: Rework and testing, commit message, CPUID 0x1 and CPU0 support ]
> [ seanc: Fix stray override of initial_gs in common_cpu_up() ]
> [ Oleksandr Natalenko: reported suspend/resume issue fixed in
> x86_acpi_suspend_lowlevel ]
>
> Co-developed-by: Thomas Gleixner <tglx@linutronix.de>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Co-developed-by: Brian Gerst <brgerst@gmail.com>
> Signed-off-by: Brian Gerst <brgerst@gmail.com>
> Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
> Signed-off-by: Usama Arif <usama.arif@bytedance.com>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> ---
> arch/x86/include/asm/apic.h | 2
> arch/x86/include/asm/realmode.h | 3 +
> arch/x86/include/asm/smp.h | 8 +++
> arch/x86/kernel/acpi/sleep.c | 9 +++
> arch/x86/kernel/apic/apic.c | 2
> arch/x86/kernel/head_64.S | 79 ++++++++++++++++++++++++++++++++++-
> arch/x86/kernel/smpboot.c | 5 --
> arch/x86/realmode/init.c | 3 +
> arch/x86/realmode/rm/trampoline_64.S | 27 +++++++++--
> 9 files changed, 125 insertions(+), 13 deletions(-)
>
> --- a/arch/x86/include/asm/apic.h
> +++ b/arch/x86/include/asm/apic.h
> @@ -55,6 +55,8 @@ extern int local_apic_timer_c2_ok;
> extern int disable_apic;
> extern unsigned int lapic_timer_period;
>
> +extern int cpuid_to_apicid[];
> +
> extern enum apic_intr_mode_id apic_intr_mode;
> enum apic_intr_mode_id {
> APIC_PIC,
> --- a/arch/x86/include/asm/realmode.h
> +++ b/arch/x86/include/asm/realmode.h
> @@ -52,6 +52,7 @@ struct trampoline_header {
> u64 efer;
> u32 cr4;
> u32 flags;
> + u32 lock;
> #endif
> };
>
> @@ -64,6 +65,8 @@ extern unsigned long initial_stack;
> extern unsigned long initial_vc_handler;
> #endif
>
> +extern u32 *trampoline_lock;
> +
> extern unsigned char real_mode_blob[];
> extern unsigned char real_mode_relocs[];
>
> --- a/arch/x86/include/asm/smp.h
> +++ b/arch/x86/include/asm/smp.h
> @@ -198,4 +198,12 @@ extern unsigned int smpboot_control;
>
> #endif /* !__ASSEMBLY__ */
>
> +/* Control bits for startup_64 */
> +#define STARTUP_APICID_CPUID_1F 0x80000000
> +#define STARTUP_APICID_CPUID_0B 0x40000000
> +#define STARTUP_APICID_CPUID_01 0x20000000
> +
> +/* Top 8 bits are reserved for control */
> +#define STARTUP_PARALLEL_MASK 0xFF000000
> +
> #endif /* _ASM_X86_SMP_H */
> --- a/arch/x86/kernel/acpi/sleep.c
> +++ b/arch/x86/kernel/acpi/sleep.c
> @@ -16,6 +16,7 @@
> #include <asm/cacheflush.h>
> #include <asm/realmode.h>
> #include <asm/hypervisor.h>
> +#include <asm/smp.h>
>
> #include <linux/ftrace.h>
> #include "../../realmode/rm/wakeup.h"
> @@ -127,7 +128,13 @@ int x86_acpi_suspend_lowlevel(void)
> * value is in the actual %rsp register.
> */
> current->thread.sp = (unsigned long)temp_stack + sizeof(temp_stack);
> - smpboot_control = smp_processor_id();
> + /*
> + * Ensure the CPU knows which one it is when it comes back, if
> + * it isn't in parallel mode and expected to work that out for
> + * itself.
> + */
> + if (!(smpboot_control & STARTUP_PARALLEL_MASK))
> + smpboot_control = smp_processor_id();
> #endif
> initial_code = (unsigned long)wakeup_long64;
> saved_magic = 0x123456789abcdef0L;
> --- a/arch/x86/kernel/apic/apic.c
> +++ b/arch/x86/kernel/apic/apic.c
> @@ -2377,7 +2377,7 @@ static int nr_logical_cpuids = 1;
> /*
> * Used to store mapping between logical CPU IDs and APIC IDs.
> */
> -static int cpuid_to_apicid[] = {
> +int cpuid_to_apicid[] = {
> [0 ... NR_CPUS - 1] = -1,
> };
>
> --- a/arch/x86/kernel/head_64.S
> +++ b/arch/x86/kernel/head_64.S
> @@ -25,6 +25,7 @@
> #include <asm/export.h>
> #include <asm/nospec-branch.h>
> #include <asm/fixmap.h>
> +#include <asm/smp.h>
>
> /*
> * We are not able to switch in one step to the final KERNEL ADDRESS SPACE
> @@ -234,8 +235,70 @@ SYM_INNER_LABEL(secondary_startup_64_no_
> ANNOTATE_NOENDBR // above
>
> #ifdef CONFIG_SMP
> + /*
> + * For parallel boot, the APIC ID is retrieved from CPUID, and then
> + * used to look up the CPU number. For booting a single CPU, the
> + * CPU number is encoded in smpboot_control.
> + *
> + * Bit 31 STARTUP_APICID_CPUID_1F flag (use CPUID 0x1f)
> + * Bit 30 STARTUP_APICID_CPUID_0B flag (use CPUID 0x0b)
> + * Bit 29 STARTUP_APICID_CPUID_01 flag (use CPUID 0x01)
> + * Bit 0-23 CPU# if STARTUP_APICID_CPUID_xx flags are not set
> + */
> movl smpboot_control(%rip), %ecx
> + testl $STARTUP_APICID_CPUID_1F, %ecx
> + jnz .Luse_cpuid_1f
> + testl $STARTUP_APICID_CPUID_0B, %ecx
> + jnz .Luse_cpuid_0b
> + testl $STARTUP_APICID_CPUID_01, %ecx
> + jnz .Luse_cpuid_01
> + andl $(~STARTUP_PARALLEL_MASK), %ecx
> + jmp .Lsetup_cpu
> +
> +.Luse_cpuid_01:
> + mov $0x01, %eax
> + cpuid
> + mov %ebx, %edx
> + shr $24, %edx
> + jmp .Lsetup_AP
> +
> +.Luse_cpuid_0b:
> + mov $0x0B, %eax
> + xorl %ecx, %ecx
> + cpuid
> + jmp .Lsetup_AP
> +
> +.Luse_cpuid_1f:
> + mov $0x1f, %eax
> + xorl %ecx, %ecx
> + cpuid
>
> +.Lsetup_AP:
> + /* EDX contains the APIC ID of the current CPU */
> + xorq %rcx, %rcx
> + leaq cpuid_to_apicid(%rip), %rbx
> +
> +.Lfind_cpunr:
> + cmpl (%rbx,%rcx,4), %edx
> + jz .Lsetup_cpu
> + inc %ecx
> +#ifdef CONFIG_FORCE_NR_CPUS
> + cmpl $NR_CPUS, %ecx
> +#else
> + cmpl nr_cpu_ids(%rip), %ecx
> +#endif
> + jb .Lfind_cpunr
> +
> + /* APIC ID not found in the table. Drop the trampoline lock and bail. */
> + movq trampoline_lock(%rip), %rax
> + lock
> + btrl $0, (%rax)
> +
> +1: cli
> + hlt
> + jmp 1b
> +
> +.Lsetup_cpu:
> /* Get the per cpu offset for the given CPU# which is in ECX */
> movq __per_cpu_offset(,%rcx,8), %rdx
> #else
> @@ -248,10 +311,20 @@ SYM_INNER_LABEL(secondary_startup_64_no_
> *
> * RDX contains the per-cpu offset
> */
> - movq pcpu_hot + X86_current_task(%rdx), %rax
> - movq TASK_threadsp(%rax), %rsp
> + movq pcpu_hot + X86_top_of_stack(%rdx), %rsp
Switching to using pcpu_hot.top_of_stack is ok, but it's not
completely equivalent. top_of_stack points to the end of the pt_regs
structure, while the kernel stack starts below pt_regs even for kernel
threads. So you need to subtract PTREGS_SIZE from the stack pointer
after this.
This change should also be a separate patch.
--
Brian Gerst
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 35/37] x86/smpboot: Support parallel startup of secondary CPUs
@ 2023-04-15 13:22 ` Brian Gerst
0 siblings, 0 replies; 236+ messages in thread
From: Brian Gerst @ 2023-04-15 13:22 UTC (permalink / raw)
To: Thomas Gleixner
Cc: LKML, x86, David Woodhouse, Andrew Cooper, Arjan van de Veen,
Paolo Bonzini, Paul McKenney, Tom Lendacky, Sean Christopherson,
Oleksandr Natalenko, Paul Menzel, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Juergen Gross,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Fri, Apr 14, 2023 at 7:45 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>
> From: David Woodhouse <dwmw@amazon.co.uk>
>
> Rework the real-mode startup code to allow for APs to be brought up in
> parallel. This is in two parts:
>
> 1. Introduce a bit-spinlock to prevent them from all using the real
> mode stack at the same time.
>
> 2. Avoid needing to use the global smpboot_control variable to pass
> each AP its CPU number.
>
> To achieve the latter, export the cpuid_to_apicid[] array so that each
> AP can find its own CPU number by searching therein based on its APIC ID.
>
> Introduce flags in the top bits of smpboot_control which indicate methods
> by which an AP should find its CPU number. For a serialized bringup, the
> CPU number is explicitly passed in the low bits of smpboot_control as
> before. For parallel mode there are flags directing the AP to find its APIC
> ID in CPUID leaf 0x0b or 1x1f (for X2APIC mode) or CPUID leaf 0x01 where 8
> bits are sufficient, then perform the cpuid_to_apicid[] lookup with that.
>
> Aside from the fact that APs will now look up their CPU number via the
> newly-exported cpuid_to_apicid[] table, there is no behavioural change
> intended, since the parallel bootup has not yet been enabled.
>
> [ tglx: Initial proof of concept patch with bitlock and APIC ID lookup ]
> [ dwmw2: Rework and testing, commit message, CPUID 0x1 and CPU0 support ]
> [ seanc: Fix stray override of initial_gs in common_cpu_up() ]
> [ Oleksandr Natalenko: reported suspend/resume issue fixed in
> x86_acpi_suspend_lowlevel ]
>
> Co-developed-by: Thomas Gleixner <tglx@linutronix.de>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Co-developed-by: Brian Gerst <brgerst@gmail.com>
> Signed-off-by: Brian Gerst <brgerst@gmail.com>
> Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
> Signed-off-by: Usama Arif <usama.arif@bytedance.com>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> ---
> arch/x86/include/asm/apic.h | 2
> arch/x86/include/asm/realmode.h | 3 +
> arch/x86/include/asm/smp.h | 8 +++
> arch/x86/kernel/acpi/sleep.c | 9 +++
> arch/x86/kernel/apic/apic.c | 2
> arch/x86/kernel/head_64.S | 79 ++++++++++++++++++++++++++++++++++-
> arch/x86/kernel/smpboot.c | 5 --
> arch/x86/realmode/init.c | 3 +
> arch/x86/realmode/rm/trampoline_64.S | 27 +++++++++--
> 9 files changed, 125 insertions(+), 13 deletions(-)
>
> --- a/arch/x86/include/asm/apic.h
> +++ b/arch/x86/include/asm/apic.h
> @@ -55,6 +55,8 @@ extern int local_apic_timer_c2_ok;
> extern int disable_apic;
> extern unsigned int lapic_timer_period;
>
> +extern int cpuid_to_apicid[];
> +
> extern enum apic_intr_mode_id apic_intr_mode;
> enum apic_intr_mode_id {
> APIC_PIC,
> --- a/arch/x86/include/asm/realmode.h
> +++ b/arch/x86/include/asm/realmode.h
> @@ -52,6 +52,7 @@ struct trampoline_header {
> u64 efer;
> u32 cr4;
> u32 flags;
> + u32 lock;
> #endif
> };
>
> @@ -64,6 +65,8 @@ extern unsigned long initial_stack;
> extern unsigned long initial_vc_handler;
> #endif
>
> +extern u32 *trampoline_lock;
> +
> extern unsigned char real_mode_blob[];
> extern unsigned char real_mode_relocs[];
>
> --- a/arch/x86/include/asm/smp.h
> +++ b/arch/x86/include/asm/smp.h
> @@ -198,4 +198,12 @@ extern unsigned int smpboot_control;
>
> #endif /* !__ASSEMBLY__ */
>
> +/* Control bits for startup_64 */
> +#define STARTUP_APICID_CPUID_1F 0x80000000
> +#define STARTUP_APICID_CPUID_0B 0x40000000
> +#define STARTUP_APICID_CPUID_01 0x20000000
> +
> +/* Top 8 bits are reserved for control */
> +#define STARTUP_PARALLEL_MASK 0xFF000000
> +
> #endif /* _ASM_X86_SMP_H */
> --- a/arch/x86/kernel/acpi/sleep.c
> +++ b/arch/x86/kernel/acpi/sleep.c
> @@ -16,6 +16,7 @@
> #include <asm/cacheflush.h>
> #include <asm/realmode.h>
> #include <asm/hypervisor.h>
> +#include <asm/smp.h>
>
> #include <linux/ftrace.h>
> #include "../../realmode/rm/wakeup.h"
> @@ -127,7 +128,13 @@ int x86_acpi_suspend_lowlevel(void)
> * value is in the actual %rsp register.
> */
> current->thread.sp = (unsigned long)temp_stack + sizeof(temp_stack);
> - smpboot_control = smp_processor_id();
> + /*
> + * Ensure the CPU knows which one it is when it comes back, if
> + * it isn't in parallel mode and expected to work that out for
> + * itself.
> + */
> + if (!(smpboot_control & STARTUP_PARALLEL_MASK))
> + smpboot_control = smp_processor_id();
> #endif
> initial_code = (unsigned long)wakeup_long64;
> saved_magic = 0x123456789abcdef0L;
> --- a/arch/x86/kernel/apic/apic.c
> +++ b/arch/x86/kernel/apic/apic.c
> @@ -2377,7 +2377,7 @@ static int nr_logical_cpuids = 1;
> /*
> * Used to store mapping between logical CPU IDs and APIC IDs.
> */
> -static int cpuid_to_apicid[] = {
> +int cpuid_to_apicid[] = {
> [0 ... NR_CPUS - 1] = -1,
> };
>
> --- a/arch/x86/kernel/head_64.S
> +++ b/arch/x86/kernel/head_64.S
> @@ -25,6 +25,7 @@
> #include <asm/export.h>
> #include <asm/nospec-branch.h>
> #include <asm/fixmap.h>
> +#include <asm/smp.h>
>
> /*
> * We are not able to switch in one step to the final KERNEL ADDRESS SPACE
> @@ -234,8 +235,70 @@ SYM_INNER_LABEL(secondary_startup_64_no_
> ANNOTATE_NOENDBR // above
>
> #ifdef CONFIG_SMP
> + /*
> + * For parallel boot, the APIC ID is retrieved from CPUID, and then
> + * used to look up the CPU number. For booting a single CPU, the
> + * CPU number is encoded in smpboot_control.
> + *
> + * Bit 31 STARTUP_APICID_CPUID_1F flag (use CPUID 0x1f)
> + * Bit 30 STARTUP_APICID_CPUID_0B flag (use CPUID 0x0b)
> + * Bit 29 STARTUP_APICID_CPUID_01 flag (use CPUID 0x01)
> + * Bit 0-23 CPU# if STARTUP_APICID_CPUID_xx flags are not set
> + */
> movl smpboot_control(%rip), %ecx
> + testl $STARTUP_APICID_CPUID_1F, %ecx
> + jnz .Luse_cpuid_1f
> + testl $STARTUP_APICID_CPUID_0B, %ecx
> + jnz .Luse_cpuid_0b
> + testl $STARTUP_APICID_CPUID_01, %ecx
> + jnz .Luse_cpuid_01
> + andl $(~STARTUP_PARALLEL_MASK), %ecx
> + jmp .Lsetup_cpu
> +
> +.Luse_cpuid_01:
> + mov $0x01, %eax
> + cpuid
> + mov %ebx, %edx
> + shr $24, %edx
> + jmp .Lsetup_AP
> +
> +.Luse_cpuid_0b:
> + mov $0x0B, %eax
> + xorl %ecx, %ecx
> + cpuid
> + jmp .Lsetup_AP
> +
> +.Luse_cpuid_1f:
> + mov $0x1f, %eax
> + xorl %ecx, %ecx
> + cpuid
>
> +.Lsetup_AP:
> + /* EDX contains the APIC ID of the current CPU */
> + xorq %rcx, %rcx
> + leaq cpuid_to_apicid(%rip), %rbx
> +
> +.Lfind_cpunr:
> + cmpl (%rbx,%rcx,4), %edx
> + jz .Lsetup_cpu
> + inc %ecx
> +#ifdef CONFIG_FORCE_NR_CPUS
> + cmpl $NR_CPUS, %ecx
> +#else
> + cmpl nr_cpu_ids(%rip), %ecx
> +#endif
> + jb .Lfind_cpunr
> +
> + /* APIC ID not found in the table. Drop the trampoline lock and bail. */
> + movq trampoline_lock(%rip), %rax
> + lock
> + btrl $0, (%rax)
> +
> +1: cli
> + hlt
> + jmp 1b
> +
> +.Lsetup_cpu:
> /* Get the per cpu offset for the given CPU# which is in ECX */
> movq __per_cpu_offset(,%rcx,8), %rdx
> #else
> @@ -248,10 +311,20 @@ SYM_INNER_LABEL(secondary_startup_64_no_
> *
> * RDX contains the per-cpu offset
> */
> - movq pcpu_hot + X86_current_task(%rdx), %rax
> - movq TASK_threadsp(%rax), %rsp
> + movq pcpu_hot + X86_top_of_stack(%rdx), %rsp
Switching to using pcpu_hot.top_of_stack is ok, but it's not
completely equivalent. top_of_stack points to the end of the pt_regs
structure, while the kernel stack starts below pt_regs even for kernel
threads. So you need to subtract PTREGS_SIZE from the stack pointer
after this.
This change should also be a separate patch.
--
Brian Gerst
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 35/37] x86/smpboot: Support parallel startup of secondary CPUs
@ 2023-04-15 13:22 ` Brian Gerst
0 siblings, 0 replies; 236+ messages in thread
From: Brian Gerst @ 2023-04-15 13:22 UTC (permalink / raw)
To: Thomas Gleixner
Cc: LKML, x86, David Woodhouse, Andrew Cooper, Arjan van de Veen,
Paolo Bonzini, Paul McKenney, Tom Lendacky, Sean Christopherson,
Oleksandr Natalenko, Paul Menzel, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Juergen Gross,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Fri, Apr 14, 2023 at 7:45 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>
> From: David Woodhouse <dwmw@amazon.co.uk>
>
> Rework the real-mode startup code to allow for APs to be brought up in
> parallel. This is in two parts:
>
> 1. Introduce a bit-spinlock to prevent them from all using the real
> mode stack at the same time.
>
> 2. Avoid needing to use the global smpboot_control variable to pass
> each AP its CPU number.
>
> To achieve the latter, export the cpuid_to_apicid[] array so that each
> AP can find its own CPU number by searching therein based on its APIC ID.
>
> Introduce flags in the top bits of smpboot_control which indicate methods
> by which an AP should find its CPU number. For a serialized bringup, the
> CPU number is explicitly passed in the low bits of smpboot_control as
> before. For parallel mode there are flags directing the AP to find its APIC
> ID in CPUID leaf 0x0b or 1x1f (for X2APIC mode) or CPUID leaf 0x01 where 8
> bits are sufficient, then perform the cpuid_to_apicid[] lookup with that.
>
> Aside from the fact that APs will now look up their CPU number via the
> newly-exported cpuid_to_apicid[] table, there is no behavioural change
> intended, since the parallel bootup has not yet been enabled.
>
> [ tglx: Initial proof of concept patch with bitlock and APIC ID lookup ]
> [ dwmw2: Rework and testing, commit message, CPUID 0x1 and CPU0 support ]
> [ seanc: Fix stray override of initial_gs in common_cpu_up() ]
> [ Oleksandr Natalenko: reported suspend/resume issue fixed in
> x86_acpi_suspend_lowlevel ]
>
> Co-developed-by: Thomas Gleixner <tglx@linutronix.de>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Co-developed-by: Brian Gerst <brgerst@gmail.com>
> Signed-off-by: Brian Gerst <brgerst@gmail.com>
> Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
> Signed-off-by: Usama Arif <usama.arif@bytedance.com>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> ---
> arch/x86/include/asm/apic.h | 2
> arch/x86/include/asm/realmode.h | 3 +
> arch/x86/include/asm/smp.h | 8 +++
> arch/x86/kernel/acpi/sleep.c | 9 +++
> arch/x86/kernel/apic/apic.c | 2
> arch/x86/kernel/head_64.S | 79 ++++++++++++++++++++++++++++++++++-
> arch/x86/kernel/smpboot.c | 5 --
> arch/x86/realmode/init.c | 3 +
> arch/x86/realmode/rm/trampoline_64.S | 27 +++++++++--
> 9 files changed, 125 insertions(+), 13 deletions(-)
>
> --- a/arch/x86/include/asm/apic.h
> +++ b/arch/x86/include/asm/apic.h
> @@ -55,6 +55,8 @@ extern int local_apic_timer_c2_ok;
> extern int disable_apic;
> extern unsigned int lapic_timer_period;
>
> +extern int cpuid_to_apicid[];
> +
> extern enum apic_intr_mode_id apic_intr_mode;
> enum apic_intr_mode_id {
> APIC_PIC,
> --- a/arch/x86/include/asm/realmode.h
> +++ b/arch/x86/include/asm/realmode.h
> @@ -52,6 +52,7 @@ struct trampoline_header {
> u64 efer;
> u32 cr4;
> u32 flags;
> + u32 lock;
> #endif
> };
>
> @@ -64,6 +65,8 @@ extern unsigned long initial_stack;
> extern unsigned long initial_vc_handler;
> #endif
>
> +extern u32 *trampoline_lock;
> +
> extern unsigned char real_mode_blob[];
> extern unsigned char real_mode_relocs[];
>
> --- a/arch/x86/include/asm/smp.h
> +++ b/arch/x86/include/asm/smp.h
> @@ -198,4 +198,12 @@ extern unsigned int smpboot_control;
>
> #endif /* !__ASSEMBLY__ */
>
> +/* Control bits for startup_64 */
> +#define STARTUP_APICID_CPUID_1F 0x80000000
> +#define STARTUP_APICID_CPUID_0B 0x40000000
> +#define STARTUP_APICID_CPUID_01 0x20000000
> +
> +/* Top 8 bits are reserved for control */
> +#define STARTUP_PARALLEL_MASK 0xFF000000
> +
> #endif /* _ASM_X86_SMP_H */
> --- a/arch/x86/kernel/acpi/sleep.c
> +++ b/arch/x86/kernel/acpi/sleep.c
> @@ -16,6 +16,7 @@
> #include <asm/cacheflush.h>
> #include <asm/realmode.h>
> #include <asm/hypervisor.h>
> +#include <asm/smp.h>
>
> #include <linux/ftrace.h>
> #include "../../realmode/rm/wakeup.h"
> @@ -127,7 +128,13 @@ int x86_acpi_suspend_lowlevel(void)
> * value is in the actual %rsp register.
> */
> current->thread.sp = (unsigned long)temp_stack + sizeof(temp_stack);
> - smpboot_control = smp_processor_id();
> + /*
> + * Ensure the CPU knows which one it is when it comes back, if
> + * it isn't in parallel mode and expected to work that out for
> + * itself.
> + */
> + if (!(smpboot_control & STARTUP_PARALLEL_MASK))
> + smpboot_control = smp_processor_id();
> #endif
> initial_code = (unsigned long)wakeup_long64;
> saved_magic = 0x123456789abcdef0L;
> --- a/arch/x86/kernel/apic/apic.c
> +++ b/arch/x86/kernel/apic/apic.c
> @@ -2377,7 +2377,7 @@ static int nr_logical_cpuids = 1;
> /*
> * Used to store mapping between logical CPU IDs and APIC IDs.
> */
> -static int cpuid_to_apicid[] = {
> +int cpuid_to_apicid[] = {
> [0 ... NR_CPUS - 1] = -1,
> };
>
> --- a/arch/x86/kernel/head_64.S
> +++ b/arch/x86/kernel/head_64.S
> @@ -25,6 +25,7 @@
> #include <asm/export.h>
> #include <asm/nospec-branch.h>
> #include <asm/fixmap.h>
> +#include <asm/smp.h>
>
> /*
> * We are not able to switch in one step to the final KERNEL ADDRESS SPACE
> @@ -234,8 +235,70 @@ SYM_INNER_LABEL(secondary_startup_64_no_
> ANNOTATE_NOENDBR // above
>
> #ifdef CONFIG_SMP
> + /*
> + * For parallel boot, the APIC ID is retrieved from CPUID, and then
> + * used to look up the CPU number. For booting a single CPU, the
> + * CPU number is encoded in smpboot_control.
> + *
> + * Bit 31 STARTUP_APICID_CPUID_1F flag (use CPUID 0x1f)
> + * Bit 30 STARTUP_APICID_CPUID_0B flag (use CPUID 0x0b)
> + * Bit 29 STARTUP_APICID_CPUID_01 flag (use CPUID 0x01)
> + * Bit 0-23 CPU# if STARTUP_APICID_CPUID_xx flags are not set
> + */
> movl smpboot_control(%rip), %ecx
> + testl $STARTUP_APICID_CPUID_1F, %ecx
> + jnz .Luse_cpuid_1f
> + testl $STARTUP_APICID_CPUID_0B, %ecx
> + jnz .Luse_cpuid_0b
> + testl $STARTUP_APICID_CPUID_01, %ecx
> + jnz .Luse_cpuid_01
> + andl $(~STARTUP_PARALLEL_MASK), %ecx
> + jmp .Lsetup_cpu
> +
> +.Luse_cpuid_01:
> + mov $0x01, %eax
> + cpuid
> + mov %ebx, %edx
> + shr $24, %edx
> + jmp .Lsetup_AP
> +
> +.Luse_cpuid_0b:
> + mov $0x0B, %eax
> + xorl %ecx, %ecx
> + cpuid
> + jmp .Lsetup_AP
> +
> +.Luse_cpuid_1f:
> + mov $0x1f, %eax
> + xorl %ecx, %ecx
> + cpuid
>
> +.Lsetup_AP:
> + /* EDX contains the APIC ID of the current CPU */
> + xorq %rcx, %rcx
> + leaq cpuid_to_apicid(%rip), %rbx
> +
> +.Lfind_cpunr:
> + cmpl (%rbx,%rcx,4), %edx
> + jz .Lsetup_cpu
> + inc %ecx
> +#ifdef CONFIG_FORCE_NR_CPUS
> + cmpl $NR_CPUS, %ecx
> +#else
> + cmpl nr_cpu_ids(%rip), %ecx
> +#endif
> + jb .Lfind_cpunr
> +
> + /* APIC ID not found in the table. Drop the trampoline lock and bail. */
> + movq trampoline_lock(%rip), %rax
> + lock
> + btrl $0, (%rax)
> +
> +1: cli
> + hlt
> + jmp 1b
> +
> +.Lsetup_cpu:
> /* Get the per cpu offset for the given CPU# which is in ECX */
> movq __per_cpu_offset(,%rcx,8), %rdx
> #else
> @@ -248,10 +311,20 @@ SYM_INNER_LABEL(secondary_startup_64_no_
> *
> * RDX contains the per-cpu offset
> */
> - movq pcpu_hot + X86_current_task(%rdx), %rax
> - movq TASK_threadsp(%rax), %rsp
> + movq pcpu_hot + X86_top_of_stack(%rdx), %rsp
Switching to using pcpu_hot.top_of_stack is ok, but it's not
completely equivalent. top_of_stack points to the end of the pt_regs
structure, while the kernel stack starts below pt_regs even for kernel
threads. So you need to subtract PTREGS_SIZE from the stack pointer
after this.
This change should also be a separate patch.
--
Brian Gerst
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 19/37] x86/smpboot: Switch to hotplug core state synchronization
2023-04-15 12:58 ` Brian Gerst
(?)
@ 2023-04-15 21:04 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-15 21:04 UTC (permalink / raw)
To: Brian Gerst
Cc: LKML, x86, David Woodhouse, Andrew Cooper, Arjan van de Veen,
Paolo Bonzini, Paul McKenney, Tom Lendacky, Sean Christopherson,
Oleksandr Natalenko, Paul Menzel, Guilherme G. Piccoli,
Piotr Gorski, Juergen Gross, Boris Ostrovsky, xen-devel,
David Woodhouse, Usama Arif, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Sat, Apr 15 2023 at 08:58, Brian Gerst wrote:
> On Fri, Apr 14, 2023 at 7:44 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>> pr_debug("Before bogomips\n");
>> - for_each_possible_cpu(cpu)
>> - if (cpumask_test_cpu(cpu, cpu_callout_mask))
>> + for_each_possible_cpu(cpu) {
>> + if (cpumask_test_cpu(cpu, cpu_online_mask))
>> bogosum += cpu_data(cpu).loops_per_jiffy;
>
> This should be the same as for_each_online_cpu().
Duh, yes. Obviously...
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 19/37] x86/smpboot: Switch to hotplug core state synchronization
@ 2023-04-15 21:04 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-15 21:04 UTC (permalink / raw)
To: Brian Gerst
Cc: LKML, x86, David Woodhouse, Andrew Cooper, Arjan van de Veen,
Paolo Bonzini, Paul McKenney, Tom Lendacky, Sean Christopherson,
Oleksandr Natalenko, Paul Menzel, Guilherme G. Piccoli,
Piotr Gorski, Juergen Gross, Boris Ostrovsky, xen-devel,
David Woodhouse, Usama Arif, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Sat, Apr 15 2023 at 08:58, Brian Gerst wrote:
> On Fri, Apr 14, 2023 at 7:44 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>> pr_debug("Before bogomips\n");
>> - for_each_possible_cpu(cpu)
>> - if (cpumask_test_cpu(cpu, cpu_callout_mask))
>> + for_each_possible_cpu(cpu) {
>> + if (cpumask_test_cpu(cpu, cpu_online_mask))
>> bogosum += cpu_data(cpu).loops_per_jiffy;
>
> This should be the same as for_each_online_cpu().
Duh, yes. Obviously...
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 19/37] x86/smpboot: Switch to hotplug core state synchronization
@ 2023-04-15 21:04 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-15 21:04 UTC (permalink / raw)
To: Brian Gerst
Cc: LKML, x86, David Woodhouse, Andrew Cooper, Arjan van de Veen,
Paolo Bonzini, Paul McKenney, Tom Lendacky, Sean Christopherson,
Oleksandr Natalenko, Paul Menzel, Guilherme G. Piccoli,
Piotr Gorski, Juergen Gross, Boris Ostrovsky, xen-devel,
David Woodhouse, Usama Arif, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Sat, Apr 15 2023 at 08:58, Brian Gerst wrote:
> On Fri, Apr 14, 2023 at 7:44 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>> pr_debug("Before bogomips\n");
>> - for_each_possible_cpu(cpu)
>> - if (cpumask_test_cpu(cpu, cpu_callout_mask))
>> + for_each_possible_cpu(cpu) {
>> + if (cpumask_test_cpu(cpu, cpu_online_mask))
>> bogosum += cpu_data(cpu).loops_per_jiffy;
>
> This should be the same as for_each_online_cpu().
Duh, yes. Obviously...
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 35/37] x86/smpboot: Support parallel startup of secondary CPUs
2023-04-15 13:22 ` Brian Gerst
(?)
@ 2023-04-15 21:06 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-15 21:06 UTC (permalink / raw)
To: Brian Gerst
Cc: LKML, x86, David Woodhouse, Andrew Cooper, Arjan van de Veen,
Paolo Bonzini, Paul McKenney, Tom Lendacky, Sean Christopherson,
Oleksandr Natalenko, Paul Menzel, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Juergen Gross,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Sat, Apr 15 2023 at 09:22, Brian Gerst wrote:
> On Fri, Apr 14, 2023 at 7:45 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>> @@ -248,10 +311,20 @@ SYM_INNER_LABEL(secondary_startup_64_no_
>> *
>> * RDX contains the per-cpu offset
>> */
>> - movq pcpu_hot + X86_current_task(%rdx), %rax
>> - movq TASK_threadsp(%rax), %rsp
>> + movq pcpu_hot + X86_top_of_stack(%rdx), %rsp
>
> Switching to using pcpu_hot.top_of_stack is ok, but it's not
> completely equivalent. top_of_stack points to the end of the pt_regs
> structure, while the kernel stack starts below pt_regs even for kernel
> threads. So you need to subtract PTREGS_SIZE from the stack pointer
> after this.
>
> This change should also be a separate patch.
You're right on both counts.
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 35/37] x86/smpboot: Support parallel startup of secondary CPUs
@ 2023-04-15 21:06 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-15 21:06 UTC (permalink / raw)
To: Brian Gerst
Cc: LKML, x86, David Woodhouse, Andrew Cooper, Arjan van de Veen,
Paolo Bonzini, Paul McKenney, Tom Lendacky, Sean Christopherson,
Oleksandr Natalenko, Paul Menzel, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Juergen Gross,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Sat, Apr 15 2023 at 09:22, Brian Gerst wrote:
> On Fri, Apr 14, 2023 at 7:45 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>> @@ -248,10 +311,20 @@ SYM_INNER_LABEL(secondary_startup_64_no_
>> *
>> * RDX contains the per-cpu offset
>> */
>> - movq pcpu_hot + X86_current_task(%rdx), %rax
>> - movq TASK_threadsp(%rax), %rsp
>> + movq pcpu_hot + X86_top_of_stack(%rdx), %rsp
>
> Switching to using pcpu_hot.top_of_stack is ok, but it's not
> completely equivalent. top_of_stack points to the end of the pt_regs
> structure, while the kernel stack starts below pt_regs even for kernel
> threads. So you need to subtract PTREGS_SIZE from the stack pointer
> after this.
>
> This change should also be a separate patch.
You're right on both counts.
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 35/37] x86/smpboot: Support parallel startup of secondary CPUs
@ 2023-04-15 21:06 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-15 21:06 UTC (permalink / raw)
To: Brian Gerst
Cc: LKML, x86, David Woodhouse, Andrew Cooper, Arjan van de Veen,
Paolo Bonzini, Paul McKenney, Tom Lendacky, Sean Christopherson,
Oleksandr Natalenko, Paul Menzel, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Juergen Gross,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Sat, Apr 15 2023 at 09:22, Brian Gerst wrote:
> On Fri, Apr 14, 2023 at 7:45 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>> @@ -248,10 +311,20 @@ SYM_INNER_LABEL(secondary_startup_64_no_
>> *
>> * RDX contains the per-cpu offset
>> */
>> - movq pcpu_hot + X86_current_task(%rdx), %rax
>> - movq TASK_threadsp(%rax), %rsp
>> + movq pcpu_hot + X86_top_of_stack(%rdx), %rsp
>
> Switching to using pcpu_hot.top_of_stack is ok, but it's not
> completely equivalent. top_of_stack points to the end of the pt_regs
> structure, while the kernel stack starts below pt_regs even for kernel
> threads. So you need to subtract PTREGS_SIZE from the stack pointer
> after this.
>
> This change should also be a separate patch.
You're right on both counts.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-17 8:35 ` Juergen Gross
-1 siblings, 0 replies; 236+ messages in thread
From: Juergen Gross @ 2023-04-17 8:35 UTC (permalink / raw)
To: Thomas Gleixner, LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
[-- Attachment #1.1.1: Type: text/plain, Size: 30261 bytes --]
On 15.04.23 01:44, Thomas Gleixner wrote:
> Hi!
>
> This is a complete rework of the parallel bringup patch series (V17)
>
> https://lore.kernel.org/lkml/20230328195758.1049469-1-usama.arif@bytedance.com
>
> to address the issues which were discovered in review:
>
> 1) The X86 microcode loader serialization requirement
>
> https://lore.kernel.org/lkml/87v8iirxun.ffs@tglx
>
> Microcode loading on HT enabled X86 CPUs requires that the microcode is
> loaded on the primary thread. The sibling thread(s) must be in
> quiescent state; either looping in a place which is aware of potential
> changes by the microcode update (see late loading) or in fully quiescent
> state, i.e. waiting for INIT/SIPI.
>
> This is required by hardware/firmware on Intel. Aside of that it's a
> vendor independent software correctness issue. Assume the following
> sequence:
>
> CPU1.0 CPU1.1
> CPUID($A)
> Load microcode.
> Changes CPUID($A, $B)
> CPUID($B)
>
> CPU1.1 makes a decision on $A and $B which might be inconsistent due
> to the microcode update.
>
> The solution for this is to bringup the primary threads first and after
> that the siblings. Loading microcode on the siblings is a NOOP on Intel
> and on AMD it is guaranteed to only modify thread local state.
>
> This ensures that the APs can load microcode before reaching the alive
> synchronization point w/o doing any further x86 specific
> synchronization between the core siblings.
>
> 2) The general design issues discussed in V16
>
> https://lore.kernel.org/lkml/87pm8y6yme.ffs@tglx
>
> The previous parallel bringup patches just glued this mechanism into
> the existing code without a deeper analysis of the synchronization
> mechanisms and without generalizing it so that the control logic is
> mostly in the core code and not made an architecture specific tinker
> space.
>
> Much of that had been pointed out 2 years ago in the discussions about
> the early versions of parallel bringup already.
>
>
> The series is based on:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip x86/apic
>
> and also available from git:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git hotplug
>
>
> Background
> ----------
>
> The reason why people are interested in parallel bringup is to shorten
> the (kexec) reboot time of cloud servers to reduce the downtime of the
> VM tenants. There are obviously other interesting use cases for this
> like VM startup time, embedded devices...
>
> The current fully serialized bringup does the following per AP:
>
> 1) Prepare callbacks (allocate, intialize, create threads)
> 2) Kick the AP alive (e.g. INIT/SIPI on x86)
> 3) Wait for the AP to report alive state
> 4) Let the AP continue through the atomic bringup
> 5) Let the AP run the threaded bringup to full online state
>
> There are two significant delays:
>
> #3 The time for an AP to report alive state in start_secondary() on x86
> has been measured in the range between 350us and 3.5ms depending on
> vendor and CPU type, BIOS microcode size etc.
>
> #4 The atomic bringup does the microcode update. This has been measured
> to take up to ~8ms on the primary threads depending on the microcode
> patch size to apply.
>
> On a two socket SKL server with 56 cores (112 threads) the boot CPU spends
> on current mainline about 800ms busy waiting for the APs to come up and
> apply microcode. That's more than 80% of the actual onlining procedure.
>
> By splitting the actual bringup mechanism into two parts this can be
> reduced to waiting for the first AP to report alive or if the system is
> large enough the first AP is already waiting when the boot CPU finished the
> wake-up of the last AP.
>
>
> The actual solution comes in several parts
> ------------------------------------------
>
> 1) [P 1-2] General cleanups (init annotations, kernel doc...)
>
> 2) [P 3] The obvious
>
> Avoid pointless delay calibration when TSC is synchronized across
> sockets. That removes a whopping 100ms delay for the first CPU of a
> socket. This is an improvement independent of parallel bringup and had
> been discussed two years ago already.
>
> 2) [P 3-6] Removal of the CPU0 hotplug hack.
>
> This was added 11 years ago with the promise to make this a real
> hardware mechanism, but that never materialized. As physical CPU
> hotplug is not really supported and the physical unplugging of CPU0
> never materialized there is no reason to keep this cruft around. It's
> just maintenance ballast for no value and the removal makes
> implementing the parallel bringup feature way simpler.
>
> 3) [P 7-16] Cleanup of the existing bringup mechanism:
>
> a) Code reorganisation so that the general hotplug specific code is
> in smpboot.c and not sprinkled all over the place
>
> b) Decouple MTRR/PAT initialization from smp_callout_mask to prepare
> for replacing that mask with a hotplug core code synchronization
> mechanism.
>
> c) Make TSC synchronization function call based so that the control CPU
> does not have to busy wait for nothing if synchronization is not
> required.
>
> d) Remove the smp_callin_mask synchronization point as its not longer
> required due to #3c.
>
> e) Rework the sparse_irq_lock held region in the core code so that the
> next polling synchronization point in the x86 code can be removed to.
>
> f) Due to #3e it's not longer required to spin wait for the AP to set
> it's online bit. Remove wait_cpu_online() and the XENPV
> counterpart. So the control CPU can directly wait for the online
> idle completion by the AP and free the control CPU up for other
> work.
>
> This reduces the synchronization points in the x86 code to one, which
> is the AP alive one. This synchronization will be moved to core
> infrastructure in the next section.
>
> 4) [P 17-27] Replace the disconnected CPU state tracking
>
> The extra CPU state tracking which is used by a few architectures is
> completely separate from the CPU hotplug core code.
>
> Replacing it by a variant integrated in the core hotplug machinery
> allows to reduce architecture specific code and provides a generic
> synchronization mechanism for (parallel) CPU bringup/teardown.
>
> - Convert x86 over and replace the AP alive synchronization on x86 with
> the core variant which removes the remaining x86 hotplug
> synchronization masks.
>
> - Convert the other architectures usage and remove the old interface
> and code.
>
> 5) [P 28-30] Split the bringup into two steps
>
> First step invokes the wakeup function on the BP, e.g. SIPI/STARTUP on
> x86. The second one waits on the BP for the AP to report alive and
> releases it for the complete onlining.
>
> As the hotplug state machine allows partial bringup this allows later
> to kick all APs alive in a first iteration and then bring them up
> completely one by one afterwards.
>
> 6) [P 31] Switch the primary thread detection to a cpumask
>
> This makes the parallel bringup a simple cpumask based mechanism
> without tons of conditionals and checks for primary threads.
>
> 7) [P 32] Implement the parallel bringup core code
>
> The parallel bringup looks like this:
>
> 1) Bring up the primary SMT threads to the CPUHP_KICK_AP_ALIVE step
> one by one
>
> 2) Bring up the primary SMT threads to the CPUHP_ONLINE step one by
> one
>
> 3) Bring up the secondary SMT threads to the CPUHP_KICK_AP_ALIVE
> step one by one
>
> 4) Bring up the secondary SMT threads to the CPUHP_ONLINE
> step one by one
>
> In case that SMT is not supported this is obviously reduced to step #1
> and #2.
>
> 8) [P 33-37] Prepare X86 for parallel bringup and enable it
>
>
> Caveats
> -------
>
> The non X86 changes have been all compile tested. Boot and runtime
> testing has only be done on a few real hardware platforms and qemu as
> available. That definitely needs some help from the people who have
> these systems at their fingertips.
>
>
> Results and analysis
> --------------------
>
> Here are numbers for a dual socket SKL 56 cores/ 112 threads machine. All
> numbers in milliseconds. The time measured is the time which the cpu_up()
> call takes for each CPU and phase. It's not exact as the system is already
> scheduling, handling interrupts and soft interrupts, which is obviously
> skewing the picture slightly.
>
> Baseline tip tree x86/apic branch.
>
> total avg/CPU min max
> total : 912.081 8.217 3.720 113.271
>
> The max of 100ms is due to the silly delay calibration for the second
> socket which takes 100ms and was eliminated first. Also the other initial
> cleanups and improvements take some time away.
>
> So the real baseline becomes:
>
> total avg/CPU min max
> total : 785.960 7.081 3.752 36.098
>
> The max here is on the first CPU of the second socket. 20ms of that is due
> to TSC synchronization and an extra 2ms to react on the SIPI.
>
> With parallel bootup enabled this becomes:
>
> total avg/CPU min max
> prepare: 39.108 0.352 0.238 0.883
> online : 45.166 0.407 0.170 20.357
> total : 84.274 0.759 0.408 21.240
>
> That's a factor ~9.3 reduction on average.
>
> Looking at the 27 primary threads of socket 0 then this becomes even more
> interesting:
>
> total avg/CPU min max
> total : 325.764 12.065 11.981 14.125
>
> versus:
> total avg/CPU min max
> prepare: 8.945 0.331 0.238 0.834
> online : 4.830 0.179 0.170 0.212
> total : 13.775 0.510 0.408 1.046
>
> So the reduction factor is ~23.5 here. That's mostly because the 20ms TSC
> sync is not skewing the picture.
>
> For all 55 primaries, i.e with the 20ms TSC sync extra for socket 1 this
> becomes:
>
> total avg/CPU min max
> total : 685.489 12.463 11.975 36.098
>
> versus:
>
> total avg/CPU min max
> prepare: 19.080 0.353 0.238 0.883
> online : 30.283 0.561 0.170 20.357
> total : 49.363 0.914 0.408 21.240
>
> The TSC sync reduces the win to a factor of ~13.8
>
> With 'tsc=reliable' on the command line the socket sync is disabled which
> brings it back to the socket 0 numbers:
>
> total avg/CPU min max
> prepare: 18.970 0.351 0.231 0.874
> online : 10.328 0.191 0.169 0.358
> total : 29.298 0.543 0.400 1.232
>
> Now looking at the secondary threads only:
>
> total avg/CPU min max
> total : 100.471 1.794 0.375 4.745
>
> versus:
> total avg/CPU min max
> prepare: 19.753 0.353 0.257 0.512
> online : 14.671 0.262 0.179 3.461
> total : 34.424 0.615 0.436 3.973
>
> Still a factor of ~3.
>
> The average on the secondaries for the serialized bringup is significantly
> lower than for the primaries because the SIPI response time is shorter and
> the microcode update takes no time.
>
> This varies wildly with the system, whether microcode in BIOS is already up
> to date, how big the microcode patch is and how long the INIT/SIPI response
> time is. On an AMD Zen3 machine INIT/SIPI response time is amazingly fast
> (350us), but then it lacks TSC_ADJUST and does a two millisecond TSC sync
> test for _every_ AP. All of this sucks...
>
>
> Possible further enhancements
> -----------------------------
>
> It's definitely worthwhile to look into reducing the cross socket TSC sync
> test time. It's probably safe enough to use 5ms or even 2ms instead of 20ms
> on systems with TSC_ADJUST and a few other 'TSC is sane' indicators. Moving
> it out of the hotplug path is eventually possible, but that needs some deep
> thoughts.
>
> Let's take the TSC sync out of the picture by adding 'tsc=reliable" to the
> kernel command line. So the bringup of 111 APs takes:
>
> total avg/CPU min max
> prepare: 38.936 0.351 0.231 0.874
> online : 25.231 0.227 0.169 3.465
> total : 64.167 0.578 0.400 4.339
>
> Some of the outliers are not necessarily in the state callbacks as the
> system is already scheduling and handles interrupts and soft
> interrupts. Haven't analyzed that yet in detail.
>
> In the prepare stage which runs on the control CPU the larger steps are:
>
> smpcfd:prepare 16us avg/CPU
> threads:prepare 98us avg/CPU
> workqueue:prepare 43us avg/CPU
> trace/RB:prepare 135us avg/CPU
>
> The trace ringbuffer initialization allocates 354 pages and 354 control
> structures one by one. That probably should allocate a large page and an
> array of control structures and work from there. I'm sure that would reduce
> this significantly. Steven?
>
> smpcfd does just a percpu allocation. No idea why that takes that long.
>
> Vs. threads and workqueues. David thought about spreading out the
> preparation work and do it really in parallel. That's a nice idea, but the
> threads and workqueue prepare steps are self serializing. The workqueue one
> has a global mutex and aside of that both steps create kernel threads which
> implicitely serialize on kthreadd. alloc_percpu(), which is used by
> smpcfd:prepare is also globally serialized.
>
> The rest of the prepare steps is pretty much in the single digit
> microseconds range.
>
> On the AP side it should be possible to move some of the initialization
> steps before the alive synchronization point, but that really needs a lot
> of analysis whether the functions are safe to invoke that early and outside
> of the cpu_hotplug_lock held region for the case of two stage parallel
> bringup; see below.
>
> The largest part is:
>
> identify_secondary_cpu() 99us avg/CPU
>
> Inside of identify_secondary_cpu() the largest offender:
>
> mcheck_init() 73us avg/CPU
>
> This part is definitly worth to be looked at whether it can be at least
> partially moved to the early startup code before the alive
> synchronization point. There's a lot of deep analysis required and
> ideally we just rewrite the whole CPUID evaluation trainwreck
> completely.
>
> The rest of the AP side is low single digit microseconds except of:
>
> perf/x86:starting 14us avg/CPU
>
> smpboot/threads:online 13us avg/CPU
> workqueue:online 17us avg/CPU
> mm/vmstat:online 17us avg/CPU
> sched:active 30us avg/CPU
>
> sched:active is special. Onlining the first secondary HT thread on the
> second socket creates a 3.2ms outlier which skews the whole picture. That's
> caused by enabling the static key sched_smt_present which patches the world
> and some more. For all other APs this is really in the 1us range. This
> definitely could be postponed during bootup like the scheduler domain
> rebuild is done after the bringup. But that's still fully serialized and
> single threaded and obviously could be done later in the context of async
> parallel init. It's unclear why this is different with the fully serialized
> bringup where it takes significantly less time, but that's something which
> needs to be investigated.
>
>
> Is truly parallel bringup feasible?
> -----------------------------------
>
> In theory yes, realistically no. Why?
>
> 1) The preparation phase
>
> Allocating memory, creating threads for the to be brought up CPU must
> obviously happen on an already online CPU.
>
> While it would be possible to bring up a subset of CPUs first and let
> them do the preparation steps for groups of still offline CPUs
> concurrently, the actual benefit of doing so is dubious.
>
> The prime example is kernel thread creation, which is implicitely
> serialized on kthreadd.
>
> A simple experiment shows that 4 concurrent workers on 4 different
> CPUs where each is creating 14 * 5 = 70 kernel threads are 5% slower
> than a single worker creating 4 * 14 * 5 = 280 threads.
>
> So we'd need to have multiple kthreadd instances to handle that,
> which would then serialize on tasklist lock and other things.
>
> That aside the preparation phase is also affected by the problem
> below.
>
> 2) Assumptions about hotplug serialization
>
> a) There are quite some assumptions about CPU bringup being fully
> serialized across state transitions. A lot of state callbacks rely
> on that and would require local locking.
>
> Adding that local locking is surely possible, but that has several
> downsides:
>
> - It adds complexity and makes it harder for developers to get
> this correct. The subtle bugs resulting out of that are going
> to be interesting
>
> - Fine grained locking has a charm, but only if the time spent
> for the actual work is larger than the time required for
> serialization and synchronization.
>
> Serializing a callback which takes less than a microsecond and
> then having a large number of CPUs contending on the lock will
> not make it any faster at all. That's a well known issue of
> parallelizing and neither made up nor kernel specific.
>
> b) Some operations definitely require to be protected by the
> cpu_hotplug_lock, especially those which affect cpumasks as the
> masks are guaranteed to be stable in a cpus_read_lock()'ed region.
>
> As this lock cannot be taken in atomic contexts, it's required
> that the control CPU holds the lock write locked across these
> state transitions. And no, we are not making this a spinlock just
> for that and we even can't.
>
> Just slapping a lock into the x86 specific part of the cpumask
> update function does not solve anything. The relevant patch in V17
> is completely useless as it only serializes the actual cpumask/map
> modifications, but all read side users are hosed if the update
> would be moved before the alive synchronization point, i.e. into a
> non hotplug lock protected region.
>
> Even if the hotplug lock would be held accross the whole parallel
> bringup operation then this would still expose all usage of these
> masks and maps in the actual hotplug state callbacks to concurrent
> modifications.
>
> And no, we are not going to expose an architecture specific raw
> spinlock to the hotplug state callbacks, especially not to those
> in generic code.
>
> c) Some cpu_read_lock()'ed regions also expect that there is no CPU
> state transition happening which would modify their local
> state. This would again require local serialization.
>
> 3) The amount of work and churn:
>
> - Analyze the per architecture low level startup functions plus their
> descendant functions and make them ready for concurrency if
> necessary.
>
> - Analyze ~300 hotplug state callbacks and their descendant functions
> and make them ready for concurrency if necessary.
>
> - Analyze all cpus_read_lock()'ed regions and address their
> requirements.
>
> - Rewrite the core code to handle the cpu_hotplug_lock requirements
> only in distinct phases of the state machine.
>
> - Rewrite the core code to handle state callback failure and the
> related rollback in the context of the new rules.
>
> - ...
>
> Even if some people are dedicated enough to do that, it's very
> questionable whether the resulting complexity is justified.
>
> We've spent a serious amount of time to sanitize hotplug and bring it
> into a state where it is correct. This also made it reasonably simple
> for developers to implement hotplug state callbacks without having to
> become hotplug experts.
>
> Breaking this completely up will result in a flood of hard to diagnose
> subtle issues for sure. Who is going to deal with them?
>
> The experience with this series so far does not make me comfortable
> about that thought in any way.
>
>
> Summary
> -------
>
> The obvious and low hanging fruits have to be solved first:
>
> - The CPUID evaluation and related setup mechanisms
>
> - The trace/ringbuffer oddity
>
> - The sched:active oddity for the first sibling on the second socket
>
> - Some other expensive things which I'm not seeing in my test setup due
> to lack of hardware or configuration.
>
> Anything else is pretty much wishful thinking in my opinion.
>
> To be clear. I'm not standing in the way if there is a proper solution,
> but that requires to respect the basic engineering rules:
>
> 1) Correctness first
> 2) Keep it maintainable
> 3) Keep it simple
>
> So far this stuff failed already at #1.
>
> I completely understand why this is important for cloud people, but
> the real question to ask here is what are the actual requirements.
>
> As far as I understand the main goal is to make a (kexec) reboot
> almost invisible to VM tenants.
>
> Now lets look at how this works:
>
> A) Freeze VMs and persist state
> B) kexec into the new kernel
> C) Restore VMs from persistant memory
> D) Thaw VMs
>
> So the key problem is how long it takes to get from #B to #C and finally
> to #D.
>
> As far as I understand #C takes a serious amount of time and cannot be
> parallelized for whatever reasons.
>
> At the same time the number of online CPUs required to restore the VMs
> state is less than the number of online CPUs required to actually
> operate them in #D.
>
> That means it would be good enough to return to userspace with a
> limited number of online CPUs as fast as possible. A certain amount of
> CPUs are going to be busy with restoring the VMs state, i.e. one CPU
> per VM. Some remaining non-busy CPU can bringup the rest of the system
> and the APs in order to be functional for #D, i.e the restore of VM
> operation.
>
> Trying to optimize this purely in kernel space by adding complexity of
> dubious value is simply bogus in my opinion.
>
> It's already possible today to limit the number of CPUs which are
> initially onlined and online the rest later from user space.
>
> There are two issue there:
>
> a) The death by MCE broadcast problem
>
> Quite some (contemporary) x86 CPU generations are affected by
> this:
>
> - MCE can be broadcasted to all CPUs and not only issued locally
> to the CPU which triggered it.
>
> - Any CPU which has CR4.MCE == 0, even if it sits in a wait
> for INIT/SIPI state, will cause an immediate shutdown of the
> machine if a broadcasted MCE is delivered.
>
> b) Do the parallel bringup via sysfs control knob
>
> The per CPU target state interface allows to do that today one
> by one, but it's akward and has quite some overhead.
>
> A knob to online the rest of the not yet onlined present CPUs
> with the benefit of the parallel bringup mechanism is
> missing.
>
> #a) That's a risk to take by the operator.
>
> Even the regular serialized bringup does not protect against this
> issue up to the point where all present CPUs have at least
> initialized CR4.
>
> Limiting the number of APs to online early via the kernel command
> line widens that window and increases the risk further by
> executing user space before all APs have CR4 initialized.
>
> But the same applies to a deferred online mechanism implemented in
> the kernel where some worker brings up the not yet online APs while
> the early online CPUs are already executing user space code.
>
> #b) Is a no brainer to implement on top of this.
>
>
> Conclusion
> ----------
>
> Adding the basic parallel bringup mechanism as provided by this series
> makes a lot of sense. Improving particular issues as pointed out in the
> analysis makes sense too.
>
> But trying to solve an application specific problem fully in the kernel
> with tons of complexity, without exploring straight forward and simple
> approaches first, does not make any sense at all.
>
> Thanks,
>
> tglx
>
> ---
> Documentation/admin-guide/kernel-parameters.txt | 20
> Documentation/core-api/cpu_hotplug.rst | 13
> arch/Kconfig | 23 +
> arch/arm/Kconfig | 1
> arch/arm/include/asm/smp.h | 2
> arch/arm/kernel/smp.c | 18
> arch/arm64/Kconfig | 1
> arch/arm64/include/asm/smp.h | 2
> arch/arm64/kernel/smp.c | 14
> arch/csky/Kconfig | 1
> arch/csky/include/asm/smp.h | 2
> arch/csky/kernel/smp.c | 8
> arch/mips/Kconfig | 1
> arch/mips/cavium-octeon/smp.c | 1
> arch/mips/include/asm/smp-ops.h | 1
> arch/mips/kernel/smp-bmips.c | 1
> arch/mips/kernel/smp-cps.c | 14
> arch/mips/kernel/smp.c | 8
> arch/mips/loongson64/smp.c | 1
> arch/parisc/Kconfig | 1
> arch/parisc/kernel/process.c | 4
> arch/parisc/kernel/smp.c | 7
> arch/riscv/Kconfig | 1
> arch/riscv/include/asm/smp.h | 2
> arch/riscv/kernel/cpu-hotplug.c | 14
> arch/x86/Kconfig | 45 --
> arch/x86/include/asm/apic.h | 5
> arch/x86/include/asm/cpu.h | 5
> arch/x86/include/asm/cpumask.h | 5
> arch/x86/include/asm/processor.h | 1
> arch/x86/include/asm/realmode.h | 3
> arch/x86/include/asm/sev-common.h | 3
> arch/x86/include/asm/smp.h | 26 -
> arch/x86/include/asm/topology.h | 23 -
> arch/x86/include/asm/tsc.h | 2
> arch/x86/kernel/acpi/sleep.c | 9
> arch/x86/kernel/apic/apic.c | 22 -
> arch/x86/kernel/callthunks.c | 4
> arch/x86/kernel/cpu/amd.c | 2
> arch/x86/kernel/cpu/cacheinfo.c | 21
> arch/x86/kernel/cpu/common.c | 50 --
> arch/x86/kernel/cpu/topology.c | 3
> arch/x86/kernel/head_32.S | 14
> arch/x86/kernel/head_64.S | 121 +++++
> arch/x86/kernel/sev.c | 2
> arch/x86/kernel/smp.c | 3
> arch/x86/kernel/smpboot.c | 508 ++++++++----------------
> arch/x86/kernel/topology.c | 98 ----
> arch/x86/kernel/tsc.c | 20
> arch/x86/kernel/tsc_sync.c | 36 -
> arch/x86/power/cpu.c | 37 -
> arch/x86/realmode/init.c | 3
> arch/x86/realmode/rm/trampoline_64.S | 27 +
> arch/x86/xen/enlighten_hvm.c | 11
> arch/x86/xen/smp_hvm.c | 16
> arch/x86/xen/smp_pv.c | 56 +-
> drivers/acpi/processor_idle.c | 4
> include/linux/cpu.h | 4
> include/linux/cpuhotplug.h | 17
> kernel/cpu.c | 397 +++++++++++++++++-
> kernel/smp.c | 2
> kernel/smpboot.c | 163 -------
> 62 files changed, 953 insertions(+), 976 deletions(-)
>
>
Tested with a Xen PV dom0 on an 8 cpu system, no issues found.
Tested-by: Juergen Gross <jgross@suse.com>
Juergen
[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3149 bytes --]
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-17 8:35 ` Juergen Gross
0 siblings, 0 replies; 236+ messages in thread
From: Juergen Gross @ 2023-04-17 8:35 UTC (permalink / raw)
To: Thomas Gleixner, LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
[-- Attachment #1.1.1.1: Type: text/plain, Size: 30261 bytes --]
On 15.04.23 01:44, Thomas Gleixner wrote:
> Hi!
>
> This is a complete rework of the parallel bringup patch series (V17)
>
> https://lore.kernel.org/lkml/20230328195758.1049469-1-usama.arif@bytedance.com
>
> to address the issues which were discovered in review:
>
> 1) The X86 microcode loader serialization requirement
>
> https://lore.kernel.org/lkml/87v8iirxun.ffs@tglx
>
> Microcode loading on HT enabled X86 CPUs requires that the microcode is
> loaded on the primary thread. The sibling thread(s) must be in
> quiescent state; either looping in a place which is aware of potential
> changes by the microcode update (see late loading) or in fully quiescent
> state, i.e. waiting for INIT/SIPI.
>
> This is required by hardware/firmware on Intel. Aside of that it's a
> vendor independent software correctness issue. Assume the following
> sequence:
>
> CPU1.0 CPU1.1
> CPUID($A)
> Load microcode.
> Changes CPUID($A, $B)
> CPUID($B)
>
> CPU1.1 makes a decision on $A and $B which might be inconsistent due
> to the microcode update.
>
> The solution for this is to bringup the primary threads first and after
> that the siblings. Loading microcode on the siblings is a NOOP on Intel
> and on AMD it is guaranteed to only modify thread local state.
>
> This ensures that the APs can load microcode before reaching the alive
> synchronization point w/o doing any further x86 specific
> synchronization between the core siblings.
>
> 2) The general design issues discussed in V16
>
> https://lore.kernel.org/lkml/87pm8y6yme.ffs@tglx
>
> The previous parallel bringup patches just glued this mechanism into
> the existing code without a deeper analysis of the synchronization
> mechanisms and without generalizing it so that the control logic is
> mostly in the core code and not made an architecture specific tinker
> space.
>
> Much of that had been pointed out 2 years ago in the discussions about
> the early versions of parallel bringup already.
>
>
> The series is based on:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip x86/apic
>
> and also available from git:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git hotplug
>
>
> Background
> ----------
>
> The reason why people are interested in parallel bringup is to shorten
> the (kexec) reboot time of cloud servers to reduce the downtime of the
> VM tenants. There are obviously other interesting use cases for this
> like VM startup time, embedded devices...
>
> The current fully serialized bringup does the following per AP:
>
> 1) Prepare callbacks (allocate, intialize, create threads)
> 2) Kick the AP alive (e.g. INIT/SIPI on x86)
> 3) Wait for the AP to report alive state
> 4) Let the AP continue through the atomic bringup
> 5) Let the AP run the threaded bringup to full online state
>
> There are two significant delays:
>
> #3 The time for an AP to report alive state in start_secondary() on x86
> has been measured in the range between 350us and 3.5ms depending on
> vendor and CPU type, BIOS microcode size etc.
>
> #4 The atomic bringup does the microcode update. This has been measured
> to take up to ~8ms on the primary threads depending on the microcode
> patch size to apply.
>
> On a two socket SKL server with 56 cores (112 threads) the boot CPU spends
> on current mainline about 800ms busy waiting for the APs to come up and
> apply microcode. That's more than 80% of the actual onlining procedure.
>
> By splitting the actual bringup mechanism into two parts this can be
> reduced to waiting for the first AP to report alive or if the system is
> large enough the first AP is already waiting when the boot CPU finished the
> wake-up of the last AP.
>
>
> The actual solution comes in several parts
> ------------------------------------------
>
> 1) [P 1-2] General cleanups (init annotations, kernel doc...)
>
> 2) [P 3] The obvious
>
> Avoid pointless delay calibration when TSC is synchronized across
> sockets. That removes a whopping 100ms delay for the first CPU of a
> socket. This is an improvement independent of parallel bringup and had
> been discussed two years ago already.
>
> 2) [P 3-6] Removal of the CPU0 hotplug hack.
>
> This was added 11 years ago with the promise to make this a real
> hardware mechanism, but that never materialized. As physical CPU
> hotplug is not really supported and the physical unplugging of CPU0
> never materialized there is no reason to keep this cruft around. It's
> just maintenance ballast for no value and the removal makes
> implementing the parallel bringup feature way simpler.
>
> 3) [P 7-16] Cleanup of the existing bringup mechanism:
>
> a) Code reorganisation so that the general hotplug specific code is
> in smpboot.c and not sprinkled all over the place
>
> b) Decouple MTRR/PAT initialization from smp_callout_mask to prepare
> for replacing that mask with a hotplug core code synchronization
> mechanism.
>
> c) Make TSC synchronization function call based so that the control CPU
> does not have to busy wait for nothing if synchronization is not
> required.
>
> d) Remove the smp_callin_mask synchronization point as its not longer
> required due to #3c.
>
> e) Rework the sparse_irq_lock held region in the core code so that the
> next polling synchronization point in the x86 code can be removed to.
>
> f) Due to #3e it's not longer required to spin wait for the AP to set
> it's online bit. Remove wait_cpu_online() and the XENPV
> counterpart. So the control CPU can directly wait for the online
> idle completion by the AP and free the control CPU up for other
> work.
>
> This reduces the synchronization points in the x86 code to one, which
> is the AP alive one. This synchronization will be moved to core
> infrastructure in the next section.
>
> 4) [P 17-27] Replace the disconnected CPU state tracking
>
> The extra CPU state tracking which is used by a few architectures is
> completely separate from the CPU hotplug core code.
>
> Replacing it by a variant integrated in the core hotplug machinery
> allows to reduce architecture specific code and provides a generic
> synchronization mechanism for (parallel) CPU bringup/teardown.
>
> - Convert x86 over and replace the AP alive synchronization on x86 with
> the core variant which removes the remaining x86 hotplug
> synchronization masks.
>
> - Convert the other architectures usage and remove the old interface
> and code.
>
> 5) [P 28-30] Split the bringup into two steps
>
> First step invokes the wakeup function on the BP, e.g. SIPI/STARTUP on
> x86. The second one waits on the BP for the AP to report alive and
> releases it for the complete onlining.
>
> As the hotplug state machine allows partial bringup this allows later
> to kick all APs alive in a first iteration and then bring them up
> completely one by one afterwards.
>
> 6) [P 31] Switch the primary thread detection to a cpumask
>
> This makes the parallel bringup a simple cpumask based mechanism
> without tons of conditionals and checks for primary threads.
>
> 7) [P 32] Implement the parallel bringup core code
>
> The parallel bringup looks like this:
>
> 1) Bring up the primary SMT threads to the CPUHP_KICK_AP_ALIVE step
> one by one
>
> 2) Bring up the primary SMT threads to the CPUHP_ONLINE step one by
> one
>
> 3) Bring up the secondary SMT threads to the CPUHP_KICK_AP_ALIVE
> step one by one
>
> 4) Bring up the secondary SMT threads to the CPUHP_ONLINE
> step one by one
>
> In case that SMT is not supported this is obviously reduced to step #1
> and #2.
>
> 8) [P 33-37] Prepare X86 for parallel bringup and enable it
>
>
> Caveats
> -------
>
> The non X86 changes have been all compile tested. Boot and runtime
> testing has only be done on a few real hardware platforms and qemu as
> available. That definitely needs some help from the people who have
> these systems at their fingertips.
>
>
> Results and analysis
> --------------------
>
> Here are numbers for a dual socket SKL 56 cores/ 112 threads machine. All
> numbers in milliseconds. The time measured is the time which the cpu_up()
> call takes for each CPU and phase. It's not exact as the system is already
> scheduling, handling interrupts and soft interrupts, which is obviously
> skewing the picture slightly.
>
> Baseline tip tree x86/apic branch.
>
> total avg/CPU min max
> total : 912.081 8.217 3.720 113.271
>
> The max of 100ms is due to the silly delay calibration for the second
> socket which takes 100ms and was eliminated first. Also the other initial
> cleanups and improvements take some time away.
>
> So the real baseline becomes:
>
> total avg/CPU min max
> total : 785.960 7.081 3.752 36.098
>
> The max here is on the first CPU of the second socket. 20ms of that is due
> to TSC synchronization and an extra 2ms to react on the SIPI.
>
> With parallel bootup enabled this becomes:
>
> total avg/CPU min max
> prepare: 39.108 0.352 0.238 0.883
> online : 45.166 0.407 0.170 20.357
> total : 84.274 0.759 0.408 21.240
>
> That's a factor ~9.3 reduction on average.
>
> Looking at the 27 primary threads of socket 0 then this becomes even more
> interesting:
>
> total avg/CPU min max
> total : 325.764 12.065 11.981 14.125
>
> versus:
> total avg/CPU min max
> prepare: 8.945 0.331 0.238 0.834
> online : 4.830 0.179 0.170 0.212
> total : 13.775 0.510 0.408 1.046
>
> So the reduction factor is ~23.5 here. That's mostly because the 20ms TSC
> sync is not skewing the picture.
>
> For all 55 primaries, i.e with the 20ms TSC sync extra for socket 1 this
> becomes:
>
> total avg/CPU min max
> total : 685.489 12.463 11.975 36.098
>
> versus:
>
> total avg/CPU min max
> prepare: 19.080 0.353 0.238 0.883
> online : 30.283 0.561 0.170 20.357
> total : 49.363 0.914 0.408 21.240
>
> The TSC sync reduces the win to a factor of ~13.8
>
> With 'tsc=reliable' on the command line the socket sync is disabled which
> brings it back to the socket 0 numbers:
>
> total avg/CPU min max
> prepare: 18.970 0.351 0.231 0.874
> online : 10.328 0.191 0.169 0.358
> total : 29.298 0.543 0.400 1.232
>
> Now looking at the secondary threads only:
>
> total avg/CPU min max
> total : 100.471 1.794 0.375 4.745
>
> versus:
> total avg/CPU min max
> prepare: 19.753 0.353 0.257 0.512
> online : 14.671 0.262 0.179 3.461
> total : 34.424 0.615 0.436 3.973
>
> Still a factor of ~3.
>
> The average on the secondaries for the serialized bringup is significantly
> lower than for the primaries because the SIPI response time is shorter and
> the microcode update takes no time.
>
> This varies wildly with the system, whether microcode in BIOS is already up
> to date, how big the microcode patch is and how long the INIT/SIPI response
> time is. On an AMD Zen3 machine INIT/SIPI response time is amazingly fast
> (350us), but then it lacks TSC_ADJUST and does a two millisecond TSC sync
> test for _every_ AP. All of this sucks...
>
>
> Possible further enhancements
> -----------------------------
>
> It's definitely worthwhile to look into reducing the cross socket TSC sync
> test time. It's probably safe enough to use 5ms or even 2ms instead of 20ms
> on systems with TSC_ADJUST and a few other 'TSC is sane' indicators. Moving
> it out of the hotplug path is eventually possible, but that needs some deep
> thoughts.
>
> Let's take the TSC sync out of the picture by adding 'tsc=reliable" to the
> kernel command line. So the bringup of 111 APs takes:
>
> total avg/CPU min max
> prepare: 38.936 0.351 0.231 0.874
> online : 25.231 0.227 0.169 3.465
> total : 64.167 0.578 0.400 4.339
>
> Some of the outliers are not necessarily in the state callbacks as the
> system is already scheduling and handles interrupts and soft
> interrupts. Haven't analyzed that yet in detail.
>
> In the prepare stage which runs on the control CPU the larger steps are:
>
> smpcfd:prepare 16us avg/CPU
> threads:prepare 98us avg/CPU
> workqueue:prepare 43us avg/CPU
> trace/RB:prepare 135us avg/CPU
>
> The trace ringbuffer initialization allocates 354 pages and 354 control
> structures one by one. That probably should allocate a large page and an
> array of control structures and work from there. I'm sure that would reduce
> this significantly. Steven?
>
> smpcfd does just a percpu allocation. No idea why that takes that long.
>
> Vs. threads and workqueues. David thought about spreading out the
> preparation work and do it really in parallel. That's a nice idea, but the
> threads and workqueue prepare steps are self serializing. The workqueue one
> has a global mutex and aside of that both steps create kernel threads which
> implicitely serialize on kthreadd. alloc_percpu(), which is used by
> smpcfd:prepare is also globally serialized.
>
> The rest of the prepare steps is pretty much in the single digit
> microseconds range.
>
> On the AP side it should be possible to move some of the initialization
> steps before the alive synchronization point, but that really needs a lot
> of analysis whether the functions are safe to invoke that early and outside
> of the cpu_hotplug_lock held region for the case of two stage parallel
> bringup; see below.
>
> The largest part is:
>
> identify_secondary_cpu() 99us avg/CPU
>
> Inside of identify_secondary_cpu() the largest offender:
>
> mcheck_init() 73us avg/CPU
>
> This part is definitly worth to be looked at whether it can be at least
> partially moved to the early startup code before the alive
> synchronization point. There's a lot of deep analysis required and
> ideally we just rewrite the whole CPUID evaluation trainwreck
> completely.
>
> The rest of the AP side is low single digit microseconds except of:
>
> perf/x86:starting 14us avg/CPU
>
> smpboot/threads:online 13us avg/CPU
> workqueue:online 17us avg/CPU
> mm/vmstat:online 17us avg/CPU
> sched:active 30us avg/CPU
>
> sched:active is special. Onlining the first secondary HT thread on the
> second socket creates a 3.2ms outlier which skews the whole picture. That's
> caused by enabling the static key sched_smt_present which patches the world
> and some more. For all other APs this is really in the 1us range. This
> definitely could be postponed during bootup like the scheduler domain
> rebuild is done after the bringup. But that's still fully serialized and
> single threaded and obviously could be done later in the context of async
> parallel init. It's unclear why this is different with the fully serialized
> bringup where it takes significantly less time, but that's something which
> needs to be investigated.
>
>
> Is truly parallel bringup feasible?
> -----------------------------------
>
> In theory yes, realistically no. Why?
>
> 1) The preparation phase
>
> Allocating memory, creating threads for the to be brought up CPU must
> obviously happen on an already online CPU.
>
> While it would be possible to bring up a subset of CPUs first and let
> them do the preparation steps for groups of still offline CPUs
> concurrently, the actual benefit of doing so is dubious.
>
> The prime example is kernel thread creation, which is implicitely
> serialized on kthreadd.
>
> A simple experiment shows that 4 concurrent workers on 4 different
> CPUs where each is creating 14 * 5 = 70 kernel threads are 5% slower
> than a single worker creating 4 * 14 * 5 = 280 threads.
>
> So we'd need to have multiple kthreadd instances to handle that,
> which would then serialize on tasklist lock and other things.
>
> That aside the preparation phase is also affected by the problem
> below.
>
> 2) Assumptions about hotplug serialization
>
> a) There are quite some assumptions about CPU bringup being fully
> serialized across state transitions. A lot of state callbacks rely
> on that and would require local locking.
>
> Adding that local locking is surely possible, but that has several
> downsides:
>
> - It adds complexity and makes it harder for developers to get
> this correct. The subtle bugs resulting out of that are going
> to be interesting
>
> - Fine grained locking has a charm, but only if the time spent
> for the actual work is larger than the time required for
> serialization and synchronization.
>
> Serializing a callback which takes less than a microsecond and
> then having a large number of CPUs contending on the lock will
> not make it any faster at all. That's a well known issue of
> parallelizing and neither made up nor kernel specific.
>
> b) Some operations definitely require to be protected by the
> cpu_hotplug_lock, especially those which affect cpumasks as the
> masks are guaranteed to be stable in a cpus_read_lock()'ed region.
>
> As this lock cannot be taken in atomic contexts, it's required
> that the control CPU holds the lock write locked across these
> state transitions. And no, we are not making this a spinlock just
> for that and we even can't.
>
> Just slapping a lock into the x86 specific part of the cpumask
> update function does not solve anything. The relevant patch in V17
> is completely useless as it only serializes the actual cpumask/map
> modifications, but all read side users are hosed if the update
> would be moved before the alive synchronization point, i.e. into a
> non hotplug lock protected region.
>
> Even if the hotplug lock would be held accross the whole parallel
> bringup operation then this would still expose all usage of these
> masks and maps in the actual hotplug state callbacks to concurrent
> modifications.
>
> And no, we are not going to expose an architecture specific raw
> spinlock to the hotplug state callbacks, especially not to those
> in generic code.
>
> c) Some cpu_read_lock()'ed regions also expect that there is no CPU
> state transition happening which would modify their local
> state. This would again require local serialization.
>
> 3) The amount of work and churn:
>
> - Analyze the per architecture low level startup functions plus their
> descendant functions and make them ready for concurrency if
> necessary.
>
> - Analyze ~300 hotplug state callbacks and their descendant functions
> and make them ready for concurrency if necessary.
>
> - Analyze all cpus_read_lock()'ed regions and address their
> requirements.
>
> - Rewrite the core code to handle the cpu_hotplug_lock requirements
> only in distinct phases of the state machine.
>
> - Rewrite the core code to handle state callback failure and the
> related rollback in the context of the new rules.
>
> - ...
>
> Even if some people are dedicated enough to do that, it's very
> questionable whether the resulting complexity is justified.
>
> We've spent a serious amount of time to sanitize hotplug and bring it
> into a state where it is correct. This also made it reasonably simple
> for developers to implement hotplug state callbacks without having to
> become hotplug experts.
>
> Breaking this completely up will result in a flood of hard to diagnose
> subtle issues for sure. Who is going to deal with them?
>
> The experience with this series so far does not make me comfortable
> about that thought in any way.
>
>
> Summary
> -------
>
> The obvious and low hanging fruits have to be solved first:
>
> - The CPUID evaluation and related setup mechanisms
>
> - The trace/ringbuffer oddity
>
> - The sched:active oddity for the first sibling on the second socket
>
> - Some other expensive things which I'm not seeing in my test setup due
> to lack of hardware or configuration.
>
> Anything else is pretty much wishful thinking in my opinion.
>
> To be clear. I'm not standing in the way if there is a proper solution,
> but that requires to respect the basic engineering rules:
>
> 1) Correctness first
> 2) Keep it maintainable
> 3) Keep it simple
>
> So far this stuff failed already at #1.
>
> I completely understand why this is important for cloud people, but
> the real question to ask here is what are the actual requirements.
>
> As far as I understand the main goal is to make a (kexec) reboot
> almost invisible to VM tenants.
>
> Now lets look at how this works:
>
> A) Freeze VMs and persist state
> B) kexec into the new kernel
> C) Restore VMs from persistant memory
> D) Thaw VMs
>
> So the key problem is how long it takes to get from #B to #C and finally
> to #D.
>
> As far as I understand #C takes a serious amount of time and cannot be
> parallelized for whatever reasons.
>
> At the same time the number of online CPUs required to restore the VMs
> state is less than the number of online CPUs required to actually
> operate them in #D.
>
> That means it would be good enough to return to userspace with a
> limited number of online CPUs as fast as possible. A certain amount of
> CPUs are going to be busy with restoring the VMs state, i.e. one CPU
> per VM. Some remaining non-busy CPU can bringup the rest of the system
> and the APs in order to be functional for #D, i.e the restore of VM
> operation.
>
> Trying to optimize this purely in kernel space by adding complexity of
> dubious value is simply bogus in my opinion.
>
> It's already possible today to limit the number of CPUs which are
> initially onlined and online the rest later from user space.
>
> There are two issue there:
>
> a) The death by MCE broadcast problem
>
> Quite some (contemporary) x86 CPU generations are affected by
> this:
>
> - MCE can be broadcasted to all CPUs and not only issued locally
> to the CPU which triggered it.
>
> - Any CPU which has CR4.MCE == 0, even if it sits in a wait
> for INIT/SIPI state, will cause an immediate shutdown of the
> machine if a broadcasted MCE is delivered.
>
> b) Do the parallel bringup via sysfs control knob
>
> The per CPU target state interface allows to do that today one
> by one, but it's akward and has quite some overhead.
>
> A knob to online the rest of the not yet onlined present CPUs
> with the benefit of the parallel bringup mechanism is
> missing.
>
> #a) That's a risk to take by the operator.
>
> Even the regular serialized bringup does not protect against this
> issue up to the point where all present CPUs have at least
> initialized CR4.
>
> Limiting the number of APs to online early via the kernel command
> line widens that window and increases the risk further by
> executing user space before all APs have CR4 initialized.
>
> But the same applies to a deferred online mechanism implemented in
> the kernel where some worker brings up the not yet online APs while
> the early online CPUs are already executing user space code.
>
> #b) Is a no brainer to implement on top of this.
>
>
> Conclusion
> ----------
>
> Adding the basic parallel bringup mechanism as provided by this series
> makes a lot of sense. Improving particular issues as pointed out in the
> analysis makes sense too.
>
> But trying to solve an application specific problem fully in the kernel
> with tons of complexity, without exploring straight forward and simple
> approaches first, does not make any sense at all.
>
> Thanks,
>
> tglx
>
> ---
> Documentation/admin-guide/kernel-parameters.txt | 20
> Documentation/core-api/cpu_hotplug.rst | 13
> arch/Kconfig | 23 +
> arch/arm/Kconfig | 1
> arch/arm/include/asm/smp.h | 2
> arch/arm/kernel/smp.c | 18
> arch/arm64/Kconfig | 1
> arch/arm64/include/asm/smp.h | 2
> arch/arm64/kernel/smp.c | 14
> arch/csky/Kconfig | 1
> arch/csky/include/asm/smp.h | 2
> arch/csky/kernel/smp.c | 8
> arch/mips/Kconfig | 1
> arch/mips/cavium-octeon/smp.c | 1
> arch/mips/include/asm/smp-ops.h | 1
> arch/mips/kernel/smp-bmips.c | 1
> arch/mips/kernel/smp-cps.c | 14
> arch/mips/kernel/smp.c | 8
> arch/mips/loongson64/smp.c | 1
> arch/parisc/Kconfig | 1
> arch/parisc/kernel/process.c | 4
> arch/parisc/kernel/smp.c | 7
> arch/riscv/Kconfig | 1
> arch/riscv/include/asm/smp.h | 2
> arch/riscv/kernel/cpu-hotplug.c | 14
> arch/x86/Kconfig | 45 --
> arch/x86/include/asm/apic.h | 5
> arch/x86/include/asm/cpu.h | 5
> arch/x86/include/asm/cpumask.h | 5
> arch/x86/include/asm/processor.h | 1
> arch/x86/include/asm/realmode.h | 3
> arch/x86/include/asm/sev-common.h | 3
> arch/x86/include/asm/smp.h | 26 -
> arch/x86/include/asm/topology.h | 23 -
> arch/x86/include/asm/tsc.h | 2
> arch/x86/kernel/acpi/sleep.c | 9
> arch/x86/kernel/apic/apic.c | 22 -
> arch/x86/kernel/callthunks.c | 4
> arch/x86/kernel/cpu/amd.c | 2
> arch/x86/kernel/cpu/cacheinfo.c | 21
> arch/x86/kernel/cpu/common.c | 50 --
> arch/x86/kernel/cpu/topology.c | 3
> arch/x86/kernel/head_32.S | 14
> arch/x86/kernel/head_64.S | 121 +++++
> arch/x86/kernel/sev.c | 2
> arch/x86/kernel/smp.c | 3
> arch/x86/kernel/smpboot.c | 508 ++++++++----------------
> arch/x86/kernel/topology.c | 98 ----
> arch/x86/kernel/tsc.c | 20
> arch/x86/kernel/tsc_sync.c | 36 -
> arch/x86/power/cpu.c | 37 -
> arch/x86/realmode/init.c | 3
> arch/x86/realmode/rm/trampoline_64.S | 27 +
> arch/x86/xen/enlighten_hvm.c | 11
> arch/x86/xen/smp_hvm.c | 16
> arch/x86/xen/smp_pv.c | 56 +-
> drivers/acpi/processor_idle.c | 4
> include/linux/cpu.h | 4
> include/linux/cpuhotplug.h | 17
> kernel/cpu.c | 397 +++++++++++++++++-
> kernel/smp.c | 2
> kernel/smpboot.c | 163 -------
> 62 files changed, 953 insertions(+), 976 deletions(-)
>
>
Tested with a Xen PV dom0 on an 8 cpu system, no issues found.
Tested-by: Juergen Gross <jgross@suse.com>
Juergen
[-- Attachment #1.1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3149 bytes --]
[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]
[-- Attachment #2: Type: text/plain, Size: 161 bytes --]
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-17 8:35 ` Juergen Gross
0 siblings, 0 replies; 236+ messages in thread
From: Juergen Gross @ 2023-04-17 8:35 UTC (permalink / raw)
To: Thomas Gleixner, LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
[-- Attachment #1.1.1.1: Type: text/plain, Size: 30261 bytes --]
On 15.04.23 01:44, Thomas Gleixner wrote:
> Hi!
>
> This is a complete rework of the parallel bringup patch series (V17)
>
> https://lore.kernel.org/lkml/20230328195758.1049469-1-usama.arif@bytedance.com
>
> to address the issues which were discovered in review:
>
> 1) The X86 microcode loader serialization requirement
>
> https://lore.kernel.org/lkml/87v8iirxun.ffs@tglx
>
> Microcode loading on HT enabled X86 CPUs requires that the microcode is
> loaded on the primary thread. The sibling thread(s) must be in
> quiescent state; either looping in a place which is aware of potential
> changes by the microcode update (see late loading) or in fully quiescent
> state, i.e. waiting for INIT/SIPI.
>
> This is required by hardware/firmware on Intel. Aside of that it's a
> vendor independent software correctness issue. Assume the following
> sequence:
>
> CPU1.0 CPU1.1
> CPUID($A)
> Load microcode.
> Changes CPUID($A, $B)
> CPUID($B)
>
> CPU1.1 makes a decision on $A and $B which might be inconsistent due
> to the microcode update.
>
> The solution for this is to bringup the primary threads first and after
> that the siblings. Loading microcode on the siblings is a NOOP on Intel
> and on AMD it is guaranteed to only modify thread local state.
>
> This ensures that the APs can load microcode before reaching the alive
> synchronization point w/o doing any further x86 specific
> synchronization between the core siblings.
>
> 2) The general design issues discussed in V16
>
> https://lore.kernel.org/lkml/87pm8y6yme.ffs@tglx
>
> The previous parallel bringup patches just glued this mechanism into
> the existing code without a deeper analysis of the synchronization
> mechanisms and without generalizing it so that the control logic is
> mostly in the core code and not made an architecture specific tinker
> space.
>
> Much of that had been pointed out 2 years ago in the discussions about
> the early versions of parallel bringup already.
>
>
> The series is based on:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip x86/apic
>
> and also available from git:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git hotplug
>
>
> Background
> ----------
>
> The reason why people are interested in parallel bringup is to shorten
> the (kexec) reboot time of cloud servers to reduce the downtime of the
> VM tenants. There are obviously other interesting use cases for this
> like VM startup time, embedded devices...
>
> The current fully serialized bringup does the following per AP:
>
> 1) Prepare callbacks (allocate, intialize, create threads)
> 2) Kick the AP alive (e.g. INIT/SIPI on x86)
> 3) Wait for the AP to report alive state
> 4) Let the AP continue through the atomic bringup
> 5) Let the AP run the threaded bringup to full online state
>
> There are two significant delays:
>
> #3 The time for an AP to report alive state in start_secondary() on x86
> has been measured in the range between 350us and 3.5ms depending on
> vendor and CPU type, BIOS microcode size etc.
>
> #4 The atomic bringup does the microcode update. This has been measured
> to take up to ~8ms on the primary threads depending on the microcode
> patch size to apply.
>
> On a two socket SKL server with 56 cores (112 threads) the boot CPU spends
> on current mainline about 800ms busy waiting for the APs to come up and
> apply microcode. That's more than 80% of the actual onlining procedure.
>
> By splitting the actual bringup mechanism into two parts this can be
> reduced to waiting for the first AP to report alive or if the system is
> large enough the first AP is already waiting when the boot CPU finished the
> wake-up of the last AP.
>
>
> The actual solution comes in several parts
> ------------------------------------------
>
> 1) [P 1-2] General cleanups (init annotations, kernel doc...)
>
> 2) [P 3] The obvious
>
> Avoid pointless delay calibration when TSC is synchronized across
> sockets. That removes a whopping 100ms delay for the first CPU of a
> socket. This is an improvement independent of parallel bringup and had
> been discussed two years ago already.
>
> 2) [P 3-6] Removal of the CPU0 hotplug hack.
>
> This was added 11 years ago with the promise to make this a real
> hardware mechanism, but that never materialized. As physical CPU
> hotplug is not really supported and the physical unplugging of CPU0
> never materialized there is no reason to keep this cruft around. It's
> just maintenance ballast for no value and the removal makes
> implementing the parallel bringup feature way simpler.
>
> 3) [P 7-16] Cleanup of the existing bringup mechanism:
>
> a) Code reorganisation so that the general hotplug specific code is
> in smpboot.c and not sprinkled all over the place
>
> b) Decouple MTRR/PAT initialization from smp_callout_mask to prepare
> for replacing that mask with a hotplug core code synchronization
> mechanism.
>
> c) Make TSC synchronization function call based so that the control CPU
> does not have to busy wait for nothing if synchronization is not
> required.
>
> d) Remove the smp_callin_mask synchronization point as its not longer
> required due to #3c.
>
> e) Rework the sparse_irq_lock held region in the core code so that the
> next polling synchronization point in the x86 code can be removed to.
>
> f) Due to #3e it's not longer required to spin wait for the AP to set
> it's online bit. Remove wait_cpu_online() and the XENPV
> counterpart. So the control CPU can directly wait for the online
> idle completion by the AP and free the control CPU up for other
> work.
>
> This reduces the synchronization points in the x86 code to one, which
> is the AP alive one. This synchronization will be moved to core
> infrastructure in the next section.
>
> 4) [P 17-27] Replace the disconnected CPU state tracking
>
> The extra CPU state tracking which is used by a few architectures is
> completely separate from the CPU hotplug core code.
>
> Replacing it by a variant integrated in the core hotplug machinery
> allows to reduce architecture specific code and provides a generic
> synchronization mechanism for (parallel) CPU bringup/teardown.
>
> - Convert x86 over and replace the AP alive synchronization on x86 with
> the core variant which removes the remaining x86 hotplug
> synchronization masks.
>
> - Convert the other architectures usage and remove the old interface
> and code.
>
> 5) [P 28-30] Split the bringup into two steps
>
> First step invokes the wakeup function on the BP, e.g. SIPI/STARTUP on
> x86. The second one waits on the BP for the AP to report alive and
> releases it for the complete onlining.
>
> As the hotplug state machine allows partial bringup this allows later
> to kick all APs alive in a first iteration and then bring them up
> completely one by one afterwards.
>
> 6) [P 31] Switch the primary thread detection to a cpumask
>
> This makes the parallel bringup a simple cpumask based mechanism
> without tons of conditionals and checks for primary threads.
>
> 7) [P 32] Implement the parallel bringup core code
>
> The parallel bringup looks like this:
>
> 1) Bring up the primary SMT threads to the CPUHP_KICK_AP_ALIVE step
> one by one
>
> 2) Bring up the primary SMT threads to the CPUHP_ONLINE step one by
> one
>
> 3) Bring up the secondary SMT threads to the CPUHP_KICK_AP_ALIVE
> step one by one
>
> 4) Bring up the secondary SMT threads to the CPUHP_ONLINE
> step one by one
>
> In case that SMT is not supported this is obviously reduced to step #1
> and #2.
>
> 8) [P 33-37] Prepare X86 for parallel bringup and enable it
>
>
> Caveats
> -------
>
> The non X86 changes have been all compile tested. Boot and runtime
> testing has only be done on a few real hardware platforms and qemu as
> available. That definitely needs some help from the people who have
> these systems at their fingertips.
>
>
> Results and analysis
> --------------------
>
> Here are numbers for a dual socket SKL 56 cores/ 112 threads machine. All
> numbers in milliseconds. The time measured is the time which the cpu_up()
> call takes for each CPU and phase. It's not exact as the system is already
> scheduling, handling interrupts and soft interrupts, which is obviously
> skewing the picture slightly.
>
> Baseline tip tree x86/apic branch.
>
> total avg/CPU min max
> total : 912.081 8.217 3.720 113.271
>
> The max of 100ms is due to the silly delay calibration for the second
> socket which takes 100ms and was eliminated first. Also the other initial
> cleanups and improvements take some time away.
>
> So the real baseline becomes:
>
> total avg/CPU min max
> total : 785.960 7.081 3.752 36.098
>
> The max here is on the first CPU of the second socket. 20ms of that is due
> to TSC synchronization and an extra 2ms to react on the SIPI.
>
> With parallel bootup enabled this becomes:
>
> total avg/CPU min max
> prepare: 39.108 0.352 0.238 0.883
> online : 45.166 0.407 0.170 20.357
> total : 84.274 0.759 0.408 21.240
>
> That's a factor ~9.3 reduction on average.
>
> Looking at the 27 primary threads of socket 0 then this becomes even more
> interesting:
>
> total avg/CPU min max
> total : 325.764 12.065 11.981 14.125
>
> versus:
> total avg/CPU min max
> prepare: 8.945 0.331 0.238 0.834
> online : 4.830 0.179 0.170 0.212
> total : 13.775 0.510 0.408 1.046
>
> So the reduction factor is ~23.5 here. That's mostly because the 20ms TSC
> sync is not skewing the picture.
>
> For all 55 primaries, i.e with the 20ms TSC sync extra for socket 1 this
> becomes:
>
> total avg/CPU min max
> total : 685.489 12.463 11.975 36.098
>
> versus:
>
> total avg/CPU min max
> prepare: 19.080 0.353 0.238 0.883
> online : 30.283 0.561 0.170 20.357
> total : 49.363 0.914 0.408 21.240
>
> The TSC sync reduces the win to a factor of ~13.8
>
> With 'tsc=reliable' on the command line the socket sync is disabled which
> brings it back to the socket 0 numbers:
>
> total avg/CPU min max
> prepare: 18.970 0.351 0.231 0.874
> online : 10.328 0.191 0.169 0.358
> total : 29.298 0.543 0.400 1.232
>
> Now looking at the secondary threads only:
>
> total avg/CPU min max
> total : 100.471 1.794 0.375 4.745
>
> versus:
> total avg/CPU min max
> prepare: 19.753 0.353 0.257 0.512
> online : 14.671 0.262 0.179 3.461
> total : 34.424 0.615 0.436 3.973
>
> Still a factor of ~3.
>
> The average on the secondaries for the serialized bringup is significantly
> lower than for the primaries because the SIPI response time is shorter and
> the microcode update takes no time.
>
> This varies wildly with the system, whether microcode in BIOS is already up
> to date, how big the microcode patch is and how long the INIT/SIPI response
> time is. On an AMD Zen3 machine INIT/SIPI response time is amazingly fast
> (350us), but then it lacks TSC_ADJUST and does a two millisecond TSC sync
> test for _every_ AP. All of this sucks...
>
>
> Possible further enhancements
> -----------------------------
>
> It's definitely worthwhile to look into reducing the cross socket TSC sync
> test time. It's probably safe enough to use 5ms or even 2ms instead of 20ms
> on systems with TSC_ADJUST and a few other 'TSC is sane' indicators. Moving
> it out of the hotplug path is eventually possible, but that needs some deep
> thoughts.
>
> Let's take the TSC sync out of the picture by adding 'tsc=reliable" to the
> kernel command line. So the bringup of 111 APs takes:
>
> total avg/CPU min max
> prepare: 38.936 0.351 0.231 0.874
> online : 25.231 0.227 0.169 3.465
> total : 64.167 0.578 0.400 4.339
>
> Some of the outliers are not necessarily in the state callbacks as the
> system is already scheduling and handles interrupts and soft
> interrupts. Haven't analyzed that yet in detail.
>
> In the prepare stage which runs on the control CPU the larger steps are:
>
> smpcfd:prepare 16us avg/CPU
> threads:prepare 98us avg/CPU
> workqueue:prepare 43us avg/CPU
> trace/RB:prepare 135us avg/CPU
>
> The trace ringbuffer initialization allocates 354 pages and 354 control
> structures one by one. That probably should allocate a large page and an
> array of control structures and work from there. I'm sure that would reduce
> this significantly. Steven?
>
> smpcfd does just a percpu allocation. No idea why that takes that long.
>
> Vs. threads and workqueues. David thought about spreading out the
> preparation work and do it really in parallel. That's a nice idea, but the
> threads and workqueue prepare steps are self serializing. The workqueue one
> has a global mutex and aside of that both steps create kernel threads which
> implicitely serialize on kthreadd. alloc_percpu(), which is used by
> smpcfd:prepare is also globally serialized.
>
> The rest of the prepare steps is pretty much in the single digit
> microseconds range.
>
> On the AP side it should be possible to move some of the initialization
> steps before the alive synchronization point, but that really needs a lot
> of analysis whether the functions are safe to invoke that early and outside
> of the cpu_hotplug_lock held region for the case of two stage parallel
> bringup; see below.
>
> The largest part is:
>
> identify_secondary_cpu() 99us avg/CPU
>
> Inside of identify_secondary_cpu() the largest offender:
>
> mcheck_init() 73us avg/CPU
>
> This part is definitly worth to be looked at whether it can be at least
> partially moved to the early startup code before the alive
> synchronization point. There's a lot of deep analysis required and
> ideally we just rewrite the whole CPUID evaluation trainwreck
> completely.
>
> The rest of the AP side is low single digit microseconds except of:
>
> perf/x86:starting 14us avg/CPU
>
> smpboot/threads:online 13us avg/CPU
> workqueue:online 17us avg/CPU
> mm/vmstat:online 17us avg/CPU
> sched:active 30us avg/CPU
>
> sched:active is special. Onlining the first secondary HT thread on the
> second socket creates a 3.2ms outlier which skews the whole picture. That's
> caused by enabling the static key sched_smt_present which patches the world
> and some more. For all other APs this is really in the 1us range. This
> definitely could be postponed during bootup like the scheduler domain
> rebuild is done after the bringup. But that's still fully serialized and
> single threaded and obviously could be done later in the context of async
> parallel init. It's unclear why this is different with the fully serialized
> bringup where it takes significantly less time, but that's something which
> needs to be investigated.
>
>
> Is truly parallel bringup feasible?
> -----------------------------------
>
> In theory yes, realistically no. Why?
>
> 1) The preparation phase
>
> Allocating memory, creating threads for the to be brought up CPU must
> obviously happen on an already online CPU.
>
> While it would be possible to bring up a subset of CPUs first and let
> them do the preparation steps for groups of still offline CPUs
> concurrently, the actual benefit of doing so is dubious.
>
> The prime example is kernel thread creation, which is implicitely
> serialized on kthreadd.
>
> A simple experiment shows that 4 concurrent workers on 4 different
> CPUs where each is creating 14 * 5 = 70 kernel threads are 5% slower
> than a single worker creating 4 * 14 * 5 = 280 threads.
>
> So we'd need to have multiple kthreadd instances to handle that,
> which would then serialize on tasklist lock and other things.
>
> That aside the preparation phase is also affected by the problem
> below.
>
> 2) Assumptions about hotplug serialization
>
> a) There are quite some assumptions about CPU bringup being fully
> serialized across state transitions. A lot of state callbacks rely
> on that and would require local locking.
>
> Adding that local locking is surely possible, but that has several
> downsides:
>
> - It adds complexity and makes it harder for developers to get
> this correct. The subtle bugs resulting out of that are going
> to be interesting
>
> - Fine grained locking has a charm, but only if the time spent
> for the actual work is larger than the time required for
> serialization and synchronization.
>
> Serializing a callback which takes less than a microsecond and
> then having a large number of CPUs contending on the lock will
> not make it any faster at all. That's a well known issue of
> parallelizing and neither made up nor kernel specific.
>
> b) Some operations definitely require to be protected by the
> cpu_hotplug_lock, especially those which affect cpumasks as the
> masks are guaranteed to be stable in a cpus_read_lock()'ed region.
>
> As this lock cannot be taken in atomic contexts, it's required
> that the control CPU holds the lock write locked across these
> state transitions. And no, we are not making this a spinlock just
> for that and we even can't.
>
> Just slapping a lock into the x86 specific part of the cpumask
> update function does not solve anything. The relevant patch in V17
> is completely useless as it only serializes the actual cpumask/map
> modifications, but all read side users are hosed if the update
> would be moved before the alive synchronization point, i.e. into a
> non hotplug lock protected region.
>
> Even if the hotplug lock would be held accross the whole parallel
> bringup operation then this would still expose all usage of these
> masks and maps in the actual hotplug state callbacks to concurrent
> modifications.
>
> And no, we are not going to expose an architecture specific raw
> spinlock to the hotplug state callbacks, especially not to those
> in generic code.
>
> c) Some cpu_read_lock()'ed regions also expect that there is no CPU
> state transition happening which would modify their local
> state. This would again require local serialization.
>
> 3) The amount of work and churn:
>
> - Analyze the per architecture low level startup functions plus their
> descendant functions and make them ready for concurrency if
> necessary.
>
> - Analyze ~300 hotplug state callbacks and their descendant functions
> and make them ready for concurrency if necessary.
>
> - Analyze all cpus_read_lock()'ed regions and address their
> requirements.
>
> - Rewrite the core code to handle the cpu_hotplug_lock requirements
> only in distinct phases of the state machine.
>
> - Rewrite the core code to handle state callback failure and the
> related rollback in the context of the new rules.
>
> - ...
>
> Even if some people are dedicated enough to do that, it's very
> questionable whether the resulting complexity is justified.
>
> We've spent a serious amount of time to sanitize hotplug and bring it
> into a state where it is correct. This also made it reasonably simple
> for developers to implement hotplug state callbacks without having to
> become hotplug experts.
>
> Breaking this completely up will result in a flood of hard to diagnose
> subtle issues for sure. Who is going to deal with them?
>
> The experience with this series so far does not make me comfortable
> about that thought in any way.
>
>
> Summary
> -------
>
> The obvious and low hanging fruits have to be solved first:
>
> - The CPUID evaluation and related setup mechanisms
>
> - The trace/ringbuffer oddity
>
> - The sched:active oddity for the first sibling on the second socket
>
> - Some other expensive things which I'm not seeing in my test setup due
> to lack of hardware or configuration.
>
> Anything else is pretty much wishful thinking in my opinion.
>
> To be clear. I'm not standing in the way if there is a proper solution,
> but that requires to respect the basic engineering rules:
>
> 1) Correctness first
> 2) Keep it maintainable
> 3) Keep it simple
>
> So far this stuff failed already at #1.
>
> I completely understand why this is important for cloud people, but
> the real question to ask here is what are the actual requirements.
>
> As far as I understand the main goal is to make a (kexec) reboot
> almost invisible to VM tenants.
>
> Now lets look at how this works:
>
> A) Freeze VMs and persist state
> B) kexec into the new kernel
> C) Restore VMs from persistant memory
> D) Thaw VMs
>
> So the key problem is how long it takes to get from #B to #C and finally
> to #D.
>
> As far as I understand #C takes a serious amount of time and cannot be
> parallelized for whatever reasons.
>
> At the same time the number of online CPUs required to restore the VMs
> state is less than the number of online CPUs required to actually
> operate them in #D.
>
> That means it would be good enough to return to userspace with a
> limited number of online CPUs as fast as possible. A certain amount of
> CPUs are going to be busy with restoring the VMs state, i.e. one CPU
> per VM. Some remaining non-busy CPU can bringup the rest of the system
> and the APs in order to be functional for #D, i.e the restore of VM
> operation.
>
> Trying to optimize this purely in kernel space by adding complexity of
> dubious value is simply bogus in my opinion.
>
> It's already possible today to limit the number of CPUs which are
> initially onlined and online the rest later from user space.
>
> There are two issue there:
>
> a) The death by MCE broadcast problem
>
> Quite some (contemporary) x86 CPU generations are affected by
> this:
>
> - MCE can be broadcasted to all CPUs and not only issued locally
> to the CPU which triggered it.
>
> - Any CPU which has CR4.MCE == 0, even if it sits in a wait
> for INIT/SIPI state, will cause an immediate shutdown of the
> machine if a broadcasted MCE is delivered.
>
> b) Do the parallel bringup via sysfs control knob
>
> The per CPU target state interface allows to do that today one
> by one, but it's akward and has quite some overhead.
>
> A knob to online the rest of the not yet onlined present CPUs
> with the benefit of the parallel bringup mechanism is
> missing.
>
> #a) That's a risk to take by the operator.
>
> Even the regular serialized bringup does not protect against this
> issue up to the point where all present CPUs have at least
> initialized CR4.
>
> Limiting the number of APs to online early via the kernel command
> line widens that window and increases the risk further by
> executing user space before all APs have CR4 initialized.
>
> But the same applies to a deferred online mechanism implemented in
> the kernel where some worker brings up the not yet online APs while
> the early online CPUs are already executing user space code.
>
> #b) Is a no brainer to implement on top of this.
>
>
> Conclusion
> ----------
>
> Adding the basic parallel bringup mechanism as provided by this series
> makes a lot of sense. Improving particular issues as pointed out in the
> analysis makes sense too.
>
> But trying to solve an application specific problem fully in the kernel
> with tons of complexity, without exploring straight forward and simple
> approaches first, does not make any sense at all.
>
> Thanks,
>
> tglx
>
> ---
> Documentation/admin-guide/kernel-parameters.txt | 20
> Documentation/core-api/cpu_hotplug.rst | 13
> arch/Kconfig | 23 +
> arch/arm/Kconfig | 1
> arch/arm/include/asm/smp.h | 2
> arch/arm/kernel/smp.c | 18
> arch/arm64/Kconfig | 1
> arch/arm64/include/asm/smp.h | 2
> arch/arm64/kernel/smp.c | 14
> arch/csky/Kconfig | 1
> arch/csky/include/asm/smp.h | 2
> arch/csky/kernel/smp.c | 8
> arch/mips/Kconfig | 1
> arch/mips/cavium-octeon/smp.c | 1
> arch/mips/include/asm/smp-ops.h | 1
> arch/mips/kernel/smp-bmips.c | 1
> arch/mips/kernel/smp-cps.c | 14
> arch/mips/kernel/smp.c | 8
> arch/mips/loongson64/smp.c | 1
> arch/parisc/Kconfig | 1
> arch/parisc/kernel/process.c | 4
> arch/parisc/kernel/smp.c | 7
> arch/riscv/Kconfig | 1
> arch/riscv/include/asm/smp.h | 2
> arch/riscv/kernel/cpu-hotplug.c | 14
> arch/x86/Kconfig | 45 --
> arch/x86/include/asm/apic.h | 5
> arch/x86/include/asm/cpu.h | 5
> arch/x86/include/asm/cpumask.h | 5
> arch/x86/include/asm/processor.h | 1
> arch/x86/include/asm/realmode.h | 3
> arch/x86/include/asm/sev-common.h | 3
> arch/x86/include/asm/smp.h | 26 -
> arch/x86/include/asm/topology.h | 23 -
> arch/x86/include/asm/tsc.h | 2
> arch/x86/kernel/acpi/sleep.c | 9
> arch/x86/kernel/apic/apic.c | 22 -
> arch/x86/kernel/callthunks.c | 4
> arch/x86/kernel/cpu/amd.c | 2
> arch/x86/kernel/cpu/cacheinfo.c | 21
> arch/x86/kernel/cpu/common.c | 50 --
> arch/x86/kernel/cpu/topology.c | 3
> arch/x86/kernel/head_32.S | 14
> arch/x86/kernel/head_64.S | 121 +++++
> arch/x86/kernel/sev.c | 2
> arch/x86/kernel/smp.c | 3
> arch/x86/kernel/smpboot.c | 508 ++++++++----------------
> arch/x86/kernel/topology.c | 98 ----
> arch/x86/kernel/tsc.c | 20
> arch/x86/kernel/tsc_sync.c | 36 -
> arch/x86/power/cpu.c | 37 -
> arch/x86/realmode/init.c | 3
> arch/x86/realmode/rm/trampoline_64.S | 27 +
> arch/x86/xen/enlighten_hvm.c | 11
> arch/x86/xen/smp_hvm.c | 16
> arch/x86/xen/smp_pv.c | 56 +-
> drivers/acpi/processor_idle.c | 4
> include/linux/cpu.h | 4
> include/linux/cpuhotplug.h | 17
> kernel/cpu.c | 397 +++++++++++++++++-
> kernel/smp.c | 2
> kernel/smpboot.c | 163 -------
> 62 files changed, 953 insertions(+), 976 deletions(-)
>
>
Tested with a Xen PV dom0 on an 8 cpu system, no issues found.
Tested-by: Juergen Gross <jgross@suse.com>
Juergen
[-- Attachment #1.1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3149 bytes --]
[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 495 bytes --]
[-- Attachment #2: Type: text/plain, Size: 176 bytes --]
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-17 10:30 ` Peter Zijlstra
-1 siblings, 0 replies; 236+ messages in thread
From: Peter Zijlstra @ 2023-04-17 10:30 UTC (permalink / raw)
To: Thomas Gleixner
Cc: LKML, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Sat, Apr 15, 2023 at 01:44:13AM +0200, Thomas Gleixner wrote:
> Background
> ----------
>
> The reason why people are interested in parallel bringup is to shorten
> the (kexec) reboot time of cloud servers to reduce the downtime of the
> VM tenants. There are obviously other interesting use cases for this
> like VM startup time, embedded devices...
...
> There are two issue there:
>
> a) The death by MCE broadcast problem
>
> Quite some (contemporary) x86 CPU generations are affected by
> this:
>
> - MCE can be broadcasted to all CPUs and not only issued locally
> to the CPU which triggered it.
>
> - Any CPU which has CR4.MCE == 0, even if it sits in a wait
> for INIT/SIPI state, will cause an immediate shutdown of the
> machine if a broadcasted MCE is delivered.
When doing kexec, CR4.MCE should already have been set to 1 by the prior
kernel, no?
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-17 10:30 ` Peter Zijlstra
0 siblings, 0 replies; 236+ messages in thread
From: Peter Zijlstra @ 2023-04-17 10:30 UTC (permalink / raw)
To: Thomas Gleixner
Cc: LKML, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Sat, Apr 15, 2023 at 01:44:13AM +0200, Thomas Gleixner wrote:
> Background
> ----------
>
> The reason why people are interested in parallel bringup is to shorten
> the (kexec) reboot time of cloud servers to reduce the downtime of the
> VM tenants. There are obviously other interesting use cases for this
> like VM startup time, embedded devices...
...
> There are two issue there:
>
> a) The death by MCE broadcast problem
>
> Quite some (contemporary) x86 CPU generations are affected by
> this:
>
> - MCE can be broadcasted to all CPUs and not only issued locally
> to the CPU which triggered it.
>
> - Any CPU which has CR4.MCE == 0, even if it sits in a wait
> for INIT/SIPI state, will cause an immediate shutdown of the
> machine if a broadcasted MCE is delivered.
When doing kexec, CR4.MCE should already have been set to 1 by the prior
kernel, no?
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-17 10:30 ` Peter Zijlstra
0 siblings, 0 replies; 236+ messages in thread
From: Peter Zijlstra @ 2023-04-17 10:30 UTC (permalink / raw)
To: Thomas Gleixner
Cc: LKML, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Sat, Apr 15, 2023 at 01:44:13AM +0200, Thomas Gleixner wrote:
> Background
> ----------
>
> The reason why people are interested in parallel bringup is to shorten
> the (kexec) reboot time of cloud servers to reduce the downtime of the
> VM tenants. There are obviously other interesting use cases for this
> like VM startup time, embedded devices...
...
> There are two issue there:
>
> a) The death by MCE broadcast problem
>
> Quite some (contemporary) x86 CPU generations are affected by
> this:
>
> - MCE can be broadcasted to all CPUs and not only issued locally
> to the CPU which triggered it.
>
> - Any CPU which has CR4.MCE == 0, even if it sits in a wait
> for INIT/SIPI state, will cause an immediate shutdown of the
> machine if a broadcasted MCE is delivered.
When doing kexec, CR4.MCE should already have been set to 1 by the prior
kernel, no?
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
2023-04-17 10:30 ` Peter Zijlstra
(?)
@ 2023-04-17 10:44 ` Andrew Cooper
-1 siblings, 0 replies; 236+ messages in thread
From: Andrew Cooper @ 2023-04-17 10:44 UTC (permalink / raw)
To: Peter Zijlstra, Thomas Gleixner
Cc: LKML, x86, David Woodhouse, Brian Gerst, Arjan van de Veen,
Paolo Bonzini, Paul McKenney, Tom Lendacky, Sean Christopherson,
Oleksandr Natalenko, Paul Menzel, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Juergen Gross,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On 17/04/2023 11:30 am, Peter Zijlstra wrote:
> On Sat, Apr 15, 2023 at 01:44:13AM +0200, Thomas Gleixner wrote:
>
>> Background
>> ----------
>>
>> The reason why people are interested in parallel bringup is to shorten
>> the (kexec) reboot time of cloud servers to reduce the downtime of the
>> VM tenants. There are obviously other interesting use cases for this
>> like VM startup time, embedded devices...
> ...
>
>> There are two issue there:
>>
>> a) The death by MCE broadcast problem
>>
>> Quite some (contemporary) x86 CPU generations are affected by
>> this:
>>
>> - MCE can be broadcasted to all CPUs and not only issued locally
>> to the CPU which triggered it.
>>
>> - Any CPU which has CR4.MCE == 0, even if it sits in a wait
>> for INIT/SIPI state, will cause an immediate shutdown of the
>> machine if a broadcasted MCE is delivered.
> When doing kexec, CR4.MCE should already have been set to 1 by the prior
> kernel, no?
No(ish). Purgatory can't take #MC, or NMIs for that matter.
It's cleaner to explicitly disable CR4.MCE and let the system reset
(with all the MC banks properly preserved), than it is to take #MC while
the IDT isn't in sync with the handlers, and wander off into the weeds.
~Andrew
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-17 10:44 ` Andrew Cooper
0 siblings, 0 replies; 236+ messages in thread
From: Andrew Cooper @ 2023-04-17 10:44 UTC (permalink / raw)
To: Peter Zijlstra, Thomas Gleixner
Cc: LKML, x86, David Woodhouse, Brian Gerst, Arjan van de Veen,
Paolo Bonzini, Paul McKenney, Tom Lendacky, Sean Christopherson,
Oleksandr Natalenko, Paul Menzel, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Juergen Gross,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On 17/04/2023 11:30 am, Peter Zijlstra wrote:
> On Sat, Apr 15, 2023 at 01:44:13AM +0200, Thomas Gleixner wrote:
>
>> Background
>> ----------
>>
>> The reason why people are interested in parallel bringup is to shorten
>> the (kexec) reboot time of cloud servers to reduce the downtime of the
>> VM tenants. There are obviously other interesting use cases for this
>> like VM startup time, embedded devices...
> ...
>
>> There are two issue there:
>>
>> a) The death by MCE broadcast problem
>>
>> Quite some (contemporary) x86 CPU generations are affected by
>> this:
>>
>> - MCE can be broadcasted to all CPUs and not only issued locally
>> to the CPU which triggered it.
>>
>> - Any CPU which has CR4.MCE == 0, even if it sits in a wait
>> for INIT/SIPI state, will cause an immediate shutdown of the
>> machine if a broadcasted MCE is delivered.
> When doing kexec, CR4.MCE should already have been set to 1 by the prior
> kernel, no?
No(ish). Purgatory can't take #MC, or NMIs for that matter.
It's cleaner to explicitly disable CR4.MCE and let the system reset
(with all the MC banks properly preserved), than it is to take #MC while
the IDT isn't in sync with the handlers, and wander off into the weeds.
~Andrew
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-17 10:44 ` Andrew Cooper
0 siblings, 0 replies; 236+ messages in thread
From: Andrew Cooper @ 2023-04-17 10:44 UTC (permalink / raw)
To: Peter Zijlstra, Thomas Gleixner
Cc: LKML, x86, David Woodhouse, Brian Gerst, Arjan van de Veen,
Paolo Bonzini, Paul McKenney, Tom Lendacky, Sean Christopherson,
Oleksandr Natalenko, Paul Menzel, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Juergen Gross,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On 17/04/2023 11:30 am, Peter Zijlstra wrote:
> On Sat, Apr 15, 2023 at 01:44:13AM +0200, Thomas Gleixner wrote:
>
>> Background
>> ----------
>>
>> The reason why people are interested in parallel bringup is to shorten
>> the (kexec) reboot time of cloud servers to reduce the downtime of the
>> VM tenants. There are obviously other interesting use cases for this
>> like VM startup time, embedded devices...
> ...
>
>> There are two issue there:
>>
>> a) The death by MCE broadcast problem
>>
>> Quite some (contemporary) x86 CPU generations are affected by
>> this:
>>
>> - MCE can be broadcasted to all CPUs and not only issued locally
>> to the CPU which triggered it.
>>
>> - Any CPU which has CR4.MCE == 0, even if it sits in a wait
>> for INIT/SIPI state, will cause an immediate shutdown of the
>> machine if a broadcasted MCE is delivered.
> When doing kexec, CR4.MCE should already have been set to 1 by the prior
> kernel, no?
No(ish). Purgatory can't take #MC, or NMIs for that matter.
It's cleaner to explicitly disable CR4.MCE and let the system reset
(with all the MC banks properly preserved), than it is to take #MC while
the IDT isn't in sync with the handlers, and wander off into the weeds.
~Andrew
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-17 11:19 ` Paul Menzel
-1 siblings, 0 replies; 236+ messages in thread
From: Paul Menzel @ 2023-04-17 11:19 UTC (permalink / raw)
To: Thomas Gleixner
Cc: linux-kernel, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
[-- Attachment #1: Type: text/plain, Size: 1532 bytes --]
Dear Thomas,
Am 15.04.23 um 01:44 schrieb Thomas Gleixner:
> This is a complete rework of the parallel bringup patch series (V17)
>
> https://lore.kernel.org/lkml/20230328195758.1049469-1-usama.arif@bytedance.com
>
> to address the issues which were discovered in review:
[…]
Thank you very much for your rework.
I tested this on the ASUS F2A85-M PRO, and get a delay of ten seconds.
```
[…]
[ 0.258193] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD
Graphics (family: 0x15, model: 0x13, stepping: 0x1)
[…]
[ 0.259329] smp: Bringing up secondary CPUs ...
[ 0.259527] x86: Booting SMP configuration:
[ 0.259528] .... node #0, CPUs: #1
[ 0.261007] After schedule_preempt_disabled
[ 10.260990] CPU1 failed to report alive state
[ 10.261070] smp: Brought up 1 node, 1 CPU
[ 10.261073] smpboot: Max logical packages: 2
[ 10.261074] smpboot: Total of 1 processors activated (7800.54 BogoMIPS)
[ 10.261601] devtmpfs: initialized
[ 10.261697] x86/mm: Memory block size: 128MB
```
This delay has been there with v6.3-rc6-46-gde4664485abbc and some
custom (printk) patches on top and merging dwmw2/parallel-6.2-rc3-v16
into it. I only tested this. I think dwmw2/parallel-6.2-v17 failed to
build for me, when trying to merge it into Linus’ master version at that
time. I didn’t come around to report it, and you posted your rework, so
I am replying here.
I am going to try your branch directly in the next days, but just wanted
to report back already.
Kind regards,
Paul
[-- Attachment #2: kodi-linux-6.3-rc6-smp-tglx.txt --]
[-- Type: text/plain, Size: 61769 bytes --]
[ 0.000000] Linux version 6.3.0-rc6-00311-gde8224969f66 (root@bf16f3646a84) (gcc (Debian 11.2.0-12) 11.2.0, GNU ld (GNU Binutils for Debian) 2.40) #446 SMP PREEMPT_DYNAMIC Sat Apr 15 14:12:29 UTC 2023
[ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc6-00311-gde8224969f66 root=/dev/sda3 rw quiet noisapnp cryptomgr.notests ipv6.disable_ipv6=1 selinux=0
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256
[ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format.
[ 0.000000] signal: max sigframe size: 1776
[ 0.000000] BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
[ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005fe4cfff] usable
[ 0.000000] BIOS-e820: [mem 0x000000005fe4d000-0x000000007fffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000f8000000-0x00000000fbffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000fec10000-0x00000000fec10fff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000017effffff] usable
[ 0.000000] NX (Execute Disable) protection: active
[ 0.000000] SMBIOS 3.0.0 present.
[ 0.000000] DMI: ASUS F2A85-M_PRO/F2A85-M_PRO, BIOS 4.18-9-g9917d2d915 04/17/2023
[ 0.000000] tsc: Fast TSC calibration using PIT
[ 0.000000] tsc: Initial usec timer 6035615
[ 0.000000] tsc: Detected 3900.273 MHz processor
[ 0.000756] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
[ 0.000759] e820: remove [mem 0x000a0000-0x000fffff] usable
[ 0.000763] last_pfn = 0x17f000 max_arch_pfn = 0x400000000
[ 0.000768] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT
[ 0.000942] last_pfn = 0x5fe4d max_arch_pfn = 0x400000000
[ 0.004000] Using GB pages for direct mapping
[ 0.004000] ACPI: Early table checksum verification disabled
[ 0.004000] ACPI: RSDP 0x00000000000F6830 000024 (v02 COREv4)
[ 0.004000] ACPI: XSDT 0x000000005FE5A0E0 000074 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: FACP 0x000000005FE5BBC0 000114 (v06 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: DSDT 0x000000005FE5A280 00193A (v02 COREv4 COREBOOT 00010001 INTL 20200925)
[ 0.004000] ACPI: FACS 0x000000005FE5A240 000040
[ 0.004000] ACPI: FACS 0x000000005FE5A240 000040
[ 0.004000] ACPI: SSDT 0x000000005FE5BCE0 00008A (v02 COREv4 COREBOOT 0000002A CORE 20200925)
[ 0.004000] ACPI: MCFG 0x000000005FE5BD70 00003C (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: APIC 0x000000005FE5BDB0 000062 (v03 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: HPET 0x000000005FE5BE20 000038 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: HEST 0x000000005FE5BE60 0001D0 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: IVRS 0x000000005FE5C030 000070 (v02 AMD AMDIOMMU 00000001 AMD 00000000)
[ 0.004000] ACPI: SSDT 0x000000005FE5C0A0 00051F (v02 AMD ALIB 00000001 MSFT 04000000)
[ 0.004000] ACPI: SSDT 0x000000005FE5C5C0 0006B2 (v01 AMD POWERNOW 00000001 AMD 00000001)
[ 0.004000] ACPI: VFCT 0x000000005FE5CC80 00F269 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: Reserving FACP table memory at [mem 0x5fe5bbc0-0x5fe5bcd3]
[ 0.004000] ACPI: Reserving DSDT table memory at [mem 0x5fe5a280-0x5fe5bbb9]
[ 0.004000] ACPI: Reserving FACS table memory at [mem 0x5fe5a240-0x5fe5a27f]
[ 0.004000] ACPI: Reserving FACS table memory at [mem 0x5fe5a240-0x5fe5a27f]
[ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe5bce0-0x5fe5bd69]
[ 0.004000] ACPI: Reserving MCFG table memory at [mem 0x5fe5bd70-0x5fe5bdab]
[ 0.004000] ACPI: Reserving APIC table memory at [mem 0x5fe5bdb0-0x5fe5be11]
[ 0.004000] ACPI: Reserving HPET table memory at [mem 0x5fe5be20-0x5fe5be57]
[ 0.004000] ACPI: Reserving HEST table memory at [mem 0x5fe5be60-0x5fe5c02f]
[ 0.004000] ACPI: Reserving IVRS table memory at [mem 0x5fe5c030-0x5fe5c09f]
[ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe5c0a0-0x5fe5c5be]
[ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe5c5c0-0x5fe5cc71]
[ 0.004000] ACPI: Reserving VFCT table memory at [mem 0x5fe5cc80-0x5fe6bee8]
[ 0.004000] No NUMA configuration found
[ 0.004000] Faking a node at [mem 0x0000000000000000-0x000000017effffff]
[ 0.004000] NODE_DATA(0) allocated [mem 0x17efe7000-0x17effdfff]
[ 0.004000] Zone ranges:
[ 0.004000] DMA [mem 0x0000000000001000-0x0000000000ffffff]
[ 0.004000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]
[ 0.004000] Normal [mem 0x0000000100000000-0x000000017effffff]
[ 0.004000] Device empty
[ 0.004000] Movable zone start for each node
[ 0.004000] Early memory node ranges
[ 0.004000] node 0: [mem 0x0000000000001000-0x000000000009efff]
[ 0.004000] node 0: [mem 0x0000000000100000-0x000000005fe4cfff]
[ 0.004000] node 0: [mem 0x0000000100000000-0x000000017effffff]
[ 0.004000] Initmem setup node 0 [mem 0x0000000000001000-0x000000017effffff]
[ 0.004000] On node 0, zone DMA: 1 pages in unavailable ranges
[ 0.004000] On node 0, zone DMA: 97 pages in unavailable ranges
[ 0.004000] On node 0, zone Normal: 435 pages in unavailable ranges
[ 0.004000] On node 0, zone Normal: 4096 pages in unavailable ranges
[ 0.004000] ACPI: PM-Timer IO Port: 0x818
[ 0.004000] ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1])
[ 0.004000] IOAPIC[0]: apic_id 4, version 33, address 0xfec00000, GSI 0-23
[ 0.004000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[ 0.004000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 low level)
[ 0.004000] ACPI: Using ACPI (MADT) for SMP configuration information
[ 0.004000] ACPI: HPET id: 0x10228210 base: 0xfed00000
[ 0.004000] smpboot: Allowing 2 CPUs, 0 hotplug CPUs
[ 0.004000] smpboot: smpboot: XXX end of prefill_possible_map
[ 0.004000] After prefill_possible_map
[ 0.004000] After init_cpu_to_node
[ 0.004000] After init_gi_nodes
[ 0.004000] After io_apic_init_mappings
[ 0.004000] After x86_init.hyper.guest_late_init
[ 0.004000] [mem 0x80000000-0xf7ffffff] available for PCI devices
[ 0.004000] After e820
[ 0.004000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns
[ 0.004000] After unwind_init
[ 0.004000] After setup_arch
[ 0.004000] After setup_command_line
[ 0.004000] After setup_nr_cpu_ids
[ 0.004000] setup_percpu: NR_CPUS:64 nr_cpumask_bits:2 nr_cpu_ids:2 nr_node_ids:1
[ 0.004000] percpu: Embedded 55 pages/cpu s188328 r8192 d28760 u1048576
[ 0.004000] pcpu-alloc: s188328 r8192 d28760 u1048576 alloc=1*2097152
[ 0.004000] pcpu-alloc: [0] 0 1
[ 0.004000] After setup_per_cpu_areas
[ 0.004000] After smp_perpare_boot_cpu
[ 0.004000] After boot_cpu_hotplug_init
[ 0.004000] Fallback order for Node 0: 0
[ 0.004000] Built 1 zonelists, mobility grouping on. Total pages: 898451
[ 0.004000] Policy zone: Normal
[ 0.004000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc6-00311-gde8224969f66 root=/dev/sda3 rw quiet noisapnp cryptomgr.notests ipv6.disable_ipv6=1 selinux=0
[ 0.004000] Unknown kernel command line parameters "noisapnp BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc6-00311-gde8224969f66", will be passed to user space.
[ 0.004000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[ 0.004000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[ 0.004000] mem auto-init: stack:off, heap alloc:off, heap free:off
[ 0.004000] stackdepot: allocating hash table via alloc_large_system_hash
[ 0.004000] stackdepot hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[ 0.004000] software IO TLB: area num 2.
[ 0.004000] Memory: 3477160K/3651500K available (14336K kernel code, 2340K rwdata, 5308K rodata, 2908K init, 11060K bss, 174080K reserved, 0K cma-reserved)
[ 0.004000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
[ 0.004000] After mm_init
[ 0.004000] After poking_init
[ 0.004000] ftrace: allocating 38664 entries in 152 pages
[ 0.004000] ftrace: allocated 152 pages with 3 groups
[ 0.004000] Dynamic Preempt: full
[ 0.004000] After sched_init
[ 0.004000] rcu: Preemptible hierarchical RCU implementation.
[ 0.004000] rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=2.
[ 0.004000] Trampoline variant of Tasks RCU enabled.
[ 0.004000] Rude variant of Tasks RCU enabled.
[ 0.004000] Tracing variant of Tasks RCU enabled.
[ 0.004000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[ 0.004000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2
[ 0.004000] After rcu_init
[ 0.004000] NR_IRQS: 4352, nr_irqs: 440, preallocated irqs: 16
[ 0.004000] rcu: srcu_init: Setting srcu_struct sizes based on contention.
[ 0.004000] After random_init()
[ 0.004000] After boot_init_stack_canary
[ 0.004000] spurious 8259A interrupt: IRQ7.
[ 0.004000] Console: colour VGA+ 80x25
[ 0.004000] printk: console [tty0] enabled
[ 0.004000] ACPI: Core revision 20221020
[ 0.004000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 133484873504 ns
[ 0.004000] APIC: Switch to symmetric I/O mode setup
[ 0.004000] AMD-Vi: Using global IVHD EFR:0x0, EFR2:0x0
[ 0.004000] APIC: Done
[ 0.004000] Before apic_bsb_setup
[ 0.004000] check_timer begin
[ 0.004000] check_timer after local_irq_disable
[ 0.004000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[ 0.004000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x7070b6fd6fc, max_idle_ns: 881590611710 ns
[ 0.144982] Calibrating delay loop (skipped), value calculated using timer frequency.. 7800.54 BogoMIPS (lpj=15601092)
[ 0.144986] pid_max: default: 32768 minimum: 301
[ 0.145079] LSM: initializing lsm=capability
[ 0.145173] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[ 0.145189] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[ 0.145564] Bit 30 in CPUID ECX not set.
[ 0.145587] Last level iTLB entries: 4KB 512, 2MB 1024, 4MB 512
[ 0.145589] Last level dTLB entries: 4KB 1024, 2MB 1024, 4MB 512, 1GB 0
[ 0.145594] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization
[ 0.145597] Spectre V2 : Mitigation: Retpolines
[ 0.145598] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch
[ 0.145599] Spectre V2 : Spectre v2 / SpectreRSB : Filling RSB on VMEXIT
[ 0.145600] Spectre V2 : Enabling Speculation Barrier for firmware calls
[ 0.145600] RETBleed: Mitigation: untrained return thunk
[ 0.145602] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier
[ 0.145604] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl
[ 0.150046] Freeing SMP alternatives memory: 32K
[ 0.150051] After check_bugs
[ 0.150052] After acpi_subsystem_init
[ 0.150053] After arch_post_acpi_subsys_init
[ 0.150054] After rcu_scheduler_starting
[ 0.150126] After find_task_by_pid_ns and PF_NO_SETAFFINITY
[ 0.150131] After numa_default_policy
[ 0.150151] After rcu_read_lock
[ 0.150152] After rcu_read_unlock
[ 0.150153] After kthreadd_done
[ 0.150165] smpboot: Start of smp_prepare_cpus_common
[ 0.150167] smpboot: smpboot: zalloc 0
[ 0.150168] smpboot: smpboot: zalloc 1
[ 0.150169] smpboot: smpboot: After set_sched_topology()
[ 0.150171] smpboot: smpboot: After smp_sanity_check()
[ 0.150172] smpboot: smpboot: Before x86_init.timers.setup_percpu_clockev()
[ 0.258192] smpboot: smpboot: After x86_init.timers.setup_percpu_clockev()
[ 0.258193] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD Graphics (family: 0x15, model: 0x13, stepping: 0x1)
[ 0.258425] cblist_init_generic: Setting adjustable number of callback queues.
[ 0.258427] cblist_init_generic: Setting shift to 1 and lim to 1.
[ 0.258458] cblist_init_generic: Setting shift to 1 and lim to 1.
[ 0.258484] cblist_init_generic: Setting shift to 1 and lim to 1.
[ 0.258513] Performance Events: Fam15h core perfctr, AMD PMU driver.
[ 0.258536] ... version: 0
[ 0.258537] ... bit width: 48
[ 0.258538] ... generic registers: 6
[ 0.258539] ... value mask: 0000ffffffffffff
[ 0.258540] ... max period: 00007fffffffffff
[ 0.258541] ... fixed-purpose events: 0
[ 0.258542] ... event mask: 000000000000003f
[ 0.258662] rcu: Hierarchical SRCU implementation.
[ 0.258663] rcu: Max phase no-delay instances is 1000.
[ 0.259257] NMI watchdog: Enabled. Permanently consumes one hw-PMU counter.
[ 0.259329] smp: Bringing up secondary CPUs ...
[ 0.259527] x86: Booting SMP configuration:
[ 0.259528] .... node #0, CPUs: #1
[ 0.261007] After schedule_preempt_disabled
[ 10.260990] CPU1 failed to report alive state
[ 10.261070] smp: Brought up 1 node, 1 CPU
[ 10.261073] smpboot: Max logical packages: 2
[ 10.261074] smpboot: Total of 1 processors activated (7800.54 BogoMIPS)
[ 10.261601] devtmpfs: initialized
[ 10.261697] x86/mm: Memory block size: 128MB
[ 10.262788] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[ 10.262795] futex hash table entries: 512 (order: 3, 32768 bytes, linear)
[ 10.262883] pinctrl core: initialized pinctrl subsystem
[ 10.262953] PM: RTC time: 07:39:50, date: 2023-04-17
[ 10.263709] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[ 10.263937] audit: initializing netlink subsys (disabled)
[ 10.264192] thermal_sys: Registered thermal governor 'fair_share'
[ 10.264193] thermal_sys: Registered thermal governor 'bang_bang'
[ 10.264194] thermal_sys: Registered thermal governor 'step_wise'
[ 10.264195] thermal_sys: Registered thermal governor 'user_space'
[ 10.264215] cpuidle: using governor ladder
[ 10.264220] cpuidle: using governor menu
[ 10.264427] PCI: MMCONFIG for domain 0000 [bus 00-3f] at [mem 0xf8000000-0xfbffffff] (base 0xf8000000)
[ 10.264432] PCI: MMCONFIG at [mem 0xf8000000-0xfbffffff] reserved as E820 entry
[ 10.264444] PCI: Using configuration type 1 for base access
[ 10.264646] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.
[ 10.269064] audit: type=2000 audit(1681717190.140:1): state=initialized audit_enabled=0 res=1
[ 10.281080] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages
[ 10.281084] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page
[ 10.281085] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages
[ 10.281086] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page
[ 10.286166] cryptd: max_cpu_qlen set to 1000
[ 10.289615] ACPI: Added _OSI(Module Device)
[ 10.289617] ACPI: Added _OSI(Processor Device)
[ 10.289618] ACPI: Added _OSI(3.0 _SCP Extensions)
[ 10.289620] ACPI: Added _OSI(Processor Aggregator Device)
[ 10.293942] ACPI: DSDT successfully acquired and loaded
[ 10.295626] ACPI: 4 ACPI AML tables successfully acquired and loaded
[ 10.297161] ACPI: Interpreter enabled
[ 10.297184] ACPI: PM: (supports S0 S1 S3 S5)
[ 10.297186] ACPI: Using IOAPIC for interrupt routing
[ 10.297236] HEST: Table parsing has been initialized.
[ 10.297258] GHES: Failed to enable APEI firmware first mode.
[ 10.297261] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[ 10.297262] PCI: Ignoring E820 reservations for host bridge windows
[ 10.297510] ACPI: Enabled 8 GPEs in block 00 to 1F
[ 10.303230] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[ 10.303241] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]
[ 10.303320] acpi PNP0A03:00: _OSC: OS now controls [PME AER PCIeCapability LTR]
[ 10.303335] acpi PNP0A03:00: [Firmware Info]: MMCONFIG for domain 0000 [bus 00-3f] only partially covers this bridge
[ 10.303414] acpi PNP0A03:00: host bridge window expanded to [io 0x0000-0x0cf7 window]; [io 0x03b0-0x03df window] ignored
[ 10.303651] PCI host bridge to bus 0000:00
[ 10.303653] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]
[ 10.303655] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]
[ 10.303658] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000dffff]
[ 10.303660] pci_bus 0000:00: root bus resource [mem 0x80000000-0xffffffff]
[ 10.303662] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 10.303686] pci 0000:00:00.0: [1022:1410] type 00 class 0x060000
[ 10.303830] pci 0000:00:00.2: [1022:1419] type 00 class 0x080600
[ 10.303921] pci 0000:00:01.0: [1002:9996] type 00 class 0x030000
[ 10.303929] pci 0000:00:01.0: reg 0x10: [mem 0xe0000000-0xefffffff pref]
[ 10.303934] pci 0000:00:01.0: reg 0x14: [io 0x1000-0x10ff]
[ 10.303939] pci 0000:00:01.0: reg 0x18: [mem 0xf0180000-0xf01bffff]
[ 10.303955] pci 0000:00:01.0: enabling Extended Tags
[ 10.303965] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]
[ 10.303983] pci 0000:00:01.0: supports D1 D2
[ 10.304048] pci 0000:00:01.1: [1002:9902] type 00 class 0x040300
[ 10.304056] pci 0000:00:01.1: reg 0x10: [mem 0xf01c0000-0xf01c3fff]
[ 10.304077] pci 0000:00:01.1: enabling Extended Tags
[ 10.304101] pci 0000:00:01.1: supports D1 D2
[ 10.304185] pci 0000:00:11.0: [1022:7801] type 00 class 0x010601
[ 10.304198] pci 0000:00:11.0: reg 0x10: [io 0x1410-0x1417]
[ 10.304206] pci 0000:00:11.0: reg 0x14: [io 0x1420-0x1423]
[ 10.304213] pci 0000:00:11.0: reg 0x18: [io 0x1418-0x141f]
[ 10.304221] pci 0000:00:11.0: reg 0x1c: [io 0x1424-0x1427]
[ 10.304228] pci 0000:00:11.0: reg 0x20: [io 0x1400-0x140f]
[ 10.304235] pci 0000:00:11.0: reg 0x24: [mem 0xf01cc000-0xf01cc7ff]
[ 10.304392] pci 0000:00:12.0: [1022:7807] type 00 class 0x0c0310
[ 10.304406] pci 0000:00:12.0: reg 0x10: [mem 0xf01c8000-0xf01c8fff]
[ 10.304587] pci 0000:00:12.2: [1022:7808] type 00 class 0x0c0320
[ 10.304601] pci 0000:00:12.2: reg 0x10: [mem 0xf01cd000-0xf01cd0ff]
[ 10.304666] pci 0000:00:12.2: supports D1 D2
[ 10.304667] pci 0000:00:12.2: PME# supported from D0 D1 D2 D3hot
[ 10.304668] pci 0000:00:12.2: pme_poll = true
[ 10.304669] pci 0000:00:12.2: after device_set_wakeup_capable()
[ 10.304673] pci 0000:00:12.2: after pci_pme_active()
[ 10.304814] pci 0000:00:13.0: [1022:7807] type 00 class 0x0c0310
[ 10.304827] pci 0000:00:13.0: reg 0x10: [mem 0xf01c9000-0xf01c9fff]
[ 10.305566] pci 0000:00:13.2: [1022:7808] type 00 class 0x0c0320
[ 10.305583] pci 0000:00:13.2: reg 0x10: [mem 0xf01ce000-0xf01ce0ff]
[ 10.305654] pci 0000:00:13.2: supports D1 D2
[ 10.305656] pci 0000:00:13.2: PME# supported from D0 D1 D2 D3hot
[ 10.305657] pci 0000:00:13.2: pme_poll = true
[ 10.305659] pci 0000:00:13.2: after device_set_wakeup_capable()
[ 10.305662] pci 0000:00:13.2: after pci_pme_active()
[ 10.305803] pci 0000:00:14.0: [1022:780b] type 00 class 0x0c0500
[ 10.305975] pci 0000:00:14.2: [1022:780d] type 00 class 0x040300
[ 10.305992] pci 0000:00:14.2: reg 0x10: [mem 0xf01c4000-0xf01c7fff 64bit]
[ 10.306047] pci 0000:00:14.2: PME# supported from D0 D3hot D3cold
[ 10.306049] pci 0000:00:14.2: pme_poll = true
[ 10.306050] pci 0000:00:14.2: after device_set_wakeup_capable()
[ 10.306052] pci 0000:00:14.2: after pci_pme_active()
[ 10.306185] pci 0000:00:14.3: [1022:780e] type 00 class 0x060100
[ 10.306360] pci 0000:00:14.4: [1022:780f] type 01 class 0x060401
[ 10.306502] pci 0000:00:14.5: [1022:7809] type 00 class 0x0c0310
[ 10.306515] pci 0000:00:14.5: reg 0x10: [mem 0xf01ca000-0xf01cafff]
[ 10.306682] pci 0000:00:15.0: [1022:43a0] type 01 class 0x060400
[ 10.306711] pci 0000:00:15.0: enabling Extended Tags
[ 10.306751] pci 0000:00:15.0: supports D1 D2
[ 10.306911] pci 0000:00:15.1: [1022:43a1] type 01 class 0x060400
[ 10.306942] pci 0000:00:15.1: enabling Extended Tags
[ 10.306981] pci 0000:00:15.1: supports D1 D2
[ 10.307137] pci 0000:00:15.2: [1022:43a2] type 01 class 0x060400
[ 10.307166] pci 0000:00:15.2: enabling Extended Tags
[ 10.307205] pci 0000:00:15.2: supports D1 D2
[ 10.307280] pci 0000:00:16.0: [1022:7807] type 00 class 0x0c0310
[ 10.307293] pci 0000:00:16.0: reg 0x10: [mem 0xf01cb000-0xf01cbfff]
[ 10.307467] pci 0000:00:16.2: [1022:7808] type 00 class 0x0c0320
[ 10.307481] pci 0000:00:16.2: reg 0x10: [mem 0xf01cf000-0xf01cf0ff]
[ 10.307546] pci 0000:00:16.2: supports D1 D2
[ 10.307547] pci 0000:00:16.2: PME# supported from D0 D1 D2 D3hot
[ 10.307548] pci 0000:00:16.2: pme_poll = true
[ 10.307549] pci 0000:00:16.2: after device_set_wakeup_capable()
[ 10.307552] pci 0000:00:16.2: after pci_pme_active()
[ 10.307691] pci 0000:00:18.0: [1022:1400] type 00 class 0x060000
[ 10.307758] pci 0000:00:18.1: [1022:1401] type 00 class 0x060000
[ 10.307819] pci 0000:00:18.2: [1022:1402] type 00 class 0x060000
[ 10.307880] pci 0000:00:18.3: [1022:1403] type 00 class 0x060000
[ 10.308016] pci 0000:00:18.4: [1022:1404] type 00 class 0x060000
[ 10.308079] pci 0000:00:18.5: [1022:1405] type 00 class 0x060000
[ 10.308153] pci_bus 0000:01: extended config space not accessible
[ 10.308218] pci 0000:00:14.4: PCI bridge to [bus 01] (subtractive decode)
[ 10.308227] pci 0000:00:14.4: bridge window [io 0x0000-0x0cf7 window] (subtractive decode)
[ 10.308230] pci 0000:00:14.4: bridge window [io 0x0d00-0xffff window] (subtractive decode)
[ 10.308232] pci 0000:00:14.4: bridge window [mem 0x000a0000-0x000dffff] (subtractive decode)
[ 10.308235] pci 0000:00:14.4: bridge window [mem 0x80000000-0xffffffff] (subtractive decode)
[ 10.308283] pci 0000:00:15.0: PCI bridge to [bus 02]
[ 10.308368] pci 0000:03:00.0: [1b21:1042] type 00 class 0x0c0330
[ 10.308404] pci 0000:03:00.0: reg 0x10: [mem 0xf0000000-0xf0007fff 64bit]
[ 10.308579] pci 0000:03:00.0: PME# supported from D3hot D3cold
[ 10.308581] pci 0000:03:00.0: pme_poll = true
[ 10.308582] pci 0000:03:00.0: after device_set_wakeup_capable()
[ 10.308587] pci 0000:03:00.0: after pci_pme_active()
[ 10.308625] pci 0000:03:00.0: 2.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s PCIe x1 link at 0000:00:15.1 (capable of 4.000 Gb/s with 5.0 GT/s PCIe x1 link)
[ 10.317133] pci 0000:00:15.1: PCI bridge to [bus 03]
[ 10.317145] pci 0000:00:15.1: bridge window [mem 0xf0000000-0xf00fffff]
[ 10.317154] pci 0000:00:15.2: bridge configuration invalid ([bus 00-00]), reconfiguring
[ 10.317278] pci 0000:04:00.0: [10ec:8168] type 00 class 0x020000
[ 10.317296] pci 0000:04:00.0: reg 0x10: [io 0x0000-0x00ff]
[ 10.317318] pci 0000:04:00.0: reg 0x18: [mem 0x00000000-0x00000fff 64bit pref]
[ 10.317332] pci 0000:04:00.0: reg 0x20: [mem 0x00000000-0x00003fff 64bit pref]
[ 10.317440] pci 0000:04:00.0: supports D1 D2
[ 10.317442] pci 0000:04:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[ 10.317444] pci 0000:04:00.0: pme_poll = true
[ 10.317445] pci 0000:04:00.0: after device_set_wakeup_capable()
[ 10.317449] pci 0000:04:00.0: after pci_pme_active()
[ 10.329042] pci 0000:00:15.2: PCI bridge to [bus 04-ff]
[ 10.329053] pci 0000:00:15.2: bridge window [io 0x0000-0x0fff]
[ 10.329057] pci 0000:00:15.2: bridge window [mem 0x00000000-0x000fffff]
[ 10.329062] pci 0000:00:15.2: bridge window [mem 0x00000000-0x000fffff 64bit pref]
[ 10.329065] pci_bus 0000:04: busn_res: [bus 04-ff] end is updated to 04
[ 10.329575] ACPI: PCI: Interrupt link INTA configured for IRQ 0
[ 10.329669] ACPI: PCI: Interrupt link INTB configured for IRQ 0
[ 10.329759] ACPI: PCI: Interrupt link INTC configured for IRQ 0
[ 10.329849] ACPI: PCI: Interrupt link INTD configured for IRQ 0
[ 10.329940] ACPI: PCI: Interrupt link INTE configured for IRQ 0
[ 10.330029] ACPI: PCI: Interrupt link INTF configured for IRQ 0
[ 10.330119] ACPI: PCI: Interrupt link INTG configured for IRQ 0
[ 10.330210] ACPI: PCI: Interrupt link INTH configured for IRQ 0
[ 10.330444] iommu: Default domain type: Translated
[ 10.330446] iommu: DMA domain TLB invalidation policy: lazy mode
[ 10.330624] SCSI subsystem initialized
[ 10.330718] libata version 3.00 loaded.
[ 10.330752] ACPI: bus type USB registered
[ 10.330775] usbcore: registered new interface driver usbfs
[ 10.330786] usbcore: registered new interface driver hub
[ 10.330798] usbcore: registered new device driver usb
[ 10.331125] PCI: Using ACPI for IRQ routing
[ 10.332694] PCI: pci_cache_line_size set to 64 bytes
[ 10.332745] e820: reserve RAM buffer [mem 0x0009fc00-0x0009ffff]
[ 10.332748] e820: reserve RAM buffer [mem 0x5fe4d000-0x5fffffff]
[ 10.332750] e820: reserve RAM buffer [mem 0x17f000000-0x17fffffff]
[ 10.332794] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0
[ 10.332799] hpet0: 3 comparators, 32-bit 14.318180 MHz counter
[ 10.334067] clocksource: Switched to clocksource tsc-early
[ 10.350705] VFS: Disk quotas dquot_6.6.0
[ 10.350734] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[ 10.350847] pnp: PnP ACPI init
[ 10.351137] system 00:00: [mem 0xfec10002-0xfec11001] could not be reserved
[ 10.351446] pnp: PnP ACPI: found 2 devices
[ 10.358408] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
[ 10.358544] NET: Registered PF_INET protocol family
[ 10.358686] IP idents hash table entries: 65536 (order: 7, 524288 bytes, linear)
[ 10.360256] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear)
[ 10.360271] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
[ 10.360277] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear)
[ 10.360344] TCP bind hash table entries: 32768 (order: 8, 1048576 bytes, linear)
[ 10.360670] TCP: Hash tables configured (established 32768 bind 32768)
[ 10.360739] UDP hash table entries: 2048 (order: 4, 65536 bytes, linear)
[ 10.360763] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear)
[ 10.360870] NET: Registered PF_UNIX/PF_LOCAL protocol family
[ 10.360906] pci 0000:00:15.2: BAR 15: assigned [mem 0x80000000-0x800fffff 64bit pref]
[ 10.360912] pci 0000:00:15.2: BAR 13: assigned [io 0x2000-0x2fff]
[ 10.360917] pci 0000:00:14.4: PCI bridge to [bus 01]
[ 10.360928] pci 0000:00:15.0: PCI bridge to [bus 02]
[ 10.360936] pci 0000:00:15.1: PCI bridge to [bus 03]
[ 10.360939] pci 0000:00:15.1: bridge window [mem 0xf0000000-0xf00fffff]
[ 10.360947] pci 0000:04:00.0: BAR 4: assigned [mem 0x80000000-0x80003fff 64bit pref]
[ 10.360959] pci 0000:04:00.0: BAR 2: assigned [mem 0x80004000-0x80004fff 64bit pref]
[ 10.360971] pci 0000:04:00.0: BAR 0: assigned [io 0x2000-0x20ff]
[ 10.360976] pci 0000:00:15.2: PCI bridge to [bus 04]
[ 10.361116] pci 0000:00:15.2: bridge window [io 0x2000-0x2fff]
[ 10.361122] pci 0000:00:15.2: bridge window [mem 0x80000000-0x800fffff 64bit pref]
[ 10.361128] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]
[ 10.361130] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]
[ 10.361132] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000dffff]
[ 10.361133] pci_bus 0000:00: resource 7 [mem 0x80000000-0xffffffff]
[ 10.361135] pci_bus 0000:01: resource 4 [io 0x0000-0x0cf7 window]
[ 10.361137] pci_bus 0000:01: resource 5 [io 0x0d00-0xffff window]
[ 10.361138] pci_bus 0000:01: resource 6 [mem 0x000a0000-0x000dffff]
[ 10.361140] pci_bus 0000:01: resource 7 [mem 0x80000000-0xffffffff]
[ 10.361141] pci_bus 0000:03: resource 1 [mem 0xf0000000-0xf00fffff]
[ 10.361143] pci_bus 0000:04: resource 0 [io 0x2000-0x2fff]
[ 10.361144] pci_bus 0000:04: resource 2 [mem 0x80000000-0x800fffff 64bit pref]
[ 10.361239] pci 0000:00:01.1: D0 power state depends on 0000:00:01.0
[ 10.361456] pci 0000:00:12.0: AMD USB device
[ 10.361478] pci 0000:00:12.0: AMD USB ohci handoff
[ 10.361761] pci 0000:00:12.2: AMD USB device
[ 10.361775] pci 0000:00:12.2: AMD USB ehci handoff
[ 10.361903] pci 0000:00:12.2: PME# does not work under D3, disabling it
[ 10.362049] pci 0000:00:13.0: AMD USB device
[ 10.362062] pci 0000:00:13.0: AMD USB ohci handoff
[ 10.362328] pci 0000:00:13.2: AMD USB device
[ 10.362339] pci 0000:00:13.2: AMD USB ehci handoff
[ 10.362469] pci 0000:00:13.2: PME# does not work under D3, disabling it
[ 10.362622] pci 0000:00:14.5: AMD USB device
[ 10.362634] pci 0000:00:14.5: AMD USB ohci handoff
[ 10.362915] pci 0000:00:16.0: AMD USB device
[ 10.362928] pci 0000:00:16.0: AMD USB ohci handoff
[ 10.363195] pci 0000:00:16.2: AMD USB device
[ 10.363206] pci 0000:00:16.2: AMD USB ehci handoff
[ 10.363334] pci 0000:00:16.2: PME# does not work under D3, disabling it
[ 10.363561] pci 0000:03:00.0: AMD USB xhci handoff
[ 10.363610] PCI: CLS 64 bytes, default 64
[ 10.363723] pci 0000:00:00.2: AMD-Vi: Applying erratum 746 workaround
[ 10.363810] pci 0000:00:01.0: Adding to iommu group 0
[ 10.363826] pci 0000:00:01.1: Adding to iommu group 0
[ 10.363851] pci 0000:00:11.0: Adding to iommu group 1
[ 10.363886] pci 0000:00:12.0: Adding to iommu group 2
[ 10.363903] pci 0000:00:12.2: Adding to iommu group 2
[ 10.363937] pci 0000:00:13.0: Adding to iommu group 3
[ 10.363953] pci 0000:00:13.2: Adding to iommu group 3
[ 10.363991] pci 0000:00:14.0: Adding to iommu group 4
[ 10.364009] pci 0000:00:14.2: Adding to iommu group 4
[ 10.364025] pci 0000:00:14.3: Adding to iommu group 4
[ 10.364048] pci 0000:00:14.4: Adding to iommu group 5
[ 10.364070] pci 0000:00:14.5: Adding to iommu group 6
[ 10.364104] pci 0000:00:15.0: Adding to iommu group 7
[ 10.364123] pci 0000:00:15.1: Adding to iommu group 7
[ 10.364139] pci 0000:00:15.2: Adding to iommu group 7
[ 10.364178] pci 0000:00:16.0: Adding to iommu group 8
[ 10.364194] pci 0000:00:16.2: Adding to iommu group 8
[ 10.364246] pci 0000:00:18.0: Adding to iommu group 9
[ 10.364266] pci 0000:00:18.1: Adding to iommu group 9
[ 10.364283] pci 0000:00:18.2: Adding to iommu group 9
[ 10.364303] pci 0000:00:18.3: Adding to iommu group 9
[ 10.364321] pci 0000:00:18.4: Adding to iommu group 9
[ 10.364340] pci 0000:00:18.5: Adding to iommu group 9
[ 10.364352] pci 0000:03:00.0: Adding to iommu group 7
[ 10.364360] pci 0000:04:00.0: Adding to iommu group 7
[ 10.366500] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[ 10.366505] AMD-Vi: Extended features (0x800000853, 0x0): PreF PPR GT IA
[ 10.366510] AMD-Vi: Interrupt remapping enabled
[ 10.366691] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[ 10.366693] software IO TLB: mapped [mem 0x000000005be4d000-0x000000005fe4d000] (64MB)
[ 10.366742] LVT offset 0 assigned for vector 0x400
[ 10.366763] perf: AMD IBS detected (0x000000ff)
[ 10.366771] amd_uncore: 4 amd_nb counters detected
[ 10.367574] workingset: timestamp_bits=37 max_order=20 bucket_order=0
[ 10.367603] zbud: loaded
[ 10.368066] NET: Registered PF_ALG protocol family
[ 10.368071] Key type asymmetric registered
[ 10.368073] Asymmetric key parser 'x509' registered
[ 10.368346] alg: self-tests disabled
[ 10.368439] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 251)
[ 10.368477] io scheduler mq-deadline registered
[ 10.368479] io scheduler kyber registered
[ 10.370011] pcieport 0000:00:15.0: PME: Signaling with IRQ 25
[ 10.370174] pcieport 0000:00:15.1: PME: Signaling with IRQ 26
[ 10.370247] pcieport 0000:00:15.2: enabling device (0000 -> 0003)
[ 10.370449] pcieport 0000:00:15.2: PME: Signaling with IRQ 27
[ 10.370706] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
[ 10.370767] ACPI: button: Power Button [PWRF]
[ 10.370823] ACPI: \_SB_.P000: Found 2 idle states
[ 10.370937] ACPI: \_SB_.P001: Found 2 idle states
[ 10.371827] thermal LNXTHERM:00: registered as thermal_zone0
[ 10.371830] ACPI: thermal: Thermal Zone [TZ00] (0 C)
[ 10.372146] Non-volatile memory driver v1.3
[ 10.372218] AMD-Vi: AMD IOMMUv2 loaded and initialized
[ 10.372338] ahci 0000:00:11.0: version 3.0
[ 10.372612] ahci 0000:00:11.0: AHCI 0001.0300 32 slots 8 ports 6 Gbps 0x40 impl SATA mode
[ 10.372616] ahci 0000:00:11.0: flags: 64bit ncq sntf ilck led clo pio
[ 10.374133] scsi host0: ahci
[ 10.374333] scsi host1: ahci
[ 10.374510] scsi host2: ahci
[ 10.374710] scsi host3: ahci
[ 10.374879] scsi host4: ahci
[ 10.375057] scsi host5: ahci
[ 10.375241] scsi host6: ahci
[ 10.375421] scsi host7: ahci
[ 10.375508] ata port1: DUMMY
[ 10.375510] ata port2: DUMMY
[ 10.375511] ata port3: DUMMY
[ 10.375512] ata port4: DUMMY
[ 10.375514] ata port5: DUMMY
[ 10.375515] ata port6: DUMMY
[ 10.375517] ata port7: SATA max UDMA/133 abar m2048@0xf01cc000 port 0xf01cc400 irq 19
[ 10.375519] ata port8: DUMMY
[ 10.375597] ACPI: bus type drm_connector registered
[ 10.375823] i8042: PNP: No PS/2 controller found.
[ 10.375824] i8042: Probing ports directly.
[ 10.378675] serio: i8042 KBD port at 0x60,0x64 irq 1
[ 10.378751] serio: i8042 AUX port at 0x60,0x64 irq 12
[ 10.378874] mousedev: PS/2 mouse device common for all mice
[ 10.378927] rtc_cmos 00:01: RTC can wake from S4
[ 10.379173] rtc_cmos 00:01: registered as rtc0
[ 10.379197] rtc_cmos 00:01: setting system clock to 2023-04-17T07:39:50 UTC (1681717190)
[ 10.379234] rtc_cmos 00:01: alarms up to one day, y3k, 114 bytes nvram, hpet irqs
[ 10.379268] device-mapper: uevent: version 1.0.3
[ 10.379337] device-mapper: ioctl: 4.47.0-ioctl (2022-07-28) initialised: dm-devel@redhat.com
[ 10.379496] hid: raw HID events driver (C) Jiri Kosina
[ 10.379531] usbcore: registered new interface driver usbhid
[ 10.379532] usbhid: USB HID core driver
[ 10.379638] Initializing XFRM netlink socket
[ 10.379648] NET: Registered PF_PACKET protocol family
[ 10.379650] x86/pm: family 0x15 cpu detected, MSR saving is needed during suspending.
[ 10.379817] microcode: CPU0: patch_level=0x0600111f
[ 10.379827] microcode: Microcode Update Driver: v2.2.
[ 10.379831] IPI shorthand broadcast: enabled
[ 10.379838] AVX version of gcm_enc/dec engaged.
[ 10.379854] AES CTR mode by8 optimization enabled
[ 10.380695] ata link7: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[ 10.380969] ata dev7.0: ATA-9: SanDisk SDSSDP064G, 2.0.0, max UDMA/133
[ 10.380972] ata dev7.0: 125045424 sectors, multi 1: LBA48 NCQ (depth 32)
[ 10.381178] ata dev7.0: configured for UDMA/133
[ 10.381289] scsi 6:0:0:0: Direct-Access ATA SanDisk SDSSDP06 0 PQ: 0 ANSI: 5
[ 10.381718] sd 6:0:0:0: [sda] 125045424 512-byte logical blocks: (64.0 GB/59.6 GiB)
[ 10.381730] sd 6:0:0:0: [sda] Write Protect is off
[ 10.381733] sd 6:0:0:0: [sda] Mode Sense: 00 3a 00 00
[ 10.381749] sd 6:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 10.381775] sd 6:0:0:0: [sda] Preferred minimum I/O size 512 bytes
[ 10.382547] sda: sda1 sda2 sda3
[ 10.382736] sd 6:0:0:0: [sda] Attached SCSI disk
[ 10.385344] sched_clock: Marking stable (10268007974, 116977866)->(10387423156, -2437316)
[ 10.385519] registered taskstats version 1
[ 10.385734] zswap: loaded using pool lzo/zbud
[ 10.390028] kmemleak: Kernel memory leak detector initialized (mem pool available: 15677)
[ 10.390033] debug_vm_pgtable: [debug_vm_pgtable ]: Validating architecture page table helpers
[ 10.393894] kmemleak: Automatic memory scanning thread started
[ 10.394488] Key type encrypted registered
[ 10.397453] PM: Magic number: 3:139:673
[ 10.397466] workqueue scsi_tmf_3: hash matches
[ 10.410334] EXT4-fs (sda3): mounted filesystem fe29e0dc-6303-4401-987c-8472bc1b9516 with ordered data mode. Quota mode: none.
[ 10.410378] VFS: Mounted root (ext4 filesystem) on device 8:3.
[ 10.412268] devtmpfs: mounted
[ 10.412286] After kernel_init_freeable
[ 10.416793] Freeing unused kernel image (initmem) memory: 2908K
[ 10.421291] Write protecting the kernel read-only data: 20480k
[ 10.421559] Freeing unused kernel image (rodata/data gap) memory: 836K
[ 10.458614] x86/mm: Checked W+X mappings: passed, no W+X pages found.
[ 10.458620] rodata_test: all tests were successful
[ 10.458621] After mark_readonly
[ 10.458621] After pti_finalize
[ 10.458637] rcu_end_inkernel_boot
[ 10.458644] Run /sbin/init as init process
[ 10.458646] with arguments:
[ 10.458648] /sbin/init
[ 10.458649] noisapnp
[ 10.458650] with environment:
[ 10.458650] HOME=/
[ 10.458651] TERM=linux
[ 10.458652] BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc6-00311-gde8224969f66
[ 10.637190] systemd[1]: Inserted module 'autofs4'
[ 10.663625] NET: Registered PF_INET6 protocol family
[ 10.664473] Segment Routing with IPv6
[ 10.664500] In-situ OAM (IOAM) with IPv6
[ 10.691376] systemd[1]: systemd 252.6-1 running in system mode (+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
[ 10.691387] systemd[1]: Detected architecture x86-64.
[ 10.696468] systemd[1]: Hostname set to <kodi>.
[ 11.000700] systemd[1]: Queued start job for default target graphical.target.
[ 11.011651] systemd[1]: Created slice system-getty.slice - Slice /system/getty.
[ 11.012747] systemd[1]: Created slice system-modprobe.slice - Slice /system/modprobe.
[ 11.013600] systemd[1]: Created slice user.slice - User and Session Slice.
[ 11.013784] systemd[1]: Started systemd-ask-password-console.path - Dispatch Password Requests to Console Directory Watch.
[ 11.013902] systemd[1]: Started systemd-ask-password-wall.path - Forward Password Requests to Wall Directory Watch.
[ 11.014324] systemd[1]: Set up automount proc-sys-fs-binfmt_misc.automount - Arbitrary Executable File Formats File System Automount Point.
[ 11.014366] systemd[1]: Reached target cryptsetup.target - Local Encrypted Volumes.
[ 11.014416] systemd[1]: Reached target integritysetup.target - Local Integrity Protected Volumes.
[ 11.014455] systemd[1]: Reached target paths.target - Path Units.
[ 11.014485] systemd[1]: Reached target remote-fs.target - Remote File Systems.
[ 11.014515] systemd[1]: Reached target slices.target - Slice Units.
[ 11.014554] systemd[1]: Reached target swap.target - Swaps.
[ 11.014592] systemd[1]: Reached target veritysetup.target - Local Verity Protected Volumes.
[ 11.017065] systemd[1]: Listening on systemd-coredump.socket - Process Core Dump Socket.
[ 11.017317] systemd[1]: Listening on systemd-fsckd.socket - fsck to fsckd communication Socket.
[ 11.017479] systemd[1]: Listening on systemd-initctl.socket - initctl Compatibility Named Pipe.
[ 11.017786] systemd[1]: Listening on systemd-journald-audit.socket - Journal Audit Socket.
[ 11.018059] systemd[1]: Listening on systemd-journald-dev-log.socket - Journal Socket (/dev/log).
[ 11.018337] systemd[1]: Listening on systemd-journald.socket - Journal Socket.
[ 11.018594] systemd[1]: Listening on systemd-networkd.socket - Network Service Netlink Socket.
[ 11.019420] systemd[1]: Listening on systemd-udevd-control.socket - udev Control Socket.
[ 11.019693] systemd[1]: Listening on systemd-udevd-kernel.socket - udev Kernel Socket.
[ 11.022504] systemd[1]: Mounting dev-hugepages.mount - Huge Pages File System...
[ 11.025189] systemd[1]: Mounting dev-mqueue.mount - POSIX Message Queue File System...
[ 11.029590] systemd[1]: Mounting sys-kernel-debug.mount - Kernel Debug File System...
[ 11.046849] systemd[1]: Mounting sys-kernel-tracing.mount - Kernel Trace File System...
[ 11.053730] systemd[1]: Starting kmod-static-nodes.service - Create List of Static Device Nodes...
[ 11.066015] systemd[1]: Starting modprobe@configfs.service - Load Kernel Module configfs...
[ 11.069414] systemd[1]: Starting modprobe@dm_mod.service - Load Kernel Module dm_mod...
[ 11.076547] systemd[1]: Starting modprobe@drm.service - Load Kernel Module drm...
[ 11.090852] systemd[1]: Starting modprobe@efi_pstore.service - Load Kernel Module efi_pstore...
[ 11.097815] systemd[1]: Starting modprobe@fuse.service - Load Kernel Module fuse...
[ 11.109388] systemd[1]: Starting modprobe@loop.service - Load Kernel Module loop...
[ 11.109456] systemd[1]: systemd-firstboot.service - First Boot Wizard was skipped because of an unmet condition check (ConditionFirstBoot=yes).
[ 11.109525] systemd[1]: systemd-fsck-root.service - File System Check on Root Device was skipped because of an unmet condition check (ConditionPathIsReadWrite=!/).
[ 11.109562] systemd[1]: Reached target local-fs.target - Local File Systems.
[ 11.109629] systemd[1]: apparmor.service - Load AppArmor profiles was skipped because of an unmet condition check (ConditionSecurity=apparmor).
[ 11.120810] loop: module loaded
[ 11.122003] systemd[1]: Starting systemd-binfmt.service - Set Up Additional Binary Formats...
[ 11.129598] fuse: init (API version 7.38)
[ 11.137426] systemd[1]: Starting systemd-journald.service - Journal Service...
[ 11.145752] systemd[1]: Starting systemd-random-seed.service - Load/Save Random Seed...
[ 11.157494] systemd[1]: Starting systemd-sysctl.service - Apply Kernel Variables...
[ 11.172634] systemd[1]: Starting systemd-sysusers.service - Create System Users...
[ 11.197939] systemd[1]: Starting systemd-udev-trigger.service - Coldplug All udev Devices...
[ 11.216247] systemd[1]: Mounted dev-hugepages.mount - Huge Pages File System.
[ 11.216451] systemd[1]: Mounted dev-mqueue.mount - POSIX Message Queue File System.
[ 11.216611] systemd[1]: Mounted sys-kernel-debug.mount - Kernel Debug File System.
[ 11.216772] systemd[1]: Mounted sys-kernel-tracing.mount - Kernel Trace File System.
[ 11.233481] systemd[1]: Finished kmod-static-nodes.service - Create List of Static Device Nodes.
[ 11.234279] systemd[1]: modprobe@configfs.service: Deactivated successfully.
[ 11.238519] systemd[1]: Finished modprobe@configfs.service - Load Kernel Module configfs.
[ 11.239177] systemd[1]: modprobe@dm_mod.service: Deactivated successfully.
[ 11.247227] systemd[1]: Finished modprobe@dm_mod.service - Load Kernel Module dm_mod.
[ 11.247980] systemd[1]: modprobe@drm.service: Deactivated successfully.
[ 11.253523] systemd[1]: Finished modprobe@drm.service - Load Kernel Module drm.
[ 11.254233] systemd[1]: modprobe@efi_pstore.service: Deactivated successfully.
[ 11.255540] systemd[1]: Finished modprobe@efi_pstore.service - Load Kernel Module efi_pstore.
[ 11.256109] systemd[1]: modprobe@fuse.service: Deactivated successfully.
[ 11.261747] systemd[1]: Finished modprobe@fuse.service - Load Kernel Module fuse.
[ 11.262375] systemd[1]: modprobe@loop.service: Deactivated successfully.
[ 11.265991] systemd[1]: Finished modprobe@loop.service - Load Kernel Module loop.
[ 11.267213] systemd[1]: Finished systemd-sysctl.service - Apply Kernel Variables.
[ 11.268186] systemd[1]: Finished systemd-sysusers.service - Create System Users.
[ 11.268690] systemd[1]: proc-sys-fs-binfmt_misc.automount: Got automount request for /proc/sys/fs/binfmt_misc, triggered by 133 (systemd-binfmt)
[ 11.282799] systemd[1]: Mounting proc-sys-fs-binfmt_misc.mount - Arbitrary Executable File Formats File System...
[ 11.309441] systemd[1]: Mounting sys-fs-fuse-connections.mount - FUSE Control File System...
[ 11.358583] systemd[1]: Mounting sys-kernel-config.mount - Kernel Configuration File System...
[ 11.358687] systemd[1]: systemd-pstore.service - Platform Persistent Storage Archival was skipped because of an unmet condition check (ConditionDirectoryNotEmpty=/sys/fs/pstore).
[ 11.358816] systemd[1]: systemd-repart.service - Repartition Root Disk was skipped because no trigger condition checks were met.
[ 11.377438] systemd[1]: Starting systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev...
[ 11.378820] systemd[1]: Mounted proc-sys-fs-binfmt_misc.mount - Arbitrary Executable File Formats File System.
[ 11.385019] tsc: Refined TSC clocksource calibration: 3900.223 MHz
[ 11.385026] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x70705a6472c, max_idle_ns: 881590586812 ns
[ 11.385038] clocksource: Switched to clocksource tsc
[ 11.397465] systemd[1]: Finished systemd-binfmt.service - Set Up Additional Binary Formats.
[ 11.397822] systemd[1]: Mounted sys-fs-fuse-connections.mount - FUSE Control File System.
[ 11.397985] systemd[1]: Mounted sys-kernel-config.mount - Kernel Configuration File System.
[ 11.448710] systemd[1]: Finished systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev.
[ 11.457254] systemd[1]: Starting systemd-udevd.service - Rule-based Manager for Device Events and Files...
[ 11.509082] systemd[1]: Started systemd-journald.service - Journal Service.
[ 11.569863] systemd-journald[134]: Received client request to flush runtime journal.
[ 12.053221] sd 6:0:0:0: Attached scsi generic sg0 type 0
[ 12.140997] random: crng init done
[ 12.367954] acpi_cpufreq: overriding BIOS provided _PSD data
[ 12.510747] QUIRK: Enable AMD PLL fix
[ 12.510804] ehci-pci 0000:00:12.2: EHCI Host Controller
[ 12.510833] ehci-pci 0000:00:12.2: new USB bus registered, assigned bus number 1
[ 12.510844] ehci-pci 0000:00:12.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
[ 12.510853] ehci-pci 0000:00:12.2: debug port 1
[ 12.511023] ehci-pci 0000:00:12.2: irq 17, io mem 0xf01cd000
[ 12.515767] piix4_smbus 0000:00:14.0: SMBus Host Controller at 0xb00, revision 0
[ 12.515773] piix4_smbus 0000:00:14.0: Using register 0x2e for SMBus port selection
[ 12.516241] piix4_smbus 0000:00:14.0: Auxiliary SMBus Host Controller at 0xb20
[ 12.525014] ehci-pci 0000:00:12.2: USB 2.0 started, EHCI 1.00
[ 12.525397] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 12.525400] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.525402] usb usb1: Product: EHCI Host Controller
[ 12.525404] usb usb1: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ehci_hcd
[ 12.525405] usb usb1: SerialNumber: 0000:00:12.2
[ 12.525862] hub 1-0:1.0: USB hub found
[ 12.525889] hub 1-0:1.0: 5 ports detected
[ 12.526592] ehci-pci 0000:00:13.2: EHCI Host Controller
[ 12.526612] ehci-pci 0000:00:13.2: new USB bus registered, assigned bus number 2
[ 12.526623] ehci-pci 0000:00:13.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
[ 12.526632] ehci-pci 0000:00:13.2: debug port 1
[ 12.526767] ehci-pci 0000:00:13.2: irq 17, io mem 0xf01ce000
[ 12.541014] ehci-pci 0000:00:13.2: USB 2.0 started, EHCI 1.00
[ 12.541263] usb usb2: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 12.541266] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.541268] usb usb2: Product: EHCI Host Controller
[ 12.541270] usb usb2: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ehci_hcd
[ 12.541271] usb usb2: SerialNumber: 0000:00:13.2
[ 12.541718] hub 2-0:1.0: USB hub found
[ 12.541746] hub 2-0:1.0: 5 ports detected
[ 12.542418] ehci-pci 0000:00:16.2: EHCI Host Controller
[ 12.542436] ehci-pci 0000:00:16.2: new USB bus registered, assigned bus number 3
[ 12.542447] ehci-pci 0000:00:16.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
[ 12.542456] ehci-pci 0000:00:16.2: debug port 1
[ 12.542584] ehci-pci 0000:00:16.2: irq 17, io mem 0xf01cf000
[ 12.557023] ehci-pci 0000:00:16.2: USB 2.0 started, EHCI 1.00
[ 12.557405] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 12.557408] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.557410] usb usb3: Product: EHCI Host Controller
[ 12.557411] usb usb3: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ehci_hcd
[ 12.557413] usb usb3: SerialNumber: 0000:00:16.2
[ 12.557858] hub 3-0:1.0: USB hub found
[ 12.557884] hub 3-0:1.0: 4 ports detected
[ 12.558447] ohci-pci 0000:00:12.0: OHCI PCI host controller
[ 12.558466] ohci-pci 0000:00:12.0: new USB bus registered, assigned bus number 4
[ 12.558608] ohci-pci 0000:00:12.0: irq 18, io mem 0xf01c8000
[ 12.558618] ohci-pci 0000:00:13.0: OHCI PCI host controller
[ 12.558635] ohci-pci 0000:00:13.0: new USB bus registered, assigned bus number 5
[ 12.558703] ohci-pci 0000:00:13.0: irq 18, io mem 0xf01c9000
[ 12.558710] ohci-pci 0000:00:14.5: OHCI PCI host controller
[ 12.558725] ohci-pci 0000:00:14.5: new USB bus registered, assigned bus number 6
[ 12.558793] ohci-pci 0000:00:14.5: irq 18, io mem 0xf01ca000
[ 12.558800] ohci-pci 0000:00:16.0: OHCI PCI host controller
[ 12.558814] ohci-pci 0000:00:16.0: new USB bus registered, assigned bus number 7
[ 12.558888] ohci-pci 0000:00:16.0: irq 18, io mem 0xf01cb000
[ 12.635709] usb usb7: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 12.635716] usb usb7: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.635718] usb usb7: Product: OHCI PCI host controller
[ 12.635720] usb usb7: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ohci_hcd
[ 12.635721] usb usb7: SerialNumber: 0000:00:16.0
[ 12.636145] hub 7-0:1.0: USB hub found
[ 12.636172] hub 7-0:1.0: 4 ports detected
[ 12.636849] usb usb5: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 12.636851] usb usb5: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.636853] usb usb5: Product: OHCI PCI host controller
[ 12.636854] usb usb5: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ohci_hcd
[ 12.636856] usb usb5: SerialNumber: 0000:00:13.0
[ 12.639549] hub 5-0:1.0: USB hub found
[ 12.639579] hub 5-0:1.0: 5 ports detected
[ 12.640465] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 12.640468] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.640470] usb usb4: Product: OHCI PCI host controller
[ 12.640471] usb usb4: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ohci_hcd
[ 12.640473] usb usb4: SerialNumber: 0000:00:12.0
[ 12.640848] hub 4-0:1.0: USB hub found
[ 12.640874] hub 4-0:1.0: 5 ports detected
[ 12.644079] usb usb6: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 12.644084] usb usb6: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.644086] usb usb6: Product: OHCI PCI host controller
[ 12.644088] usb usb6: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ohci_hcd
[ 12.644089] usb usb6: SerialNumber: 0000:00:14.5
[ 12.644484] hub 6-0:1.0: USB hub found
[ 12.644509] hub 6-0:1.0: 2 ports detected
[ 12.658738] r8169 0000:04:00.0: enabling device (0000 -> 0003)
[ 12.697540] xhci_hcd 0000:03:00.0: xHCI Host Controller
[ 12.697566] xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 8
[ 12.709878] r8169 0000:04:00.0 eth0: RTL8168f/8111f, 08:60:6e:74:7a:51, XID 480, IRQ 28
[ 12.709886] r8169 0000:04:00.0 eth0: jumbo features [frames: 9194 bytes, tx checksumming: ko]
[ 12.735756] 1
[ 12.735806] 2
[ 12.736276] snd_hda_intel 0000:00:01.1: Force to non-snoop mode
[ 12.736282] 3
[ 12.736283] 4
[ 12.736284] 5
[ 12.736284] 7
[ 12.736288] 8
[ 12.736288] 9
[ 12.740797] 1
[ 12.740846] 2
[ 12.741376] 3
[ 12.741377] 4
[ 12.741378] 5
[ 12.741379] 7
[ 12.741382] 8
[ 12.741383] 9
[ 12.769260] input: HDA ATI HDMI HDMI/DP,pcm=3 as /devices/pci0000:00/0000:00:01.1/sound/card0/input1
[ 12.769555] input: HDA ATI HDMI HDMI/DP,pcm=7 as /devices/pci0000:00/0000:00:01.1/sound/card0/input2
[ 12.784439] snd_hda_codec_realtek hdaudioC1D0: ALC892: SKU not ready 0x00000100
[ 12.785682] snd_hda_codec_realtek hdaudioC1D0: autoconfig for ALC892: line_outs=4 (0x14/0x16/0x15/0x17/0x0) type:line
[ 12.785688] snd_hda_codec_realtek hdaudioC1D0: speaker_outs=0 (0x0/0x0/0x0/0x0/0x0)
[ 12.785691] snd_hda_codec_realtek hdaudioC1D0: hp_outs=1 (0x1b/0x0/0x0/0x0/0x0)
[ 12.785693] snd_hda_codec_realtek hdaudioC1D0: mono: mono_out=0x0
[ 12.785694] snd_hda_codec_realtek hdaudioC1D0: dig-out=0x1e/0x0
[ 12.785696] snd_hda_codec_realtek hdaudioC1D0: inputs:
[ 12.785697] snd_hda_codec_realtek hdaudioC1D0: Rear Mic=0x18
[ 12.785699] snd_hda_codec_realtek hdaudioC1D0: Front Mic=0x19
[ 12.785700] snd_hda_codec_realtek hdaudioC1D0: Line=0x1a
[ 12.785701] snd_hda_codec_realtek hdaudioC1D0: CD=0x1c
[ 12.807528] xhci_hcd 0000:03:00.0: hcc params 0x0200f180 hci version 0x96 quirks 0x0000000000080010
[ 12.813178] input: HD-Audio Generic Rear Mic as /devices/pci0000:00/0000:00:14.2/sound/card1/input3
[ 12.813514] input: HD-Audio Generic Front Mic as /devices/pci0000:00/0000:00:14.2/sound/card1/input4
[ 12.813782] input: HD-Audio Generic Line as /devices/pci0000:00/0000:00:14.2/sound/card1/input5
[ 12.814048] input: HD-Audio Generic Line Out Front as /devices/pci0000:00/0000:00:14.2/sound/card1/input6
[ 12.814317] input: HD-Audio Generic Line Out Surround as /devices/pci0000:00/0000:00:14.2/sound/card1/input7
[ 12.814575] input: HD-Audio Generic Line Out CLFE as /devices/pci0000:00/0000:00:14.2/sound/card1/input8
[ 12.814833] input: HD-Audio Generic Line Out Side as /devices/pci0000:00/0000:00:14.2/sound/card1/input9
[ 12.815087] input: HD-Audio Generic Front Headphone as /devices/pci0000:00/0000:00:14.2/sound/card1/input10
[ 12.816233] xhci_hcd 0000:03:00.0: xHCI Host Controller
[ 12.816249] xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 9
[ 12.816260] xhci_hcd 0000:03:00.0: Host supports USB 3.0 SuperSpeed
[ 12.830985] usb usb8: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 12.830993] usb usb8: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.830995] usb usb8: Product: xHCI Host Controller
[ 12.830996] usb usb8: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 xhci-hcd
[ 12.830998] usb usb8: SerialNumber: 0000:03:00.0
[ 12.835543] hub 8-0:1.0: USB hub found
[ 12.842296] hub 8-0:1.0: 2 ports detected
[ 12.850842] usb usb9: We don't know the algorithms for LPM for this host, disabling LPM.
[ 12.850983] usb usb9: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 6.03
[ 12.850986] usb usb9: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.850988] usb usb9: Product: xHCI Host Controller
[ 12.850989] usb usb9: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 xhci-hcd
[ 12.850991] usb usb9: SerialNumber: 0000:03:00.0
[ 12.856278] r8169 0000:04:00.0 enp4s0: renamed from eth0
[ 12.862771] hub 9-0:1.0: USB hub found
[ 12.873601] hub 9-0:1.0: 2 ports detected
[ 13.025026] usb 4-1: new low-speed USB device number 2 using ohci-pci
[ 13.103377] r8169 0000:04:00.0: Direct firmware load for rtl_nic/rtl8168f-1.fw failed with error -2
[ 13.103389] r8169 0000:04:00.0: Unable to load firmware rtl_nic/rtl8168f-1.fw (-2)
[ 13.103907] RTL8211E Gigabit Ethernet r8169-0-400:00: attached PHY driver (mii_bus:phy_addr=r8169-0-400:00, irq=MAC)
[ 13.175505] r8169 0000:04:00.0 enp4s0: Link is Down
[ 13.179870] [drm] radeon kernel modesetting enabled.
[ 13.181586] [drm] initializing kernel modesetting (ARUBA 0x1002:0x9996 0x1002:0x9996 0x00).
[ 13.181652] ATOM BIOS: 113
[ 13.181757] radeon 0000:00:01.0: VRAM: 512M 0x0000000000000000 - 0x000000001FFFFFFF (512M used)
[ 13.181761] radeon 0000:00:01.0: GTT: 1024M 0x0000000020000000 - 0x000000005FFFFFFF
[ 13.181769] [drm] Detected VRAM RAM=512M, BAR=256M
[ 13.181770] [drm] RAM width 64bits DDR
[ 13.181947] [drm] radeon: 512M of VRAM memory ready
[ 13.181952] [drm] radeon: 1024M of GTT memory ready.
[ 13.181994] [drm] Loading ARUBA Microcode
[ 13.190350] [drm] Internal thermal controller without fan control
[ 13.191100] [drm] radeon: dpm initialized
[ 13.196015] [drm] Found VCE firmware/feedback version 50.0.1 / 17!
[ 13.196073] [drm] GART: num cpu pages 262144, num gpu pages 262144
[ 13.234337] [drm] GART: Restore entries: num cpu pages 262144, num gpu pages 262144
[ 13.237934] [drm] GART: Done restoring entries
[ 13.237938] [drm] PCIE GART of 1024M enabled (table at 0x00000000001D6000).
[ 13.238177] radeon 0000:00:01.0: WB enabled
[ 13.238180] radeon 0000:00:01.0: fence driver on ring 0 use gpu addr 0x0000000020000c00
[ 13.238558] radeon 0000:00:01.0: fence driver on ring 5 use gpu addr 0x0000000000075a18
[ 13.246045] usb 4-1: New USB device found, idVendor=413c, idProduct=2106, bcdDevice= 1.01
[ 13.246050] usb 4-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 13.246052] usb 4-1: Product: Dell QuietKey Keyboard
[ 13.246054] usb 4-1: Manufacturer: DELL
[ 13.253819] input: DELL Dell QuietKey Keyboard as /devices/pci0000:00/0000:00:12.0/usb4/4-1/4-1:1.0/0003:413C:2106.0001/input/input11
[ 13.260936] radeon 0000:00:01.0: fence driver on ring 6 use gpu addr 0x0000000020000c18
[ 13.260940] radeon 0000:00:01.0: fence driver on ring 7 use gpu addr 0x0000000020000c1c
[ 13.260942] radeon 0000:00:01.0: fence driver on ring 1 use gpu addr 0x0000000020000c04
[ 13.260944] radeon 0000:00:01.0: fence driver on ring 2 use gpu addr 0x0000000020000c08
[ 13.260945] radeon 0000:00:01.0: fence driver on ring 3 use gpu addr 0x0000000020000c0c
[ 13.260947] radeon 0000:00:01.0: fence driver on ring 4 use gpu addr 0x0000000020000c10
[ 13.262964] radeon 0000:00:01.0: radeon: MSI limited to 32-bit
[ 13.263151] radeon 0000:00:01.0: radeon: using MSI.
[ 13.263222] [drm] radeon: irq initialized.
[ 13.281648] [drm] ring test on 0 succeeded in 3 usecs
[ 13.281658] [drm] ring test on 3 succeeded in 4 usecs
[ 13.281665] [drm] ring test on 4 succeeded in 4 usecs
[ 13.295659] [drm] ring test on 5 succeeded in 2 usecs
[ 13.297656] [drm] UVD initialized successfully.
[ 13.313663] hid-generic 0003:413C:2106.0001: input,hidraw0: USB HID v1.10 Keyboard [DELL Dell QuietKey Keyboard] on usb-0000:00:12.0-1/input0
[ 13.446953] [drm] ring test on 6 succeeded in 18 usecs
[ 13.446967] [drm] ring test on 7 succeeded in 3 usecs
[ 13.446968] [drm] VCE initialized successfully.
[ 13.447122] snd_hda_intel 0000:00:01.1: bound 0000:00:01.0 (ops radeon_audio_component_bind_ops [radeon])
[ 13.447293] [drm] ib test on ring 0 succeeded in 0 usecs
[ 13.447346] [drm] ib test on ring 3 succeeded in 0 usecs
[ 13.447396] [drm] ib test on ring 4 succeeded in 0 usecs
[ 13.465099] [drm] ib test on ring 5 succeeded
[ 13.481132] [drm] ib test on ring 6 succeeded in 1 usecs
[ 13.497085] [drm] ib test on ring 7 succeeded in 1 usecs
[ 13.500056] [drm] Radeon Display Connectors
[ 13.500060] [drm] Connector 0:
[ 13.500061] [drm] DP-1
[ 13.500062] [drm] HPD1
[ 13.500062] [drm] DDC: 0x6530 0x6530 0x6534 0x6534 0x6538 0x6538 0x653c 0x653c
[ 13.500065] [drm] Encoders:
[ 13.500065] [drm] DFP1: INTERNAL_UNIPHY2
[ 13.500066] [drm] Connector 1:
[ 13.500067] [drm] VGA-1
[ 13.500068] [drm] HPD2
[ 13.500069] [drm] DDC: 0x6540 0x6540 0x6544 0x6544 0x6548 0x6548 0x654c 0x654c
[ 13.500070] [drm] Encoders:
[ 13.500071] [drm] CRT1: INTERNAL_UNIPHY2
[ 13.500072] [drm] CRT1: NUTMEG
[ 13.500072] [drm] Connector 2:
[ 13.500073] [drm] HDMI-A-1
[ 13.500074] [drm] HPD3
[ 13.500075] [drm] DDC: 0x6550 0x6550 0x6554 0x6554 0x6558 0x6558 0x655c 0x655c
[ 13.500076] [drm] Encoders:
[ 13.500077] [drm] DFP2: INTERNAL_UNIPHY
[ 13.741081] usb 4-2: new low-speed USB device number 3 using ohci-pci
[ 13.779818] [drm] fb mappable at 0xE03E9000
[ 13.779826] [drm] vram apper at 0xE0000000
[ 13.779828] [drm] size 5242880
[ 13.779830] [drm] fb depth is 24
[ 13.779832] [drm] pitch is 5120
[ 13.780398] fbcon: radeondrmfb (fb0) is primary device
[ 13.936158] usb 4-2: New USB device found, idVendor=046d, idProduct=c016, bcdDevice= 3.40
[ 13.936167] usb 4-2: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 13.936170] usb 4-2: Product: Optical USB Mouse
[ 13.936173] usb 4-2: Manufacturer: Logitech
[ 13.945345] input: Logitech Optical USB Mouse as /devices/pci0000:00/0000:00:12.0/usb4/4-2/4-2:1.0/0003:046D:C016.0002/input/input12
[ 13.946178] hid-generic 0003:046D:C016.0002: input,hidraw1: USB HID v1.10 Mouse [Logitech Optical USB Mouse] on usb-0000:00:12.0-2/input0
[ 13.968191] Console: switching to colour frame buffer device 160x64
[ 13.973460] radeon 0000:00:01.0: [drm] fb0: radeondrmfb frame buffer device
[ 13.981574] [drm] Initialized radeon 2.50.0 20080528 for 0000:00:01.0 on minor 0
[ 15.876613] r8169 0000:04:00.0 enp4s0: Link is Up - 1Gbps/Full - flow control rx/tx
[ 15.876628] IPv6: ADDRCONF(NETDEV_CHANGE): enp4s0: link becomes ready
[ 16.713266] [drm] amdgpu kernel modesetting enabled.
[ 17.110008] memfd_create() without MFD_EXEC nor MFD_NOEXEC_SEAL, pid=249 'systemd'
[-- Attachment #3: Type: text/plain, Size: 161 bytes --]
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-17 11:19 ` Paul Menzel
0 siblings, 0 replies; 236+ messages in thread
From: Paul Menzel @ 2023-04-17 11:19 UTC (permalink / raw)
To: Thomas Gleixner
Cc: linux-kernel, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
[-- Attachment #1: Type: text/plain, Size: 1532 bytes --]
Dear Thomas,
Am 15.04.23 um 01:44 schrieb Thomas Gleixner:
> This is a complete rework of the parallel bringup patch series (V17)
>
> https://lore.kernel.org/lkml/20230328195758.1049469-1-usama.arif@bytedance.com
>
> to address the issues which were discovered in review:
[…]
Thank you very much for your rework.
I tested this on the ASUS F2A85-M PRO, and get a delay of ten seconds.
```
[…]
[ 0.258193] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD
Graphics (family: 0x15, model: 0x13, stepping: 0x1)
[…]
[ 0.259329] smp: Bringing up secondary CPUs ...
[ 0.259527] x86: Booting SMP configuration:
[ 0.259528] .... node #0, CPUs: #1
[ 0.261007] After schedule_preempt_disabled
[ 10.260990] CPU1 failed to report alive state
[ 10.261070] smp: Brought up 1 node, 1 CPU
[ 10.261073] smpboot: Max logical packages: 2
[ 10.261074] smpboot: Total of 1 processors activated (7800.54 BogoMIPS)
[ 10.261601] devtmpfs: initialized
[ 10.261697] x86/mm: Memory block size: 128MB
```
This delay has been there with v6.3-rc6-46-gde4664485abbc and some
custom (printk) patches on top and merging dwmw2/parallel-6.2-rc3-v16
into it. I only tested this. I think dwmw2/parallel-6.2-v17 failed to
build for me, when trying to merge it into Linus’ master version at that
time. I didn’t come around to report it, and you posted your rework, so
I am replying here.
I am going to try your branch directly in the next days, but just wanted
to report back already.
Kind regards,
Paul
[-- Attachment #2: kodi-linux-6.3-rc6-smp-tglx.txt --]
[-- Type: text/plain, Size: 61769 bytes --]
[ 0.000000] Linux version 6.3.0-rc6-00311-gde8224969f66 (root@bf16f3646a84) (gcc (Debian 11.2.0-12) 11.2.0, GNU ld (GNU Binutils for Debian) 2.40) #446 SMP PREEMPT_DYNAMIC Sat Apr 15 14:12:29 UTC 2023
[ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc6-00311-gde8224969f66 root=/dev/sda3 rw quiet noisapnp cryptomgr.notests ipv6.disable_ipv6=1 selinux=0
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256
[ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format.
[ 0.000000] signal: max sigframe size: 1776
[ 0.000000] BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
[ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005fe4cfff] usable
[ 0.000000] BIOS-e820: [mem 0x000000005fe4d000-0x000000007fffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000f8000000-0x00000000fbffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000fec10000-0x00000000fec10fff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000017effffff] usable
[ 0.000000] NX (Execute Disable) protection: active
[ 0.000000] SMBIOS 3.0.0 present.
[ 0.000000] DMI: ASUS F2A85-M_PRO/F2A85-M_PRO, BIOS 4.18-9-g9917d2d915 04/17/2023
[ 0.000000] tsc: Fast TSC calibration using PIT
[ 0.000000] tsc: Initial usec timer 6035615
[ 0.000000] tsc: Detected 3900.273 MHz processor
[ 0.000756] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
[ 0.000759] e820: remove [mem 0x000a0000-0x000fffff] usable
[ 0.000763] last_pfn = 0x17f000 max_arch_pfn = 0x400000000
[ 0.000768] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT
[ 0.000942] last_pfn = 0x5fe4d max_arch_pfn = 0x400000000
[ 0.004000] Using GB pages for direct mapping
[ 0.004000] ACPI: Early table checksum verification disabled
[ 0.004000] ACPI: RSDP 0x00000000000F6830 000024 (v02 COREv4)
[ 0.004000] ACPI: XSDT 0x000000005FE5A0E0 000074 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: FACP 0x000000005FE5BBC0 000114 (v06 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: DSDT 0x000000005FE5A280 00193A (v02 COREv4 COREBOOT 00010001 INTL 20200925)
[ 0.004000] ACPI: FACS 0x000000005FE5A240 000040
[ 0.004000] ACPI: FACS 0x000000005FE5A240 000040
[ 0.004000] ACPI: SSDT 0x000000005FE5BCE0 00008A (v02 COREv4 COREBOOT 0000002A CORE 20200925)
[ 0.004000] ACPI: MCFG 0x000000005FE5BD70 00003C (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: APIC 0x000000005FE5BDB0 000062 (v03 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: HPET 0x000000005FE5BE20 000038 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: HEST 0x000000005FE5BE60 0001D0 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: IVRS 0x000000005FE5C030 000070 (v02 AMD AMDIOMMU 00000001 AMD 00000000)
[ 0.004000] ACPI: SSDT 0x000000005FE5C0A0 00051F (v02 AMD ALIB 00000001 MSFT 04000000)
[ 0.004000] ACPI: SSDT 0x000000005FE5C5C0 0006B2 (v01 AMD POWERNOW 00000001 AMD 00000001)
[ 0.004000] ACPI: VFCT 0x000000005FE5CC80 00F269 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: Reserving FACP table memory at [mem 0x5fe5bbc0-0x5fe5bcd3]
[ 0.004000] ACPI: Reserving DSDT table memory at [mem 0x5fe5a280-0x5fe5bbb9]
[ 0.004000] ACPI: Reserving FACS table memory at [mem 0x5fe5a240-0x5fe5a27f]
[ 0.004000] ACPI: Reserving FACS table memory at [mem 0x5fe5a240-0x5fe5a27f]
[ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe5bce0-0x5fe5bd69]
[ 0.004000] ACPI: Reserving MCFG table memory at [mem 0x5fe5bd70-0x5fe5bdab]
[ 0.004000] ACPI: Reserving APIC table memory at [mem 0x5fe5bdb0-0x5fe5be11]
[ 0.004000] ACPI: Reserving HPET table memory at [mem 0x5fe5be20-0x5fe5be57]
[ 0.004000] ACPI: Reserving HEST table memory at [mem 0x5fe5be60-0x5fe5c02f]
[ 0.004000] ACPI: Reserving IVRS table memory at [mem 0x5fe5c030-0x5fe5c09f]
[ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe5c0a0-0x5fe5c5be]
[ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe5c5c0-0x5fe5cc71]
[ 0.004000] ACPI: Reserving VFCT table memory at [mem 0x5fe5cc80-0x5fe6bee8]
[ 0.004000] No NUMA configuration found
[ 0.004000] Faking a node at [mem 0x0000000000000000-0x000000017effffff]
[ 0.004000] NODE_DATA(0) allocated [mem 0x17efe7000-0x17effdfff]
[ 0.004000] Zone ranges:
[ 0.004000] DMA [mem 0x0000000000001000-0x0000000000ffffff]
[ 0.004000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]
[ 0.004000] Normal [mem 0x0000000100000000-0x000000017effffff]
[ 0.004000] Device empty
[ 0.004000] Movable zone start for each node
[ 0.004000] Early memory node ranges
[ 0.004000] node 0: [mem 0x0000000000001000-0x000000000009efff]
[ 0.004000] node 0: [mem 0x0000000000100000-0x000000005fe4cfff]
[ 0.004000] node 0: [mem 0x0000000100000000-0x000000017effffff]
[ 0.004000] Initmem setup node 0 [mem 0x0000000000001000-0x000000017effffff]
[ 0.004000] On node 0, zone DMA: 1 pages in unavailable ranges
[ 0.004000] On node 0, zone DMA: 97 pages in unavailable ranges
[ 0.004000] On node 0, zone Normal: 435 pages in unavailable ranges
[ 0.004000] On node 0, zone Normal: 4096 pages in unavailable ranges
[ 0.004000] ACPI: PM-Timer IO Port: 0x818
[ 0.004000] ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1])
[ 0.004000] IOAPIC[0]: apic_id 4, version 33, address 0xfec00000, GSI 0-23
[ 0.004000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[ 0.004000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 low level)
[ 0.004000] ACPI: Using ACPI (MADT) for SMP configuration information
[ 0.004000] ACPI: HPET id: 0x10228210 base: 0xfed00000
[ 0.004000] smpboot: Allowing 2 CPUs, 0 hotplug CPUs
[ 0.004000] smpboot: smpboot: XXX end of prefill_possible_map
[ 0.004000] After prefill_possible_map
[ 0.004000] After init_cpu_to_node
[ 0.004000] After init_gi_nodes
[ 0.004000] After io_apic_init_mappings
[ 0.004000] After x86_init.hyper.guest_late_init
[ 0.004000] [mem 0x80000000-0xf7ffffff] available for PCI devices
[ 0.004000] After e820
[ 0.004000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns
[ 0.004000] After unwind_init
[ 0.004000] After setup_arch
[ 0.004000] After setup_command_line
[ 0.004000] After setup_nr_cpu_ids
[ 0.004000] setup_percpu: NR_CPUS:64 nr_cpumask_bits:2 nr_cpu_ids:2 nr_node_ids:1
[ 0.004000] percpu: Embedded 55 pages/cpu s188328 r8192 d28760 u1048576
[ 0.004000] pcpu-alloc: s188328 r8192 d28760 u1048576 alloc=1*2097152
[ 0.004000] pcpu-alloc: [0] 0 1
[ 0.004000] After setup_per_cpu_areas
[ 0.004000] After smp_perpare_boot_cpu
[ 0.004000] After boot_cpu_hotplug_init
[ 0.004000] Fallback order for Node 0: 0
[ 0.004000] Built 1 zonelists, mobility grouping on. Total pages: 898451
[ 0.004000] Policy zone: Normal
[ 0.004000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc6-00311-gde8224969f66 root=/dev/sda3 rw quiet noisapnp cryptomgr.notests ipv6.disable_ipv6=1 selinux=0
[ 0.004000] Unknown kernel command line parameters "noisapnp BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc6-00311-gde8224969f66", will be passed to user space.
[ 0.004000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[ 0.004000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[ 0.004000] mem auto-init: stack:off, heap alloc:off, heap free:off
[ 0.004000] stackdepot: allocating hash table via alloc_large_system_hash
[ 0.004000] stackdepot hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[ 0.004000] software IO TLB: area num 2.
[ 0.004000] Memory: 3477160K/3651500K available (14336K kernel code, 2340K rwdata, 5308K rodata, 2908K init, 11060K bss, 174080K reserved, 0K cma-reserved)
[ 0.004000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
[ 0.004000] After mm_init
[ 0.004000] After poking_init
[ 0.004000] ftrace: allocating 38664 entries in 152 pages
[ 0.004000] ftrace: allocated 152 pages with 3 groups
[ 0.004000] Dynamic Preempt: full
[ 0.004000] After sched_init
[ 0.004000] rcu: Preemptible hierarchical RCU implementation.
[ 0.004000] rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=2.
[ 0.004000] Trampoline variant of Tasks RCU enabled.
[ 0.004000] Rude variant of Tasks RCU enabled.
[ 0.004000] Tracing variant of Tasks RCU enabled.
[ 0.004000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[ 0.004000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2
[ 0.004000] After rcu_init
[ 0.004000] NR_IRQS: 4352, nr_irqs: 440, preallocated irqs: 16
[ 0.004000] rcu: srcu_init: Setting srcu_struct sizes based on contention.
[ 0.004000] After random_init()
[ 0.004000] After boot_init_stack_canary
[ 0.004000] spurious 8259A interrupt: IRQ7.
[ 0.004000] Console: colour VGA+ 80x25
[ 0.004000] printk: console [tty0] enabled
[ 0.004000] ACPI: Core revision 20221020
[ 0.004000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 133484873504 ns
[ 0.004000] APIC: Switch to symmetric I/O mode setup
[ 0.004000] AMD-Vi: Using global IVHD EFR:0x0, EFR2:0x0
[ 0.004000] APIC: Done
[ 0.004000] Before apic_bsb_setup
[ 0.004000] check_timer begin
[ 0.004000] check_timer after local_irq_disable
[ 0.004000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[ 0.004000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x7070b6fd6fc, max_idle_ns: 881590611710 ns
[ 0.144982] Calibrating delay loop (skipped), value calculated using timer frequency.. 7800.54 BogoMIPS (lpj=15601092)
[ 0.144986] pid_max: default: 32768 minimum: 301
[ 0.145079] LSM: initializing lsm=capability
[ 0.145173] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[ 0.145189] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[ 0.145564] Bit 30 in CPUID ECX not set.
[ 0.145587] Last level iTLB entries: 4KB 512, 2MB 1024, 4MB 512
[ 0.145589] Last level dTLB entries: 4KB 1024, 2MB 1024, 4MB 512, 1GB 0
[ 0.145594] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization
[ 0.145597] Spectre V2 : Mitigation: Retpolines
[ 0.145598] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch
[ 0.145599] Spectre V2 : Spectre v2 / SpectreRSB : Filling RSB on VMEXIT
[ 0.145600] Spectre V2 : Enabling Speculation Barrier for firmware calls
[ 0.145600] RETBleed: Mitigation: untrained return thunk
[ 0.145602] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier
[ 0.145604] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl
[ 0.150046] Freeing SMP alternatives memory: 32K
[ 0.150051] After check_bugs
[ 0.150052] After acpi_subsystem_init
[ 0.150053] After arch_post_acpi_subsys_init
[ 0.150054] After rcu_scheduler_starting
[ 0.150126] After find_task_by_pid_ns and PF_NO_SETAFFINITY
[ 0.150131] After numa_default_policy
[ 0.150151] After rcu_read_lock
[ 0.150152] After rcu_read_unlock
[ 0.150153] After kthreadd_done
[ 0.150165] smpboot: Start of smp_prepare_cpus_common
[ 0.150167] smpboot: smpboot: zalloc 0
[ 0.150168] smpboot: smpboot: zalloc 1
[ 0.150169] smpboot: smpboot: After set_sched_topology()
[ 0.150171] smpboot: smpboot: After smp_sanity_check()
[ 0.150172] smpboot: smpboot: Before x86_init.timers.setup_percpu_clockev()
[ 0.258192] smpboot: smpboot: After x86_init.timers.setup_percpu_clockev()
[ 0.258193] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD Graphics (family: 0x15, model: 0x13, stepping: 0x1)
[ 0.258425] cblist_init_generic: Setting adjustable number of callback queues.
[ 0.258427] cblist_init_generic: Setting shift to 1 and lim to 1.
[ 0.258458] cblist_init_generic: Setting shift to 1 and lim to 1.
[ 0.258484] cblist_init_generic: Setting shift to 1 and lim to 1.
[ 0.258513] Performance Events: Fam15h core perfctr, AMD PMU driver.
[ 0.258536] ... version: 0
[ 0.258537] ... bit width: 48
[ 0.258538] ... generic registers: 6
[ 0.258539] ... value mask: 0000ffffffffffff
[ 0.258540] ... max period: 00007fffffffffff
[ 0.258541] ... fixed-purpose events: 0
[ 0.258542] ... event mask: 000000000000003f
[ 0.258662] rcu: Hierarchical SRCU implementation.
[ 0.258663] rcu: Max phase no-delay instances is 1000.
[ 0.259257] NMI watchdog: Enabled. Permanently consumes one hw-PMU counter.
[ 0.259329] smp: Bringing up secondary CPUs ...
[ 0.259527] x86: Booting SMP configuration:
[ 0.259528] .... node #0, CPUs: #1
[ 0.261007] After schedule_preempt_disabled
[ 10.260990] CPU1 failed to report alive state
[ 10.261070] smp: Brought up 1 node, 1 CPU
[ 10.261073] smpboot: Max logical packages: 2
[ 10.261074] smpboot: Total of 1 processors activated (7800.54 BogoMIPS)
[ 10.261601] devtmpfs: initialized
[ 10.261697] x86/mm: Memory block size: 128MB
[ 10.262788] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[ 10.262795] futex hash table entries: 512 (order: 3, 32768 bytes, linear)
[ 10.262883] pinctrl core: initialized pinctrl subsystem
[ 10.262953] PM: RTC time: 07:39:50, date: 2023-04-17
[ 10.263709] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[ 10.263937] audit: initializing netlink subsys (disabled)
[ 10.264192] thermal_sys: Registered thermal governor 'fair_share'
[ 10.264193] thermal_sys: Registered thermal governor 'bang_bang'
[ 10.264194] thermal_sys: Registered thermal governor 'step_wise'
[ 10.264195] thermal_sys: Registered thermal governor 'user_space'
[ 10.264215] cpuidle: using governor ladder
[ 10.264220] cpuidle: using governor menu
[ 10.264427] PCI: MMCONFIG for domain 0000 [bus 00-3f] at [mem 0xf8000000-0xfbffffff] (base 0xf8000000)
[ 10.264432] PCI: MMCONFIG at [mem 0xf8000000-0xfbffffff] reserved as E820 entry
[ 10.264444] PCI: Using configuration type 1 for base access
[ 10.264646] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.
[ 10.269064] audit: type=2000 audit(1681717190.140:1): state=initialized audit_enabled=0 res=1
[ 10.281080] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages
[ 10.281084] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page
[ 10.281085] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages
[ 10.281086] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page
[ 10.286166] cryptd: max_cpu_qlen set to 1000
[ 10.289615] ACPI: Added _OSI(Module Device)
[ 10.289617] ACPI: Added _OSI(Processor Device)
[ 10.289618] ACPI: Added _OSI(3.0 _SCP Extensions)
[ 10.289620] ACPI: Added _OSI(Processor Aggregator Device)
[ 10.293942] ACPI: DSDT successfully acquired and loaded
[ 10.295626] ACPI: 4 ACPI AML tables successfully acquired and loaded
[ 10.297161] ACPI: Interpreter enabled
[ 10.297184] ACPI: PM: (supports S0 S1 S3 S5)
[ 10.297186] ACPI: Using IOAPIC for interrupt routing
[ 10.297236] HEST: Table parsing has been initialized.
[ 10.297258] GHES: Failed to enable APEI firmware first mode.
[ 10.297261] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[ 10.297262] PCI: Ignoring E820 reservations for host bridge windows
[ 10.297510] ACPI: Enabled 8 GPEs in block 00 to 1F
[ 10.303230] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[ 10.303241] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]
[ 10.303320] acpi PNP0A03:00: _OSC: OS now controls [PME AER PCIeCapability LTR]
[ 10.303335] acpi PNP0A03:00: [Firmware Info]: MMCONFIG for domain 0000 [bus 00-3f] only partially covers this bridge
[ 10.303414] acpi PNP0A03:00: host bridge window expanded to [io 0x0000-0x0cf7 window]; [io 0x03b0-0x03df window] ignored
[ 10.303651] PCI host bridge to bus 0000:00
[ 10.303653] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]
[ 10.303655] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]
[ 10.303658] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000dffff]
[ 10.303660] pci_bus 0000:00: root bus resource [mem 0x80000000-0xffffffff]
[ 10.303662] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 10.303686] pci 0000:00:00.0: [1022:1410] type 00 class 0x060000
[ 10.303830] pci 0000:00:00.2: [1022:1419] type 00 class 0x080600
[ 10.303921] pci 0000:00:01.0: [1002:9996] type 00 class 0x030000
[ 10.303929] pci 0000:00:01.0: reg 0x10: [mem 0xe0000000-0xefffffff pref]
[ 10.303934] pci 0000:00:01.0: reg 0x14: [io 0x1000-0x10ff]
[ 10.303939] pci 0000:00:01.0: reg 0x18: [mem 0xf0180000-0xf01bffff]
[ 10.303955] pci 0000:00:01.0: enabling Extended Tags
[ 10.303965] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]
[ 10.303983] pci 0000:00:01.0: supports D1 D2
[ 10.304048] pci 0000:00:01.1: [1002:9902] type 00 class 0x040300
[ 10.304056] pci 0000:00:01.1: reg 0x10: [mem 0xf01c0000-0xf01c3fff]
[ 10.304077] pci 0000:00:01.1: enabling Extended Tags
[ 10.304101] pci 0000:00:01.1: supports D1 D2
[ 10.304185] pci 0000:00:11.0: [1022:7801] type 00 class 0x010601
[ 10.304198] pci 0000:00:11.0: reg 0x10: [io 0x1410-0x1417]
[ 10.304206] pci 0000:00:11.0: reg 0x14: [io 0x1420-0x1423]
[ 10.304213] pci 0000:00:11.0: reg 0x18: [io 0x1418-0x141f]
[ 10.304221] pci 0000:00:11.0: reg 0x1c: [io 0x1424-0x1427]
[ 10.304228] pci 0000:00:11.0: reg 0x20: [io 0x1400-0x140f]
[ 10.304235] pci 0000:00:11.0: reg 0x24: [mem 0xf01cc000-0xf01cc7ff]
[ 10.304392] pci 0000:00:12.0: [1022:7807] type 00 class 0x0c0310
[ 10.304406] pci 0000:00:12.0: reg 0x10: [mem 0xf01c8000-0xf01c8fff]
[ 10.304587] pci 0000:00:12.2: [1022:7808] type 00 class 0x0c0320
[ 10.304601] pci 0000:00:12.2: reg 0x10: [mem 0xf01cd000-0xf01cd0ff]
[ 10.304666] pci 0000:00:12.2: supports D1 D2
[ 10.304667] pci 0000:00:12.2: PME# supported from D0 D1 D2 D3hot
[ 10.304668] pci 0000:00:12.2: pme_poll = true
[ 10.304669] pci 0000:00:12.2: after device_set_wakeup_capable()
[ 10.304673] pci 0000:00:12.2: after pci_pme_active()
[ 10.304814] pci 0000:00:13.0: [1022:7807] type 00 class 0x0c0310
[ 10.304827] pci 0000:00:13.0: reg 0x10: [mem 0xf01c9000-0xf01c9fff]
[ 10.305566] pci 0000:00:13.2: [1022:7808] type 00 class 0x0c0320
[ 10.305583] pci 0000:00:13.2: reg 0x10: [mem 0xf01ce000-0xf01ce0ff]
[ 10.305654] pci 0000:00:13.2: supports D1 D2
[ 10.305656] pci 0000:00:13.2: PME# supported from D0 D1 D2 D3hot
[ 10.305657] pci 0000:00:13.2: pme_poll = true
[ 10.305659] pci 0000:00:13.2: after device_set_wakeup_capable()
[ 10.305662] pci 0000:00:13.2: after pci_pme_active()
[ 10.305803] pci 0000:00:14.0: [1022:780b] type 00 class 0x0c0500
[ 10.305975] pci 0000:00:14.2: [1022:780d] type 00 class 0x040300
[ 10.305992] pci 0000:00:14.2: reg 0x10: [mem 0xf01c4000-0xf01c7fff 64bit]
[ 10.306047] pci 0000:00:14.2: PME# supported from D0 D3hot D3cold
[ 10.306049] pci 0000:00:14.2: pme_poll = true
[ 10.306050] pci 0000:00:14.2: after device_set_wakeup_capable()
[ 10.306052] pci 0000:00:14.2: after pci_pme_active()
[ 10.306185] pci 0000:00:14.3: [1022:780e] type 00 class 0x060100
[ 10.306360] pci 0000:00:14.4: [1022:780f] type 01 class 0x060401
[ 10.306502] pci 0000:00:14.5: [1022:7809] type 00 class 0x0c0310
[ 10.306515] pci 0000:00:14.5: reg 0x10: [mem 0xf01ca000-0xf01cafff]
[ 10.306682] pci 0000:00:15.0: [1022:43a0] type 01 class 0x060400
[ 10.306711] pci 0000:00:15.0: enabling Extended Tags
[ 10.306751] pci 0000:00:15.0: supports D1 D2
[ 10.306911] pci 0000:00:15.1: [1022:43a1] type 01 class 0x060400
[ 10.306942] pci 0000:00:15.1: enabling Extended Tags
[ 10.306981] pci 0000:00:15.1: supports D1 D2
[ 10.307137] pci 0000:00:15.2: [1022:43a2] type 01 class 0x060400
[ 10.307166] pci 0000:00:15.2: enabling Extended Tags
[ 10.307205] pci 0000:00:15.2: supports D1 D2
[ 10.307280] pci 0000:00:16.0: [1022:7807] type 00 class 0x0c0310
[ 10.307293] pci 0000:00:16.0: reg 0x10: [mem 0xf01cb000-0xf01cbfff]
[ 10.307467] pci 0000:00:16.2: [1022:7808] type 00 class 0x0c0320
[ 10.307481] pci 0000:00:16.2: reg 0x10: [mem 0xf01cf000-0xf01cf0ff]
[ 10.307546] pci 0000:00:16.2: supports D1 D2
[ 10.307547] pci 0000:00:16.2: PME# supported from D0 D1 D2 D3hot
[ 10.307548] pci 0000:00:16.2: pme_poll = true
[ 10.307549] pci 0000:00:16.2: after device_set_wakeup_capable()
[ 10.307552] pci 0000:00:16.2: after pci_pme_active()
[ 10.307691] pci 0000:00:18.0: [1022:1400] type 00 class 0x060000
[ 10.307758] pci 0000:00:18.1: [1022:1401] type 00 class 0x060000
[ 10.307819] pci 0000:00:18.2: [1022:1402] type 00 class 0x060000
[ 10.307880] pci 0000:00:18.3: [1022:1403] type 00 class 0x060000
[ 10.308016] pci 0000:00:18.4: [1022:1404] type 00 class 0x060000
[ 10.308079] pci 0000:00:18.5: [1022:1405] type 00 class 0x060000
[ 10.308153] pci_bus 0000:01: extended config space not accessible
[ 10.308218] pci 0000:00:14.4: PCI bridge to [bus 01] (subtractive decode)
[ 10.308227] pci 0000:00:14.4: bridge window [io 0x0000-0x0cf7 window] (subtractive decode)
[ 10.308230] pci 0000:00:14.4: bridge window [io 0x0d00-0xffff window] (subtractive decode)
[ 10.308232] pci 0000:00:14.4: bridge window [mem 0x000a0000-0x000dffff] (subtractive decode)
[ 10.308235] pci 0000:00:14.4: bridge window [mem 0x80000000-0xffffffff] (subtractive decode)
[ 10.308283] pci 0000:00:15.0: PCI bridge to [bus 02]
[ 10.308368] pci 0000:03:00.0: [1b21:1042] type 00 class 0x0c0330
[ 10.308404] pci 0000:03:00.0: reg 0x10: [mem 0xf0000000-0xf0007fff 64bit]
[ 10.308579] pci 0000:03:00.0: PME# supported from D3hot D3cold
[ 10.308581] pci 0000:03:00.0: pme_poll = true
[ 10.308582] pci 0000:03:00.0: after device_set_wakeup_capable()
[ 10.308587] pci 0000:03:00.0: after pci_pme_active()
[ 10.308625] pci 0000:03:00.0: 2.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s PCIe x1 link at 0000:00:15.1 (capable of 4.000 Gb/s with 5.0 GT/s PCIe x1 link)
[ 10.317133] pci 0000:00:15.1: PCI bridge to [bus 03]
[ 10.317145] pci 0000:00:15.1: bridge window [mem 0xf0000000-0xf00fffff]
[ 10.317154] pci 0000:00:15.2: bridge configuration invalid ([bus 00-00]), reconfiguring
[ 10.317278] pci 0000:04:00.0: [10ec:8168] type 00 class 0x020000
[ 10.317296] pci 0000:04:00.0: reg 0x10: [io 0x0000-0x00ff]
[ 10.317318] pci 0000:04:00.0: reg 0x18: [mem 0x00000000-0x00000fff 64bit pref]
[ 10.317332] pci 0000:04:00.0: reg 0x20: [mem 0x00000000-0x00003fff 64bit pref]
[ 10.317440] pci 0000:04:00.0: supports D1 D2
[ 10.317442] pci 0000:04:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[ 10.317444] pci 0000:04:00.0: pme_poll = true
[ 10.317445] pci 0000:04:00.0: after device_set_wakeup_capable()
[ 10.317449] pci 0000:04:00.0: after pci_pme_active()
[ 10.329042] pci 0000:00:15.2: PCI bridge to [bus 04-ff]
[ 10.329053] pci 0000:00:15.2: bridge window [io 0x0000-0x0fff]
[ 10.329057] pci 0000:00:15.2: bridge window [mem 0x00000000-0x000fffff]
[ 10.329062] pci 0000:00:15.2: bridge window [mem 0x00000000-0x000fffff 64bit pref]
[ 10.329065] pci_bus 0000:04: busn_res: [bus 04-ff] end is updated to 04
[ 10.329575] ACPI: PCI: Interrupt link INTA configured for IRQ 0
[ 10.329669] ACPI: PCI: Interrupt link INTB configured for IRQ 0
[ 10.329759] ACPI: PCI: Interrupt link INTC configured for IRQ 0
[ 10.329849] ACPI: PCI: Interrupt link INTD configured for IRQ 0
[ 10.329940] ACPI: PCI: Interrupt link INTE configured for IRQ 0
[ 10.330029] ACPI: PCI: Interrupt link INTF configured for IRQ 0
[ 10.330119] ACPI: PCI: Interrupt link INTG configured for IRQ 0
[ 10.330210] ACPI: PCI: Interrupt link INTH configured for IRQ 0
[ 10.330444] iommu: Default domain type: Translated
[ 10.330446] iommu: DMA domain TLB invalidation policy: lazy mode
[ 10.330624] SCSI subsystem initialized
[ 10.330718] libata version 3.00 loaded.
[ 10.330752] ACPI: bus type USB registered
[ 10.330775] usbcore: registered new interface driver usbfs
[ 10.330786] usbcore: registered new interface driver hub
[ 10.330798] usbcore: registered new device driver usb
[ 10.331125] PCI: Using ACPI for IRQ routing
[ 10.332694] PCI: pci_cache_line_size set to 64 bytes
[ 10.332745] e820: reserve RAM buffer [mem 0x0009fc00-0x0009ffff]
[ 10.332748] e820: reserve RAM buffer [mem 0x5fe4d000-0x5fffffff]
[ 10.332750] e820: reserve RAM buffer [mem 0x17f000000-0x17fffffff]
[ 10.332794] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0
[ 10.332799] hpet0: 3 comparators, 32-bit 14.318180 MHz counter
[ 10.334067] clocksource: Switched to clocksource tsc-early
[ 10.350705] VFS: Disk quotas dquot_6.6.0
[ 10.350734] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[ 10.350847] pnp: PnP ACPI init
[ 10.351137] system 00:00: [mem 0xfec10002-0xfec11001] could not be reserved
[ 10.351446] pnp: PnP ACPI: found 2 devices
[ 10.358408] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
[ 10.358544] NET: Registered PF_INET protocol family
[ 10.358686] IP idents hash table entries: 65536 (order: 7, 524288 bytes, linear)
[ 10.360256] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear)
[ 10.360271] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
[ 10.360277] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear)
[ 10.360344] TCP bind hash table entries: 32768 (order: 8, 1048576 bytes, linear)
[ 10.360670] TCP: Hash tables configured (established 32768 bind 32768)
[ 10.360739] UDP hash table entries: 2048 (order: 4, 65536 bytes, linear)
[ 10.360763] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear)
[ 10.360870] NET: Registered PF_UNIX/PF_LOCAL protocol family
[ 10.360906] pci 0000:00:15.2: BAR 15: assigned [mem 0x80000000-0x800fffff 64bit pref]
[ 10.360912] pci 0000:00:15.2: BAR 13: assigned [io 0x2000-0x2fff]
[ 10.360917] pci 0000:00:14.4: PCI bridge to [bus 01]
[ 10.360928] pci 0000:00:15.0: PCI bridge to [bus 02]
[ 10.360936] pci 0000:00:15.1: PCI bridge to [bus 03]
[ 10.360939] pci 0000:00:15.1: bridge window [mem 0xf0000000-0xf00fffff]
[ 10.360947] pci 0000:04:00.0: BAR 4: assigned [mem 0x80000000-0x80003fff 64bit pref]
[ 10.360959] pci 0000:04:00.0: BAR 2: assigned [mem 0x80004000-0x80004fff 64bit pref]
[ 10.360971] pci 0000:04:00.0: BAR 0: assigned [io 0x2000-0x20ff]
[ 10.360976] pci 0000:00:15.2: PCI bridge to [bus 04]
[ 10.361116] pci 0000:00:15.2: bridge window [io 0x2000-0x2fff]
[ 10.361122] pci 0000:00:15.2: bridge window [mem 0x80000000-0x800fffff 64bit pref]
[ 10.361128] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]
[ 10.361130] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]
[ 10.361132] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000dffff]
[ 10.361133] pci_bus 0000:00: resource 7 [mem 0x80000000-0xffffffff]
[ 10.361135] pci_bus 0000:01: resource 4 [io 0x0000-0x0cf7 window]
[ 10.361137] pci_bus 0000:01: resource 5 [io 0x0d00-0xffff window]
[ 10.361138] pci_bus 0000:01: resource 6 [mem 0x000a0000-0x000dffff]
[ 10.361140] pci_bus 0000:01: resource 7 [mem 0x80000000-0xffffffff]
[ 10.361141] pci_bus 0000:03: resource 1 [mem 0xf0000000-0xf00fffff]
[ 10.361143] pci_bus 0000:04: resource 0 [io 0x2000-0x2fff]
[ 10.361144] pci_bus 0000:04: resource 2 [mem 0x80000000-0x800fffff 64bit pref]
[ 10.361239] pci 0000:00:01.1: D0 power state depends on 0000:00:01.0
[ 10.361456] pci 0000:00:12.0: AMD USB device
[ 10.361478] pci 0000:00:12.0: AMD USB ohci handoff
[ 10.361761] pci 0000:00:12.2: AMD USB device
[ 10.361775] pci 0000:00:12.2: AMD USB ehci handoff
[ 10.361903] pci 0000:00:12.2: PME# does not work under D3, disabling it
[ 10.362049] pci 0000:00:13.0: AMD USB device
[ 10.362062] pci 0000:00:13.0: AMD USB ohci handoff
[ 10.362328] pci 0000:00:13.2: AMD USB device
[ 10.362339] pci 0000:00:13.2: AMD USB ehci handoff
[ 10.362469] pci 0000:00:13.2: PME# does not work under D3, disabling it
[ 10.362622] pci 0000:00:14.5: AMD USB device
[ 10.362634] pci 0000:00:14.5: AMD USB ohci handoff
[ 10.362915] pci 0000:00:16.0: AMD USB device
[ 10.362928] pci 0000:00:16.0: AMD USB ohci handoff
[ 10.363195] pci 0000:00:16.2: AMD USB device
[ 10.363206] pci 0000:00:16.2: AMD USB ehci handoff
[ 10.363334] pci 0000:00:16.2: PME# does not work under D3, disabling it
[ 10.363561] pci 0000:03:00.0: AMD USB xhci handoff
[ 10.363610] PCI: CLS 64 bytes, default 64
[ 10.363723] pci 0000:00:00.2: AMD-Vi: Applying erratum 746 workaround
[ 10.363810] pci 0000:00:01.0: Adding to iommu group 0
[ 10.363826] pci 0000:00:01.1: Adding to iommu group 0
[ 10.363851] pci 0000:00:11.0: Adding to iommu group 1
[ 10.363886] pci 0000:00:12.0: Adding to iommu group 2
[ 10.363903] pci 0000:00:12.2: Adding to iommu group 2
[ 10.363937] pci 0000:00:13.0: Adding to iommu group 3
[ 10.363953] pci 0000:00:13.2: Adding to iommu group 3
[ 10.363991] pci 0000:00:14.0: Adding to iommu group 4
[ 10.364009] pci 0000:00:14.2: Adding to iommu group 4
[ 10.364025] pci 0000:00:14.3: Adding to iommu group 4
[ 10.364048] pci 0000:00:14.4: Adding to iommu group 5
[ 10.364070] pci 0000:00:14.5: Adding to iommu group 6
[ 10.364104] pci 0000:00:15.0: Adding to iommu group 7
[ 10.364123] pci 0000:00:15.1: Adding to iommu group 7
[ 10.364139] pci 0000:00:15.2: Adding to iommu group 7
[ 10.364178] pci 0000:00:16.0: Adding to iommu group 8
[ 10.364194] pci 0000:00:16.2: Adding to iommu group 8
[ 10.364246] pci 0000:00:18.0: Adding to iommu group 9
[ 10.364266] pci 0000:00:18.1: Adding to iommu group 9
[ 10.364283] pci 0000:00:18.2: Adding to iommu group 9
[ 10.364303] pci 0000:00:18.3: Adding to iommu group 9
[ 10.364321] pci 0000:00:18.4: Adding to iommu group 9
[ 10.364340] pci 0000:00:18.5: Adding to iommu group 9
[ 10.364352] pci 0000:03:00.0: Adding to iommu group 7
[ 10.364360] pci 0000:04:00.0: Adding to iommu group 7
[ 10.366500] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[ 10.366505] AMD-Vi: Extended features (0x800000853, 0x0): PreF PPR GT IA
[ 10.366510] AMD-Vi: Interrupt remapping enabled
[ 10.366691] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[ 10.366693] software IO TLB: mapped [mem 0x000000005be4d000-0x000000005fe4d000] (64MB)
[ 10.366742] LVT offset 0 assigned for vector 0x400
[ 10.366763] perf: AMD IBS detected (0x000000ff)
[ 10.366771] amd_uncore: 4 amd_nb counters detected
[ 10.367574] workingset: timestamp_bits=37 max_order=20 bucket_order=0
[ 10.367603] zbud: loaded
[ 10.368066] NET: Registered PF_ALG protocol family
[ 10.368071] Key type asymmetric registered
[ 10.368073] Asymmetric key parser 'x509' registered
[ 10.368346] alg: self-tests disabled
[ 10.368439] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 251)
[ 10.368477] io scheduler mq-deadline registered
[ 10.368479] io scheduler kyber registered
[ 10.370011] pcieport 0000:00:15.0: PME: Signaling with IRQ 25
[ 10.370174] pcieport 0000:00:15.1: PME: Signaling with IRQ 26
[ 10.370247] pcieport 0000:00:15.2: enabling device (0000 -> 0003)
[ 10.370449] pcieport 0000:00:15.2: PME: Signaling with IRQ 27
[ 10.370706] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
[ 10.370767] ACPI: button: Power Button [PWRF]
[ 10.370823] ACPI: \_SB_.P000: Found 2 idle states
[ 10.370937] ACPI: \_SB_.P001: Found 2 idle states
[ 10.371827] thermal LNXTHERM:00: registered as thermal_zone0
[ 10.371830] ACPI: thermal: Thermal Zone [TZ00] (0 C)
[ 10.372146] Non-volatile memory driver v1.3
[ 10.372218] AMD-Vi: AMD IOMMUv2 loaded and initialized
[ 10.372338] ahci 0000:00:11.0: version 3.0
[ 10.372612] ahci 0000:00:11.0: AHCI 0001.0300 32 slots 8 ports 6 Gbps 0x40 impl SATA mode
[ 10.372616] ahci 0000:00:11.0: flags: 64bit ncq sntf ilck led clo pio
[ 10.374133] scsi host0: ahci
[ 10.374333] scsi host1: ahci
[ 10.374510] scsi host2: ahci
[ 10.374710] scsi host3: ahci
[ 10.374879] scsi host4: ahci
[ 10.375057] scsi host5: ahci
[ 10.375241] scsi host6: ahci
[ 10.375421] scsi host7: ahci
[ 10.375508] ata port1: DUMMY
[ 10.375510] ata port2: DUMMY
[ 10.375511] ata port3: DUMMY
[ 10.375512] ata port4: DUMMY
[ 10.375514] ata port5: DUMMY
[ 10.375515] ata port6: DUMMY
[ 10.375517] ata port7: SATA max UDMA/133 abar m2048@0xf01cc000 port 0xf01cc400 irq 19
[ 10.375519] ata port8: DUMMY
[ 10.375597] ACPI: bus type drm_connector registered
[ 10.375823] i8042: PNP: No PS/2 controller found.
[ 10.375824] i8042: Probing ports directly.
[ 10.378675] serio: i8042 KBD port at 0x60,0x64 irq 1
[ 10.378751] serio: i8042 AUX port at 0x60,0x64 irq 12
[ 10.378874] mousedev: PS/2 mouse device common for all mice
[ 10.378927] rtc_cmos 00:01: RTC can wake from S4
[ 10.379173] rtc_cmos 00:01: registered as rtc0
[ 10.379197] rtc_cmos 00:01: setting system clock to 2023-04-17T07:39:50 UTC (1681717190)
[ 10.379234] rtc_cmos 00:01: alarms up to one day, y3k, 114 bytes nvram, hpet irqs
[ 10.379268] device-mapper: uevent: version 1.0.3
[ 10.379337] device-mapper: ioctl: 4.47.0-ioctl (2022-07-28) initialised: dm-devel@redhat.com
[ 10.379496] hid: raw HID events driver (C) Jiri Kosina
[ 10.379531] usbcore: registered new interface driver usbhid
[ 10.379532] usbhid: USB HID core driver
[ 10.379638] Initializing XFRM netlink socket
[ 10.379648] NET: Registered PF_PACKET protocol family
[ 10.379650] x86/pm: family 0x15 cpu detected, MSR saving is needed during suspending.
[ 10.379817] microcode: CPU0: patch_level=0x0600111f
[ 10.379827] microcode: Microcode Update Driver: v2.2.
[ 10.379831] IPI shorthand broadcast: enabled
[ 10.379838] AVX version of gcm_enc/dec engaged.
[ 10.379854] AES CTR mode by8 optimization enabled
[ 10.380695] ata link7: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[ 10.380969] ata dev7.0: ATA-9: SanDisk SDSSDP064G, 2.0.0, max UDMA/133
[ 10.380972] ata dev7.0: 125045424 sectors, multi 1: LBA48 NCQ (depth 32)
[ 10.381178] ata dev7.0: configured for UDMA/133
[ 10.381289] scsi 6:0:0:0: Direct-Access ATA SanDisk SDSSDP06 0 PQ: 0 ANSI: 5
[ 10.381718] sd 6:0:0:0: [sda] 125045424 512-byte logical blocks: (64.0 GB/59.6 GiB)
[ 10.381730] sd 6:0:0:0: [sda] Write Protect is off
[ 10.381733] sd 6:0:0:0: [sda] Mode Sense: 00 3a 00 00
[ 10.381749] sd 6:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 10.381775] sd 6:0:0:0: [sda] Preferred minimum I/O size 512 bytes
[ 10.382547] sda: sda1 sda2 sda3
[ 10.382736] sd 6:0:0:0: [sda] Attached SCSI disk
[ 10.385344] sched_clock: Marking stable (10268007974, 116977866)->(10387423156, -2437316)
[ 10.385519] registered taskstats version 1
[ 10.385734] zswap: loaded using pool lzo/zbud
[ 10.390028] kmemleak: Kernel memory leak detector initialized (mem pool available: 15677)
[ 10.390033] debug_vm_pgtable: [debug_vm_pgtable ]: Validating architecture page table helpers
[ 10.393894] kmemleak: Automatic memory scanning thread started
[ 10.394488] Key type encrypted registered
[ 10.397453] PM: Magic number: 3:139:673
[ 10.397466] workqueue scsi_tmf_3: hash matches
[ 10.410334] EXT4-fs (sda3): mounted filesystem fe29e0dc-6303-4401-987c-8472bc1b9516 with ordered data mode. Quota mode: none.
[ 10.410378] VFS: Mounted root (ext4 filesystem) on device 8:3.
[ 10.412268] devtmpfs: mounted
[ 10.412286] After kernel_init_freeable
[ 10.416793] Freeing unused kernel image (initmem) memory: 2908K
[ 10.421291] Write protecting the kernel read-only data: 20480k
[ 10.421559] Freeing unused kernel image (rodata/data gap) memory: 836K
[ 10.458614] x86/mm: Checked W+X mappings: passed, no W+X pages found.
[ 10.458620] rodata_test: all tests were successful
[ 10.458621] After mark_readonly
[ 10.458621] After pti_finalize
[ 10.458637] rcu_end_inkernel_boot
[ 10.458644] Run /sbin/init as init process
[ 10.458646] with arguments:
[ 10.458648] /sbin/init
[ 10.458649] noisapnp
[ 10.458650] with environment:
[ 10.458650] HOME=/
[ 10.458651] TERM=linux
[ 10.458652] BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc6-00311-gde8224969f66
[ 10.637190] systemd[1]: Inserted module 'autofs4'
[ 10.663625] NET: Registered PF_INET6 protocol family
[ 10.664473] Segment Routing with IPv6
[ 10.664500] In-situ OAM (IOAM) with IPv6
[ 10.691376] systemd[1]: systemd 252.6-1 running in system mode (+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
[ 10.691387] systemd[1]: Detected architecture x86-64.
[ 10.696468] systemd[1]: Hostname set to <kodi>.
[ 11.000700] systemd[1]: Queued start job for default target graphical.target.
[ 11.011651] systemd[1]: Created slice system-getty.slice - Slice /system/getty.
[ 11.012747] systemd[1]: Created slice system-modprobe.slice - Slice /system/modprobe.
[ 11.013600] systemd[1]: Created slice user.slice - User and Session Slice.
[ 11.013784] systemd[1]: Started systemd-ask-password-console.path - Dispatch Password Requests to Console Directory Watch.
[ 11.013902] systemd[1]: Started systemd-ask-password-wall.path - Forward Password Requests to Wall Directory Watch.
[ 11.014324] systemd[1]: Set up automount proc-sys-fs-binfmt_misc.automount - Arbitrary Executable File Formats File System Automount Point.
[ 11.014366] systemd[1]: Reached target cryptsetup.target - Local Encrypted Volumes.
[ 11.014416] systemd[1]: Reached target integritysetup.target - Local Integrity Protected Volumes.
[ 11.014455] systemd[1]: Reached target paths.target - Path Units.
[ 11.014485] systemd[1]: Reached target remote-fs.target - Remote File Systems.
[ 11.014515] systemd[1]: Reached target slices.target - Slice Units.
[ 11.014554] systemd[1]: Reached target swap.target - Swaps.
[ 11.014592] systemd[1]: Reached target veritysetup.target - Local Verity Protected Volumes.
[ 11.017065] systemd[1]: Listening on systemd-coredump.socket - Process Core Dump Socket.
[ 11.017317] systemd[1]: Listening on systemd-fsckd.socket - fsck to fsckd communication Socket.
[ 11.017479] systemd[1]: Listening on systemd-initctl.socket - initctl Compatibility Named Pipe.
[ 11.017786] systemd[1]: Listening on systemd-journald-audit.socket - Journal Audit Socket.
[ 11.018059] systemd[1]: Listening on systemd-journald-dev-log.socket - Journal Socket (/dev/log).
[ 11.018337] systemd[1]: Listening on systemd-journald.socket - Journal Socket.
[ 11.018594] systemd[1]: Listening on systemd-networkd.socket - Network Service Netlink Socket.
[ 11.019420] systemd[1]: Listening on systemd-udevd-control.socket - udev Control Socket.
[ 11.019693] systemd[1]: Listening on systemd-udevd-kernel.socket - udev Kernel Socket.
[ 11.022504] systemd[1]: Mounting dev-hugepages.mount - Huge Pages File System...
[ 11.025189] systemd[1]: Mounting dev-mqueue.mount - POSIX Message Queue File System...
[ 11.029590] systemd[1]: Mounting sys-kernel-debug.mount - Kernel Debug File System...
[ 11.046849] systemd[1]: Mounting sys-kernel-tracing.mount - Kernel Trace File System...
[ 11.053730] systemd[1]: Starting kmod-static-nodes.service - Create List of Static Device Nodes...
[ 11.066015] systemd[1]: Starting modprobe@configfs.service - Load Kernel Module configfs...
[ 11.069414] systemd[1]: Starting modprobe@dm_mod.service - Load Kernel Module dm_mod...
[ 11.076547] systemd[1]: Starting modprobe@drm.service - Load Kernel Module drm...
[ 11.090852] systemd[1]: Starting modprobe@efi_pstore.service - Load Kernel Module efi_pstore...
[ 11.097815] systemd[1]: Starting modprobe@fuse.service - Load Kernel Module fuse...
[ 11.109388] systemd[1]: Starting modprobe@loop.service - Load Kernel Module loop...
[ 11.109456] systemd[1]: systemd-firstboot.service - First Boot Wizard was skipped because of an unmet condition check (ConditionFirstBoot=yes).
[ 11.109525] systemd[1]: systemd-fsck-root.service - File System Check on Root Device was skipped because of an unmet condition check (ConditionPathIsReadWrite=!/).
[ 11.109562] systemd[1]: Reached target local-fs.target - Local File Systems.
[ 11.109629] systemd[1]: apparmor.service - Load AppArmor profiles was skipped because of an unmet condition check (ConditionSecurity=apparmor).
[ 11.120810] loop: module loaded
[ 11.122003] systemd[1]: Starting systemd-binfmt.service - Set Up Additional Binary Formats...
[ 11.129598] fuse: init (API version 7.38)
[ 11.137426] systemd[1]: Starting systemd-journald.service - Journal Service...
[ 11.145752] systemd[1]: Starting systemd-random-seed.service - Load/Save Random Seed...
[ 11.157494] systemd[1]: Starting systemd-sysctl.service - Apply Kernel Variables...
[ 11.172634] systemd[1]: Starting systemd-sysusers.service - Create System Users...
[ 11.197939] systemd[1]: Starting systemd-udev-trigger.service - Coldplug All udev Devices...
[ 11.216247] systemd[1]: Mounted dev-hugepages.mount - Huge Pages File System.
[ 11.216451] systemd[1]: Mounted dev-mqueue.mount - POSIX Message Queue File System.
[ 11.216611] systemd[1]: Mounted sys-kernel-debug.mount - Kernel Debug File System.
[ 11.216772] systemd[1]: Mounted sys-kernel-tracing.mount - Kernel Trace File System.
[ 11.233481] systemd[1]: Finished kmod-static-nodes.service - Create List of Static Device Nodes.
[ 11.234279] systemd[1]: modprobe@configfs.service: Deactivated successfully.
[ 11.238519] systemd[1]: Finished modprobe@configfs.service - Load Kernel Module configfs.
[ 11.239177] systemd[1]: modprobe@dm_mod.service: Deactivated successfully.
[ 11.247227] systemd[1]: Finished modprobe@dm_mod.service - Load Kernel Module dm_mod.
[ 11.247980] systemd[1]: modprobe@drm.service: Deactivated successfully.
[ 11.253523] systemd[1]: Finished modprobe@drm.service - Load Kernel Module drm.
[ 11.254233] systemd[1]: modprobe@efi_pstore.service: Deactivated successfully.
[ 11.255540] systemd[1]: Finished modprobe@efi_pstore.service - Load Kernel Module efi_pstore.
[ 11.256109] systemd[1]: modprobe@fuse.service: Deactivated successfully.
[ 11.261747] systemd[1]: Finished modprobe@fuse.service - Load Kernel Module fuse.
[ 11.262375] systemd[1]: modprobe@loop.service: Deactivated successfully.
[ 11.265991] systemd[1]: Finished modprobe@loop.service - Load Kernel Module loop.
[ 11.267213] systemd[1]: Finished systemd-sysctl.service - Apply Kernel Variables.
[ 11.268186] systemd[1]: Finished systemd-sysusers.service - Create System Users.
[ 11.268690] systemd[1]: proc-sys-fs-binfmt_misc.automount: Got automount request for /proc/sys/fs/binfmt_misc, triggered by 133 (systemd-binfmt)
[ 11.282799] systemd[1]: Mounting proc-sys-fs-binfmt_misc.mount - Arbitrary Executable File Formats File System...
[ 11.309441] systemd[1]: Mounting sys-fs-fuse-connections.mount - FUSE Control File System...
[ 11.358583] systemd[1]: Mounting sys-kernel-config.mount - Kernel Configuration File System...
[ 11.358687] systemd[1]: systemd-pstore.service - Platform Persistent Storage Archival was skipped because of an unmet condition check (ConditionDirectoryNotEmpty=/sys/fs/pstore).
[ 11.358816] systemd[1]: systemd-repart.service - Repartition Root Disk was skipped because no trigger condition checks were met.
[ 11.377438] systemd[1]: Starting systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev...
[ 11.378820] systemd[1]: Mounted proc-sys-fs-binfmt_misc.mount - Arbitrary Executable File Formats File System.
[ 11.385019] tsc: Refined TSC clocksource calibration: 3900.223 MHz
[ 11.385026] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x70705a6472c, max_idle_ns: 881590586812 ns
[ 11.385038] clocksource: Switched to clocksource tsc
[ 11.397465] systemd[1]: Finished systemd-binfmt.service - Set Up Additional Binary Formats.
[ 11.397822] systemd[1]: Mounted sys-fs-fuse-connections.mount - FUSE Control File System.
[ 11.397985] systemd[1]: Mounted sys-kernel-config.mount - Kernel Configuration File System.
[ 11.448710] systemd[1]: Finished systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev.
[ 11.457254] systemd[1]: Starting systemd-udevd.service - Rule-based Manager for Device Events and Files...
[ 11.509082] systemd[1]: Started systemd-journald.service - Journal Service.
[ 11.569863] systemd-journald[134]: Received client request to flush runtime journal.
[ 12.053221] sd 6:0:0:0: Attached scsi generic sg0 type 0
[ 12.140997] random: crng init done
[ 12.367954] acpi_cpufreq: overriding BIOS provided _PSD data
[ 12.510747] QUIRK: Enable AMD PLL fix
[ 12.510804] ehci-pci 0000:00:12.2: EHCI Host Controller
[ 12.510833] ehci-pci 0000:00:12.2: new USB bus registered, assigned bus number 1
[ 12.510844] ehci-pci 0000:00:12.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
[ 12.510853] ehci-pci 0000:00:12.2: debug port 1
[ 12.511023] ehci-pci 0000:00:12.2: irq 17, io mem 0xf01cd000
[ 12.515767] piix4_smbus 0000:00:14.0: SMBus Host Controller at 0xb00, revision 0
[ 12.515773] piix4_smbus 0000:00:14.0: Using register 0x2e for SMBus port selection
[ 12.516241] piix4_smbus 0000:00:14.0: Auxiliary SMBus Host Controller at 0xb20
[ 12.525014] ehci-pci 0000:00:12.2: USB 2.0 started, EHCI 1.00
[ 12.525397] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 12.525400] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.525402] usb usb1: Product: EHCI Host Controller
[ 12.525404] usb usb1: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ehci_hcd
[ 12.525405] usb usb1: SerialNumber: 0000:00:12.2
[ 12.525862] hub 1-0:1.0: USB hub found
[ 12.525889] hub 1-0:1.0: 5 ports detected
[ 12.526592] ehci-pci 0000:00:13.2: EHCI Host Controller
[ 12.526612] ehci-pci 0000:00:13.2: new USB bus registered, assigned bus number 2
[ 12.526623] ehci-pci 0000:00:13.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
[ 12.526632] ehci-pci 0000:00:13.2: debug port 1
[ 12.526767] ehci-pci 0000:00:13.2: irq 17, io mem 0xf01ce000
[ 12.541014] ehci-pci 0000:00:13.2: USB 2.0 started, EHCI 1.00
[ 12.541263] usb usb2: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 12.541266] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.541268] usb usb2: Product: EHCI Host Controller
[ 12.541270] usb usb2: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ehci_hcd
[ 12.541271] usb usb2: SerialNumber: 0000:00:13.2
[ 12.541718] hub 2-0:1.0: USB hub found
[ 12.541746] hub 2-0:1.0: 5 ports detected
[ 12.542418] ehci-pci 0000:00:16.2: EHCI Host Controller
[ 12.542436] ehci-pci 0000:00:16.2: new USB bus registered, assigned bus number 3
[ 12.542447] ehci-pci 0000:00:16.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
[ 12.542456] ehci-pci 0000:00:16.2: debug port 1
[ 12.542584] ehci-pci 0000:00:16.2: irq 17, io mem 0xf01cf000
[ 12.557023] ehci-pci 0000:00:16.2: USB 2.0 started, EHCI 1.00
[ 12.557405] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 12.557408] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.557410] usb usb3: Product: EHCI Host Controller
[ 12.557411] usb usb3: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ehci_hcd
[ 12.557413] usb usb3: SerialNumber: 0000:00:16.2
[ 12.557858] hub 3-0:1.0: USB hub found
[ 12.557884] hub 3-0:1.0: 4 ports detected
[ 12.558447] ohci-pci 0000:00:12.0: OHCI PCI host controller
[ 12.558466] ohci-pci 0000:00:12.0: new USB bus registered, assigned bus number 4
[ 12.558608] ohci-pci 0000:00:12.0: irq 18, io mem 0xf01c8000
[ 12.558618] ohci-pci 0000:00:13.0: OHCI PCI host controller
[ 12.558635] ohci-pci 0000:00:13.0: new USB bus registered, assigned bus number 5
[ 12.558703] ohci-pci 0000:00:13.0: irq 18, io mem 0xf01c9000
[ 12.558710] ohci-pci 0000:00:14.5: OHCI PCI host controller
[ 12.558725] ohci-pci 0000:00:14.5: new USB bus registered, assigned bus number 6
[ 12.558793] ohci-pci 0000:00:14.5: irq 18, io mem 0xf01ca000
[ 12.558800] ohci-pci 0000:00:16.0: OHCI PCI host controller
[ 12.558814] ohci-pci 0000:00:16.0: new USB bus registered, assigned bus number 7
[ 12.558888] ohci-pci 0000:00:16.0: irq 18, io mem 0xf01cb000
[ 12.635709] usb usb7: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 12.635716] usb usb7: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.635718] usb usb7: Product: OHCI PCI host controller
[ 12.635720] usb usb7: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ohci_hcd
[ 12.635721] usb usb7: SerialNumber: 0000:00:16.0
[ 12.636145] hub 7-0:1.0: USB hub found
[ 12.636172] hub 7-0:1.0: 4 ports detected
[ 12.636849] usb usb5: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 12.636851] usb usb5: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.636853] usb usb5: Product: OHCI PCI host controller
[ 12.636854] usb usb5: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ohci_hcd
[ 12.636856] usb usb5: SerialNumber: 0000:00:13.0
[ 12.639549] hub 5-0:1.0: USB hub found
[ 12.639579] hub 5-0:1.0: 5 ports detected
[ 12.640465] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 12.640468] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.640470] usb usb4: Product: OHCI PCI host controller
[ 12.640471] usb usb4: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ohci_hcd
[ 12.640473] usb usb4: SerialNumber: 0000:00:12.0
[ 12.640848] hub 4-0:1.0: USB hub found
[ 12.640874] hub 4-0:1.0: 5 ports detected
[ 12.644079] usb usb6: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 12.644084] usb usb6: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.644086] usb usb6: Product: OHCI PCI host controller
[ 12.644088] usb usb6: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ohci_hcd
[ 12.644089] usb usb6: SerialNumber: 0000:00:14.5
[ 12.644484] hub 6-0:1.0: USB hub found
[ 12.644509] hub 6-0:1.0: 2 ports detected
[ 12.658738] r8169 0000:04:00.0: enabling device (0000 -> 0003)
[ 12.697540] xhci_hcd 0000:03:00.0: xHCI Host Controller
[ 12.697566] xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 8
[ 12.709878] r8169 0000:04:00.0 eth0: RTL8168f/8111f, 08:60:6e:74:7a:51, XID 480, IRQ 28
[ 12.709886] r8169 0000:04:00.0 eth0: jumbo features [frames: 9194 bytes, tx checksumming: ko]
[ 12.735756] 1
[ 12.735806] 2
[ 12.736276] snd_hda_intel 0000:00:01.1: Force to non-snoop mode
[ 12.736282] 3
[ 12.736283] 4
[ 12.736284] 5
[ 12.736284] 7
[ 12.736288] 8
[ 12.736288] 9
[ 12.740797] 1
[ 12.740846] 2
[ 12.741376] 3
[ 12.741377] 4
[ 12.741378] 5
[ 12.741379] 7
[ 12.741382] 8
[ 12.741383] 9
[ 12.769260] input: HDA ATI HDMI HDMI/DP,pcm=3 as /devices/pci0000:00/0000:00:01.1/sound/card0/input1
[ 12.769555] input: HDA ATI HDMI HDMI/DP,pcm=7 as /devices/pci0000:00/0000:00:01.1/sound/card0/input2
[ 12.784439] snd_hda_codec_realtek hdaudioC1D0: ALC892: SKU not ready 0x00000100
[ 12.785682] snd_hda_codec_realtek hdaudioC1D0: autoconfig for ALC892: line_outs=4 (0x14/0x16/0x15/0x17/0x0) type:line
[ 12.785688] snd_hda_codec_realtek hdaudioC1D0: speaker_outs=0 (0x0/0x0/0x0/0x0/0x0)
[ 12.785691] snd_hda_codec_realtek hdaudioC1D0: hp_outs=1 (0x1b/0x0/0x0/0x0/0x0)
[ 12.785693] snd_hda_codec_realtek hdaudioC1D0: mono: mono_out=0x0
[ 12.785694] snd_hda_codec_realtek hdaudioC1D0: dig-out=0x1e/0x0
[ 12.785696] snd_hda_codec_realtek hdaudioC1D0: inputs:
[ 12.785697] snd_hda_codec_realtek hdaudioC1D0: Rear Mic=0x18
[ 12.785699] snd_hda_codec_realtek hdaudioC1D0: Front Mic=0x19
[ 12.785700] snd_hda_codec_realtek hdaudioC1D0: Line=0x1a
[ 12.785701] snd_hda_codec_realtek hdaudioC1D0: CD=0x1c
[ 12.807528] xhci_hcd 0000:03:00.0: hcc params 0x0200f180 hci version 0x96 quirks 0x0000000000080010
[ 12.813178] input: HD-Audio Generic Rear Mic as /devices/pci0000:00/0000:00:14.2/sound/card1/input3
[ 12.813514] input: HD-Audio Generic Front Mic as /devices/pci0000:00/0000:00:14.2/sound/card1/input4
[ 12.813782] input: HD-Audio Generic Line as /devices/pci0000:00/0000:00:14.2/sound/card1/input5
[ 12.814048] input: HD-Audio Generic Line Out Front as /devices/pci0000:00/0000:00:14.2/sound/card1/input6
[ 12.814317] input: HD-Audio Generic Line Out Surround as /devices/pci0000:00/0000:00:14.2/sound/card1/input7
[ 12.814575] input: HD-Audio Generic Line Out CLFE as /devices/pci0000:00/0000:00:14.2/sound/card1/input8
[ 12.814833] input: HD-Audio Generic Line Out Side as /devices/pci0000:00/0000:00:14.2/sound/card1/input9
[ 12.815087] input: HD-Audio Generic Front Headphone as /devices/pci0000:00/0000:00:14.2/sound/card1/input10
[ 12.816233] xhci_hcd 0000:03:00.0: xHCI Host Controller
[ 12.816249] xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 9
[ 12.816260] xhci_hcd 0000:03:00.0: Host supports USB 3.0 SuperSpeed
[ 12.830985] usb usb8: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 12.830993] usb usb8: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.830995] usb usb8: Product: xHCI Host Controller
[ 12.830996] usb usb8: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 xhci-hcd
[ 12.830998] usb usb8: SerialNumber: 0000:03:00.0
[ 12.835543] hub 8-0:1.0: USB hub found
[ 12.842296] hub 8-0:1.0: 2 ports detected
[ 12.850842] usb usb9: We don't know the algorithms for LPM for this host, disabling LPM.
[ 12.850983] usb usb9: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 6.03
[ 12.850986] usb usb9: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.850988] usb usb9: Product: xHCI Host Controller
[ 12.850989] usb usb9: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 xhci-hcd
[ 12.850991] usb usb9: SerialNumber: 0000:03:00.0
[ 12.856278] r8169 0000:04:00.0 enp4s0: renamed from eth0
[ 12.862771] hub 9-0:1.0: USB hub found
[ 12.873601] hub 9-0:1.0: 2 ports detected
[ 13.025026] usb 4-1: new low-speed USB device number 2 using ohci-pci
[ 13.103377] r8169 0000:04:00.0: Direct firmware load for rtl_nic/rtl8168f-1.fw failed with error -2
[ 13.103389] r8169 0000:04:00.0: Unable to load firmware rtl_nic/rtl8168f-1.fw (-2)
[ 13.103907] RTL8211E Gigabit Ethernet r8169-0-400:00: attached PHY driver (mii_bus:phy_addr=r8169-0-400:00, irq=MAC)
[ 13.175505] r8169 0000:04:00.0 enp4s0: Link is Down
[ 13.179870] [drm] radeon kernel modesetting enabled.
[ 13.181586] [drm] initializing kernel modesetting (ARUBA 0x1002:0x9996 0x1002:0x9996 0x00).
[ 13.181652] ATOM BIOS: 113
[ 13.181757] radeon 0000:00:01.0: VRAM: 512M 0x0000000000000000 - 0x000000001FFFFFFF (512M used)
[ 13.181761] radeon 0000:00:01.0: GTT: 1024M 0x0000000020000000 - 0x000000005FFFFFFF
[ 13.181769] [drm] Detected VRAM RAM=512M, BAR=256M
[ 13.181770] [drm] RAM width 64bits DDR
[ 13.181947] [drm] radeon: 512M of VRAM memory ready
[ 13.181952] [drm] radeon: 1024M of GTT memory ready.
[ 13.181994] [drm] Loading ARUBA Microcode
[ 13.190350] [drm] Internal thermal controller without fan control
[ 13.191100] [drm] radeon: dpm initialized
[ 13.196015] [drm] Found VCE firmware/feedback version 50.0.1 / 17!
[ 13.196073] [drm] GART: num cpu pages 262144, num gpu pages 262144
[ 13.234337] [drm] GART: Restore entries: num cpu pages 262144, num gpu pages 262144
[ 13.237934] [drm] GART: Done restoring entries
[ 13.237938] [drm] PCIE GART of 1024M enabled (table at 0x00000000001D6000).
[ 13.238177] radeon 0000:00:01.0: WB enabled
[ 13.238180] radeon 0000:00:01.0: fence driver on ring 0 use gpu addr 0x0000000020000c00
[ 13.238558] radeon 0000:00:01.0: fence driver on ring 5 use gpu addr 0x0000000000075a18
[ 13.246045] usb 4-1: New USB device found, idVendor=413c, idProduct=2106, bcdDevice= 1.01
[ 13.246050] usb 4-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 13.246052] usb 4-1: Product: Dell QuietKey Keyboard
[ 13.246054] usb 4-1: Manufacturer: DELL
[ 13.253819] input: DELL Dell QuietKey Keyboard as /devices/pci0000:00/0000:00:12.0/usb4/4-1/4-1:1.0/0003:413C:2106.0001/input/input11
[ 13.260936] radeon 0000:00:01.0: fence driver on ring 6 use gpu addr 0x0000000020000c18
[ 13.260940] radeon 0000:00:01.0: fence driver on ring 7 use gpu addr 0x0000000020000c1c
[ 13.260942] radeon 0000:00:01.0: fence driver on ring 1 use gpu addr 0x0000000020000c04
[ 13.260944] radeon 0000:00:01.0: fence driver on ring 2 use gpu addr 0x0000000020000c08
[ 13.260945] radeon 0000:00:01.0: fence driver on ring 3 use gpu addr 0x0000000020000c0c
[ 13.260947] radeon 0000:00:01.0: fence driver on ring 4 use gpu addr 0x0000000020000c10
[ 13.262964] radeon 0000:00:01.0: radeon: MSI limited to 32-bit
[ 13.263151] radeon 0000:00:01.0: radeon: using MSI.
[ 13.263222] [drm] radeon: irq initialized.
[ 13.281648] [drm] ring test on 0 succeeded in 3 usecs
[ 13.281658] [drm] ring test on 3 succeeded in 4 usecs
[ 13.281665] [drm] ring test on 4 succeeded in 4 usecs
[ 13.295659] [drm] ring test on 5 succeeded in 2 usecs
[ 13.297656] [drm] UVD initialized successfully.
[ 13.313663] hid-generic 0003:413C:2106.0001: input,hidraw0: USB HID v1.10 Keyboard [DELL Dell QuietKey Keyboard] on usb-0000:00:12.0-1/input0
[ 13.446953] [drm] ring test on 6 succeeded in 18 usecs
[ 13.446967] [drm] ring test on 7 succeeded in 3 usecs
[ 13.446968] [drm] VCE initialized successfully.
[ 13.447122] snd_hda_intel 0000:00:01.1: bound 0000:00:01.0 (ops radeon_audio_component_bind_ops [radeon])
[ 13.447293] [drm] ib test on ring 0 succeeded in 0 usecs
[ 13.447346] [drm] ib test on ring 3 succeeded in 0 usecs
[ 13.447396] [drm] ib test on ring 4 succeeded in 0 usecs
[ 13.465099] [drm] ib test on ring 5 succeeded
[ 13.481132] [drm] ib test on ring 6 succeeded in 1 usecs
[ 13.497085] [drm] ib test on ring 7 succeeded in 1 usecs
[ 13.500056] [drm] Radeon Display Connectors
[ 13.500060] [drm] Connector 0:
[ 13.500061] [drm] DP-1
[ 13.500062] [drm] HPD1
[ 13.500062] [drm] DDC: 0x6530 0x6530 0x6534 0x6534 0x6538 0x6538 0x653c 0x653c
[ 13.500065] [drm] Encoders:
[ 13.500065] [drm] DFP1: INTERNAL_UNIPHY2
[ 13.500066] [drm] Connector 1:
[ 13.500067] [drm] VGA-1
[ 13.500068] [drm] HPD2
[ 13.500069] [drm] DDC: 0x6540 0x6540 0x6544 0x6544 0x6548 0x6548 0x654c 0x654c
[ 13.500070] [drm] Encoders:
[ 13.500071] [drm] CRT1: INTERNAL_UNIPHY2
[ 13.500072] [drm] CRT1: NUTMEG
[ 13.500072] [drm] Connector 2:
[ 13.500073] [drm] HDMI-A-1
[ 13.500074] [drm] HPD3
[ 13.500075] [drm] DDC: 0x6550 0x6550 0x6554 0x6554 0x6558 0x6558 0x655c 0x655c
[ 13.500076] [drm] Encoders:
[ 13.500077] [drm] DFP2: INTERNAL_UNIPHY
[ 13.741081] usb 4-2: new low-speed USB device number 3 using ohci-pci
[ 13.779818] [drm] fb mappable at 0xE03E9000
[ 13.779826] [drm] vram apper at 0xE0000000
[ 13.779828] [drm] size 5242880
[ 13.779830] [drm] fb depth is 24
[ 13.779832] [drm] pitch is 5120
[ 13.780398] fbcon: radeondrmfb (fb0) is primary device
[ 13.936158] usb 4-2: New USB device found, idVendor=046d, idProduct=c016, bcdDevice= 3.40
[ 13.936167] usb 4-2: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 13.936170] usb 4-2: Product: Optical USB Mouse
[ 13.936173] usb 4-2: Manufacturer: Logitech
[ 13.945345] input: Logitech Optical USB Mouse as /devices/pci0000:00/0000:00:12.0/usb4/4-2/4-2:1.0/0003:046D:C016.0002/input/input12
[ 13.946178] hid-generic 0003:046D:C016.0002: input,hidraw1: USB HID v1.10 Mouse [Logitech Optical USB Mouse] on usb-0000:00:12.0-2/input0
[ 13.968191] Console: switching to colour frame buffer device 160x64
[ 13.973460] radeon 0000:00:01.0: [drm] fb0: radeondrmfb frame buffer device
[ 13.981574] [drm] Initialized radeon 2.50.0 20080528 for 0000:00:01.0 on minor 0
[ 15.876613] r8169 0000:04:00.0 enp4s0: Link is Up - 1Gbps/Full - flow control rx/tx
[ 15.876628] IPv6: ADDRCONF(NETDEV_CHANGE): enp4s0: link becomes ready
[ 16.713266] [drm] amdgpu kernel modesetting enabled.
[ 17.110008] memfd_create() without MFD_EXEC nor MFD_NOEXEC_SEAL, pid=249 'systemd'
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-17 11:19 ` Paul Menzel
0 siblings, 0 replies; 236+ messages in thread
From: Paul Menzel @ 2023-04-17 11:19 UTC (permalink / raw)
To: Thomas Gleixner
Cc: linux-kernel, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
[-- Attachment #1: Type: text/plain, Size: 1532 bytes --]
Dear Thomas,
Am 15.04.23 um 01:44 schrieb Thomas Gleixner:
> This is a complete rework of the parallel bringup patch series (V17)
>
> https://lore.kernel.org/lkml/20230328195758.1049469-1-usama.arif@bytedance.com
>
> to address the issues which were discovered in review:
[…]
Thank you very much for your rework.
I tested this on the ASUS F2A85-M PRO, and get a delay of ten seconds.
```
[…]
[ 0.258193] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD
Graphics (family: 0x15, model: 0x13, stepping: 0x1)
[…]
[ 0.259329] smp: Bringing up secondary CPUs ...
[ 0.259527] x86: Booting SMP configuration:
[ 0.259528] .... node #0, CPUs: #1
[ 0.261007] After schedule_preempt_disabled
[ 10.260990] CPU1 failed to report alive state
[ 10.261070] smp: Brought up 1 node, 1 CPU
[ 10.261073] smpboot: Max logical packages: 2
[ 10.261074] smpboot: Total of 1 processors activated (7800.54 BogoMIPS)
[ 10.261601] devtmpfs: initialized
[ 10.261697] x86/mm: Memory block size: 128MB
```
This delay has been there with v6.3-rc6-46-gde4664485abbc and some
custom (printk) patches on top and merging dwmw2/parallel-6.2-rc3-v16
into it. I only tested this. I think dwmw2/parallel-6.2-v17 failed to
build for me, when trying to merge it into Linus’ master version at that
time. I didn’t come around to report it, and you posted your rework, so
I am replying here.
I am going to try your branch directly in the next days, but just wanted
to report back already.
Kind regards,
Paul
[-- Attachment #2: kodi-linux-6.3-rc6-smp-tglx.txt --]
[-- Type: text/plain, Size: 61769 bytes --]
[ 0.000000] Linux version 6.3.0-rc6-00311-gde8224969f66 (root@bf16f3646a84) (gcc (Debian 11.2.0-12) 11.2.0, GNU ld (GNU Binutils for Debian) 2.40) #446 SMP PREEMPT_DYNAMIC Sat Apr 15 14:12:29 UTC 2023
[ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc6-00311-gde8224969f66 root=/dev/sda3 rw quiet noisapnp cryptomgr.notests ipv6.disable_ipv6=1 selinux=0
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256
[ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format.
[ 0.000000] signal: max sigframe size: 1776
[ 0.000000] BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
[ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005fe4cfff] usable
[ 0.000000] BIOS-e820: [mem 0x000000005fe4d000-0x000000007fffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000f8000000-0x00000000fbffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000fec10000-0x00000000fec10fff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000017effffff] usable
[ 0.000000] NX (Execute Disable) protection: active
[ 0.000000] SMBIOS 3.0.0 present.
[ 0.000000] DMI: ASUS F2A85-M_PRO/F2A85-M_PRO, BIOS 4.18-9-g9917d2d915 04/17/2023
[ 0.000000] tsc: Fast TSC calibration using PIT
[ 0.000000] tsc: Initial usec timer 6035615
[ 0.000000] tsc: Detected 3900.273 MHz processor
[ 0.000756] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
[ 0.000759] e820: remove [mem 0x000a0000-0x000fffff] usable
[ 0.000763] last_pfn = 0x17f000 max_arch_pfn = 0x400000000
[ 0.000768] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT
[ 0.000942] last_pfn = 0x5fe4d max_arch_pfn = 0x400000000
[ 0.004000] Using GB pages for direct mapping
[ 0.004000] ACPI: Early table checksum verification disabled
[ 0.004000] ACPI: RSDP 0x00000000000F6830 000024 (v02 COREv4)
[ 0.004000] ACPI: XSDT 0x000000005FE5A0E0 000074 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: FACP 0x000000005FE5BBC0 000114 (v06 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: DSDT 0x000000005FE5A280 00193A (v02 COREv4 COREBOOT 00010001 INTL 20200925)
[ 0.004000] ACPI: FACS 0x000000005FE5A240 000040
[ 0.004000] ACPI: FACS 0x000000005FE5A240 000040
[ 0.004000] ACPI: SSDT 0x000000005FE5BCE0 00008A (v02 COREv4 COREBOOT 0000002A CORE 20200925)
[ 0.004000] ACPI: MCFG 0x000000005FE5BD70 00003C (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: APIC 0x000000005FE5BDB0 000062 (v03 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: HPET 0x000000005FE5BE20 000038 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: HEST 0x000000005FE5BE60 0001D0 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: IVRS 0x000000005FE5C030 000070 (v02 AMD AMDIOMMU 00000001 AMD 00000000)
[ 0.004000] ACPI: SSDT 0x000000005FE5C0A0 00051F (v02 AMD ALIB 00000001 MSFT 04000000)
[ 0.004000] ACPI: SSDT 0x000000005FE5C5C0 0006B2 (v01 AMD POWERNOW 00000001 AMD 00000001)
[ 0.004000] ACPI: VFCT 0x000000005FE5CC80 00F269 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: Reserving FACP table memory at [mem 0x5fe5bbc0-0x5fe5bcd3]
[ 0.004000] ACPI: Reserving DSDT table memory at [mem 0x5fe5a280-0x5fe5bbb9]
[ 0.004000] ACPI: Reserving FACS table memory at [mem 0x5fe5a240-0x5fe5a27f]
[ 0.004000] ACPI: Reserving FACS table memory at [mem 0x5fe5a240-0x5fe5a27f]
[ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe5bce0-0x5fe5bd69]
[ 0.004000] ACPI: Reserving MCFG table memory at [mem 0x5fe5bd70-0x5fe5bdab]
[ 0.004000] ACPI: Reserving APIC table memory at [mem 0x5fe5bdb0-0x5fe5be11]
[ 0.004000] ACPI: Reserving HPET table memory at [mem 0x5fe5be20-0x5fe5be57]
[ 0.004000] ACPI: Reserving HEST table memory at [mem 0x5fe5be60-0x5fe5c02f]
[ 0.004000] ACPI: Reserving IVRS table memory at [mem 0x5fe5c030-0x5fe5c09f]
[ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe5c0a0-0x5fe5c5be]
[ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe5c5c0-0x5fe5cc71]
[ 0.004000] ACPI: Reserving VFCT table memory at [mem 0x5fe5cc80-0x5fe6bee8]
[ 0.004000] No NUMA configuration found
[ 0.004000] Faking a node at [mem 0x0000000000000000-0x000000017effffff]
[ 0.004000] NODE_DATA(0) allocated [mem 0x17efe7000-0x17effdfff]
[ 0.004000] Zone ranges:
[ 0.004000] DMA [mem 0x0000000000001000-0x0000000000ffffff]
[ 0.004000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]
[ 0.004000] Normal [mem 0x0000000100000000-0x000000017effffff]
[ 0.004000] Device empty
[ 0.004000] Movable zone start for each node
[ 0.004000] Early memory node ranges
[ 0.004000] node 0: [mem 0x0000000000001000-0x000000000009efff]
[ 0.004000] node 0: [mem 0x0000000000100000-0x000000005fe4cfff]
[ 0.004000] node 0: [mem 0x0000000100000000-0x000000017effffff]
[ 0.004000] Initmem setup node 0 [mem 0x0000000000001000-0x000000017effffff]
[ 0.004000] On node 0, zone DMA: 1 pages in unavailable ranges
[ 0.004000] On node 0, zone DMA: 97 pages in unavailable ranges
[ 0.004000] On node 0, zone Normal: 435 pages in unavailable ranges
[ 0.004000] On node 0, zone Normal: 4096 pages in unavailable ranges
[ 0.004000] ACPI: PM-Timer IO Port: 0x818
[ 0.004000] ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1])
[ 0.004000] IOAPIC[0]: apic_id 4, version 33, address 0xfec00000, GSI 0-23
[ 0.004000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[ 0.004000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 low level)
[ 0.004000] ACPI: Using ACPI (MADT) for SMP configuration information
[ 0.004000] ACPI: HPET id: 0x10228210 base: 0xfed00000
[ 0.004000] smpboot: Allowing 2 CPUs, 0 hotplug CPUs
[ 0.004000] smpboot: smpboot: XXX end of prefill_possible_map
[ 0.004000] After prefill_possible_map
[ 0.004000] After init_cpu_to_node
[ 0.004000] After init_gi_nodes
[ 0.004000] After io_apic_init_mappings
[ 0.004000] After x86_init.hyper.guest_late_init
[ 0.004000] [mem 0x80000000-0xf7ffffff] available for PCI devices
[ 0.004000] After e820
[ 0.004000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns
[ 0.004000] After unwind_init
[ 0.004000] After setup_arch
[ 0.004000] After setup_command_line
[ 0.004000] After setup_nr_cpu_ids
[ 0.004000] setup_percpu: NR_CPUS:64 nr_cpumask_bits:2 nr_cpu_ids:2 nr_node_ids:1
[ 0.004000] percpu: Embedded 55 pages/cpu s188328 r8192 d28760 u1048576
[ 0.004000] pcpu-alloc: s188328 r8192 d28760 u1048576 alloc=1*2097152
[ 0.004000] pcpu-alloc: [0] 0 1
[ 0.004000] After setup_per_cpu_areas
[ 0.004000] After smp_perpare_boot_cpu
[ 0.004000] After boot_cpu_hotplug_init
[ 0.004000] Fallback order for Node 0: 0
[ 0.004000] Built 1 zonelists, mobility grouping on. Total pages: 898451
[ 0.004000] Policy zone: Normal
[ 0.004000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc6-00311-gde8224969f66 root=/dev/sda3 rw quiet noisapnp cryptomgr.notests ipv6.disable_ipv6=1 selinux=0
[ 0.004000] Unknown kernel command line parameters "noisapnp BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc6-00311-gde8224969f66", will be passed to user space.
[ 0.004000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[ 0.004000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[ 0.004000] mem auto-init: stack:off, heap alloc:off, heap free:off
[ 0.004000] stackdepot: allocating hash table via alloc_large_system_hash
[ 0.004000] stackdepot hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[ 0.004000] software IO TLB: area num 2.
[ 0.004000] Memory: 3477160K/3651500K available (14336K kernel code, 2340K rwdata, 5308K rodata, 2908K init, 11060K bss, 174080K reserved, 0K cma-reserved)
[ 0.004000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
[ 0.004000] After mm_init
[ 0.004000] After poking_init
[ 0.004000] ftrace: allocating 38664 entries in 152 pages
[ 0.004000] ftrace: allocated 152 pages with 3 groups
[ 0.004000] Dynamic Preempt: full
[ 0.004000] After sched_init
[ 0.004000] rcu: Preemptible hierarchical RCU implementation.
[ 0.004000] rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=2.
[ 0.004000] Trampoline variant of Tasks RCU enabled.
[ 0.004000] Rude variant of Tasks RCU enabled.
[ 0.004000] Tracing variant of Tasks RCU enabled.
[ 0.004000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[ 0.004000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2
[ 0.004000] After rcu_init
[ 0.004000] NR_IRQS: 4352, nr_irqs: 440, preallocated irqs: 16
[ 0.004000] rcu: srcu_init: Setting srcu_struct sizes based on contention.
[ 0.004000] After random_init()
[ 0.004000] After boot_init_stack_canary
[ 0.004000] spurious 8259A interrupt: IRQ7.
[ 0.004000] Console: colour VGA+ 80x25
[ 0.004000] printk: console [tty0] enabled
[ 0.004000] ACPI: Core revision 20221020
[ 0.004000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 133484873504 ns
[ 0.004000] APIC: Switch to symmetric I/O mode setup
[ 0.004000] AMD-Vi: Using global IVHD EFR:0x0, EFR2:0x0
[ 0.004000] APIC: Done
[ 0.004000] Before apic_bsb_setup
[ 0.004000] check_timer begin
[ 0.004000] check_timer after local_irq_disable
[ 0.004000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[ 0.004000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x7070b6fd6fc, max_idle_ns: 881590611710 ns
[ 0.144982] Calibrating delay loop (skipped), value calculated using timer frequency.. 7800.54 BogoMIPS (lpj=15601092)
[ 0.144986] pid_max: default: 32768 minimum: 301
[ 0.145079] LSM: initializing lsm=capability
[ 0.145173] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[ 0.145189] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[ 0.145564] Bit 30 in CPUID ECX not set.
[ 0.145587] Last level iTLB entries: 4KB 512, 2MB 1024, 4MB 512
[ 0.145589] Last level dTLB entries: 4KB 1024, 2MB 1024, 4MB 512, 1GB 0
[ 0.145594] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization
[ 0.145597] Spectre V2 : Mitigation: Retpolines
[ 0.145598] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch
[ 0.145599] Spectre V2 : Spectre v2 / SpectreRSB : Filling RSB on VMEXIT
[ 0.145600] Spectre V2 : Enabling Speculation Barrier for firmware calls
[ 0.145600] RETBleed: Mitigation: untrained return thunk
[ 0.145602] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier
[ 0.145604] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl
[ 0.150046] Freeing SMP alternatives memory: 32K
[ 0.150051] After check_bugs
[ 0.150052] After acpi_subsystem_init
[ 0.150053] After arch_post_acpi_subsys_init
[ 0.150054] After rcu_scheduler_starting
[ 0.150126] After find_task_by_pid_ns and PF_NO_SETAFFINITY
[ 0.150131] After numa_default_policy
[ 0.150151] After rcu_read_lock
[ 0.150152] After rcu_read_unlock
[ 0.150153] After kthreadd_done
[ 0.150165] smpboot: Start of smp_prepare_cpus_common
[ 0.150167] smpboot: smpboot: zalloc 0
[ 0.150168] smpboot: smpboot: zalloc 1
[ 0.150169] smpboot: smpboot: After set_sched_topology()
[ 0.150171] smpboot: smpboot: After smp_sanity_check()
[ 0.150172] smpboot: smpboot: Before x86_init.timers.setup_percpu_clockev()
[ 0.258192] smpboot: smpboot: After x86_init.timers.setup_percpu_clockev()
[ 0.258193] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD Graphics (family: 0x15, model: 0x13, stepping: 0x1)
[ 0.258425] cblist_init_generic: Setting adjustable number of callback queues.
[ 0.258427] cblist_init_generic: Setting shift to 1 and lim to 1.
[ 0.258458] cblist_init_generic: Setting shift to 1 and lim to 1.
[ 0.258484] cblist_init_generic: Setting shift to 1 and lim to 1.
[ 0.258513] Performance Events: Fam15h core perfctr, AMD PMU driver.
[ 0.258536] ... version: 0
[ 0.258537] ... bit width: 48
[ 0.258538] ... generic registers: 6
[ 0.258539] ... value mask: 0000ffffffffffff
[ 0.258540] ... max period: 00007fffffffffff
[ 0.258541] ... fixed-purpose events: 0
[ 0.258542] ... event mask: 000000000000003f
[ 0.258662] rcu: Hierarchical SRCU implementation.
[ 0.258663] rcu: Max phase no-delay instances is 1000.
[ 0.259257] NMI watchdog: Enabled. Permanently consumes one hw-PMU counter.
[ 0.259329] smp: Bringing up secondary CPUs ...
[ 0.259527] x86: Booting SMP configuration:
[ 0.259528] .... node #0, CPUs: #1
[ 0.261007] After schedule_preempt_disabled
[ 10.260990] CPU1 failed to report alive state
[ 10.261070] smp: Brought up 1 node, 1 CPU
[ 10.261073] smpboot: Max logical packages: 2
[ 10.261074] smpboot: Total of 1 processors activated (7800.54 BogoMIPS)
[ 10.261601] devtmpfs: initialized
[ 10.261697] x86/mm: Memory block size: 128MB
[ 10.262788] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[ 10.262795] futex hash table entries: 512 (order: 3, 32768 bytes, linear)
[ 10.262883] pinctrl core: initialized pinctrl subsystem
[ 10.262953] PM: RTC time: 07:39:50, date: 2023-04-17
[ 10.263709] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[ 10.263937] audit: initializing netlink subsys (disabled)
[ 10.264192] thermal_sys: Registered thermal governor 'fair_share'
[ 10.264193] thermal_sys: Registered thermal governor 'bang_bang'
[ 10.264194] thermal_sys: Registered thermal governor 'step_wise'
[ 10.264195] thermal_sys: Registered thermal governor 'user_space'
[ 10.264215] cpuidle: using governor ladder
[ 10.264220] cpuidle: using governor menu
[ 10.264427] PCI: MMCONFIG for domain 0000 [bus 00-3f] at [mem 0xf8000000-0xfbffffff] (base 0xf8000000)
[ 10.264432] PCI: MMCONFIG at [mem 0xf8000000-0xfbffffff] reserved as E820 entry
[ 10.264444] PCI: Using configuration type 1 for base access
[ 10.264646] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.
[ 10.269064] audit: type=2000 audit(1681717190.140:1): state=initialized audit_enabled=0 res=1
[ 10.281080] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages
[ 10.281084] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page
[ 10.281085] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages
[ 10.281086] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page
[ 10.286166] cryptd: max_cpu_qlen set to 1000
[ 10.289615] ACPI: Added _OSI(Module Device)
[ 10.289617] ACPI: Added _OSI(Processor Device)
[ 10.289618] ACPI: Added _OSI(3.0 _SCP Extensions)
[ 10.289620] ACPI: Added _OSI(Processor Aggregator Device)
[ 10.293942] ACPI: DSDT successfully acquired and loaded
[ 10.295626] ACPI: 4 ACPI AML tables successfully acquired and loaded
[ 10.297161] ACPI: Interpreter enabled
[ 10.297184] ACPI: PM: (supports S0 S1 S3 S5)
[ 10.297186] ACPI: Using IOAPIC for interrupt routing
[ 10.297236] HEST: Table parsing has been initialized.
[ 10.297258] GHES: Failed to enable APEI firmware first mode.
[ 10.297261] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[ 10.297262] PCI: Ignoring E820 reservations for host bridge windows
[ 10.297510] ACPI: Enabled 8 GPEs in block 00 to 1F
[ 10.303230] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[ 10.303241] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]
[ 10.303320] acpi PNP0A03:00: _OSC: OS now controls [PME AER PCIeCapability LTR]
[ 10.303335] acpi PNP0A03:00: [Firmware Info]: MMCONFIG for domain 0000 [bus 00-3f] only partially covers this bridge
[ 10.303414] acpi PNP0A03:00: host bridge window expanded to [io 0x0000-0x0cf7 window]; [io 0x03b0-0x03df window] ignored
[ 10.303651] PCI host bridge to bus 0000:00
[ 10.303653] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]
[ 10.303655] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]
[ 10.303658] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000dffff]
[ 10.303660] pci_bus 0000:00: root bus resource [mem 0x80000000-0xffffffff]
[ 10.303662] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 10.303686] pci 0000:00:00.0: [1022:1410] type 00 class 0x060000
[ 10.303830] pci 0000:00:00.2: [1022:1419] type 00 class 0x080600
[ 10.303921] pci 0000:00:01.0: [1002:9996] type 00 class 0x030000
[ 10.303929] pci 0000:00:01.0: reg 0x10: [mem 0xe0000000-0xefffffff pref]
[ 10.303934] pci 0000:00:01.0: reg 0x14: [io 0x1000-0x10ff]
[ 10.303939] pci 0000:00:01.0: reg 0x18: [mem 0xf0180000-0xf01bffff]
[ 10.303955] pci 0000:00:01.0: enabling Extended Tags
[ 10.303965] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]
[ 10.303983] pci 0000:00:01.0: supports D1 D2
[ 10.304048] pci 0000:00:01.1: [1002:9902] type 00 class 0x040300
[ 10.304056] pci 0000:00:01.1: reg 0x10: [mem 0xf01c0000-0xf01c3fff]
[ 10.304077] pci 0000:00:01.1: enabling Extended Tags
[ 10.304101] pci 0000:00:01.1: supports D1 D2
[ 10.304185] pci 0000:00:11.0: [1022:7801] type 00 class 0x010601
[ 10.304198] pci 0000:00:11.0: reg 0x10: [io 0x1410-0x1417]
[ 10.304206] pci 0000:00:11.0: reg 0x14: [io 0x1420-0x1423]
[ 10.304213] pci 0000:00:11.0: reg 0x18: [io 0x1418-0x141f]
[ 10.304221] pci 0000:00:11.0: reg 0x1c: [io 0x1424-0x1427]
[ 10.304228] pci 0000:00:11.0: reg 0x20: [io 0x1400-0x140f]
[ 10.304235] pci 0000:00:11.0: reg 0x24: [mem 0xf01cc000-0xf01cc7ff]
[ 10.304392] pci 0000:00:12.0: [1022:7807] type 00 class 0x0c0310
[ 10.304406] pci 0000:00:12.0: reg 0x10: [mem 0xf01c8000-0xf01c8fff]
[ 10.304587] pci 0000:00:12.2: [1022:7808] type 00 class 0x0c0320
[ 10.304601] pci 0000:00:12.2: reg 0x10: [mem 0xf01cd000-0xf01cd0ff]
[ 10.304666] pci 0000:00:12.2: supports D1 D2
[ 10.304667] pci 0000:00:12.2: PME# supported from D0 D1 D2 D3hot
[ 10.304668] pci 0000:00:12.2: pme_poll = true
[ 10.304669] pci 0000:00:12.2: after device_set_wakeup_capable()
[ 10.304673] pci 0000:00:12.2: after pci_pme_active()
[ 10.304814] pci 0000:00:13.0: [1022:7807] type 00 class 0x0c0310
[ 10.304827] pci 0000:00:13.0: reg 0x10: [mem 0xf01c9000-0xf01c9fff]
[ 10.305566] pci 0000:00:13.2: [1022:7808] type 00 class 0x0c0320
[ 10.305583] pci 0000:00:13.2: reg 0x10: [mem 0xf01ce000-0xf01ce0ff]
[ 10.305654] pci 0000:00:13.2: supports D1 D2
[ 10.305656] pci 0000:00:13.2: PME# supported from D0 D1 D2 D3hot
[ 10.305657] pci 0000:00:13.2: pme_poll = true
[ 10.305659] pci 0000:00:13.2: after device_set_wakeup_capable()
[ 10.305662] pci 0000:00:13.2: after pci_pme_active()
[ 10.305803] pci 0000:00:14.0: [1022:780b] type 00 class 0x0c0500
[ 10.305975] pci 0000:00:14.2: [1022:780d] type 00 class 0x040300
[ 10.305992] pci 0000:00:14.2: reg 0x10: [mem 0xf01c4000-0xf01c7fff 64bit]
[ 10.306047] pci 0000:00:14.2: PME# supported from D0 D3hot D3cold
[ 10.306049] pci 0000:00:14.2: pme_poll = true
[ 10.306050] pci 0000:00:14.2: after device_set_wakeup_capable()
[ 10.306052] pci 0000:00:14.2: after pci_pme_active()
[ 10.306185] pci 0000:00:14.3: [1022:780e] type 00 class 0x060100
[ 10.306360] pci 0000:00:14.4: [1022:780f] type 01 class 0x060401
[ 10.306502] pci 0000:00:14.5: [1022:7809] type 00 class 0x0c0310
[ 10.306515] pci 0000:00:14.5: reg 0x10: [mem 0xf01ca000-0xf01cafff]
[ 10.306682] pci 0000:00:15.0: [1022:43a0] type 01 class 0x060400
[ 10.306711] pci 0000:00:15.0: enabling Extended Tags
[ 10.306751] pci 0000:00:15.0: supports D1 D2
[ 10.306911] pci 0000:00:15.1: [1022:43a1] type 01 class 0x060400
[ 10.306942] pci 0000:00:15.1: enabling Extended Tags
[ 10.306981] pci 0000:00:15.1: supports D1 D2
[ 10.307137] pci 0000:00:15.2: [1022:43a2] type 01 class 0x060400
[ 10.307166] pci 0000:00:15.2: enabling Extended Tags
[ 10.307205] pci 0000:00:15.2: supports D1 D2
[ 10.307280] pci 0000:00:16.0: [1022:7807] type 00 class 0x0c0310
[ 10.307293] pci 0000:00:16.0: reg 0x10: [mem 0xf01cb000-0xf01cbfff]
[ 10.307467] pci 0000:00:16.2: [1022:7808] type 00 class 0x0c0320
[ 10.307481] pci 0000:00:16.2: reg 0x10: [mem 0xf01cf000-0xf01cf0ff]
[ 10.307546] pci 0000:00:16.2: supports D1 D2
[ 10.307547] pci 0000:00:16.2: PME# supported from D0 D1 D2 D3hot
[ 10.307548] pci 0000:00:16.2: pme_poll = true
[ 10.307549] pci 0000:00:16.2: after device_set_wakeup_capable()
[ 10.307552] pci 0000:00:16.2: after pci_pme_active()
[ 10.307691] pci 0000:00:18.0: [1022:1400] type 00 class 0x060000
[ 10.307758] pci 0000:00:18.1: [1022:1401] type 00 class 0x060000
[ 10.307819] pci 0000:00:18.2: [1022:1402] type 00 class 0x060000
[ 10.307880] pci 0000:00:18.3: [1022:1403] type 00 class 0x060000
[ 10.308016] pci 0000:00:18.4: [1022:1404] type 00 class 0x060000
[ 10.308079] pci 0000:00:18.5: [1022:1405] type 00 class 0x060000
[ 10.308153] pci_bus 0000:01: extended config space not accessible
[ 10.308218] pci 0000:00:14.4: PCI bridge to [bus 01] (subtractive decode)
[ 10.308227] pci 0000:00:14.4: bridge window [io 0x0000-0x0cf7 window] (subtractive decode)
[ 10.308230] pci 0000:00:14.4: bridge window [io 0x0d00-0xffff window] (subtractive decode)
[ 10.308232] pci 0000:00:14.4: bridge window [mem 0x000a0000-0x000dffff] (subtractive decode)
[ 10.308235] pci 0000:00:14.4: bridge window [mem 0x80000000-0xffffffff] (subtractive decode)
[ 10.308283] pci 0000:00:15.0: PCI bridge to [bus 02]
[ 10.308368] pci 0000:03:00.0: [1b21:1042] type 00 class 0x0c0330
[ 10.308404] pci 0000:03:00.0: reg 0x10: [mem 0xf0000000-0xf0007fff 64bit]
[ 10.308579] pci 0000:03:00.0: PME# supported from D3hot D3cold
[ 10.308581] pci 0000:03:00.0: pme_poll = true
[ 10.308582] pci 0000:03:00.0: after device_set_wakeup_capable()
[ 10.308587] pci 0000:03:00.0: after pci_pme_active()
[ 10.308625] pci 0000:03:00.0: 2.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s PCIe x1 link at 0000:00:15.1 (capable of 4.000 Gb/s with 5.0 GT/s PCIe x1 link)
[ 10.317133] pci 0000:00:15.1: PCI bridge to [bus 03]
[ 10.317145] pci 0000:00:15.1: bridge window [mem 0xf0000000-0xf00fffff]
[ 10.317154] pci 0000:00:15.2: bridge configuration invalid ([bus 00-00]), reconfiguring
[ 10.317278] pci 0000:04:00.0: [10ec:8168] type 00 class 0x020000
[ 10.317296] pci 0000:04:00.0: reg 0x10: [io 0x0000-0x00ff]
[ 10.317318] pci 0000:04:00.0: reg 0x18: [mem 0x00000000-0x00000fff 64bit pref]
[ 10.317332] pci 0000:04:00.0: reg 0x20: [mem 0x00000000-0x00003fff 64bit pref]
[ 10.317440] pci 0000:04:00.0: supports D1 D2
[ 10.317442] pci 0000:04:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[ 10.317444] pci 0000:04:00.0: pme_poll = true
[ 10.317445] pci 0000:04:00.0: after device_set_wakeup_capable()
[ 10.317449] pci 0000:04:00.0: after pci_pme_active()
[ 10.329042] pci 0000:00:15.2: PCI bridge to [bus 04-ff]
[ 10.329053] pci 0000:00:15.2: bridge window [io 0x0000-0x0fff]
[ 10.329057] pci 0000:00:15.2: bridge window [mem 0x00000000-0x000fffff]
[ 10.329062] pci 0000:00:15.2: bridge window [mem 0x00000000-0x000fffff 64bit pref]
[ 10.329065] pci_bus 0000:04: busn_res: [bus 04-ff] end is updated to 04
[ 10.329575] ACPI: PCI: Interrupt link INTA configured for IRQ 0
[ 10.329669] ACPI: PCI: Interrupt link INTB configured for IRQ 0
[ 10.329759] ACPI: PCI: Interrupt link INTC configured for IRQ 0
[ 10.329849] ACPI: PCI: Interrupt link INTD configured for IRQ 0
[ 10.329940] ACPI: PCI: Interrupt link INTE configured for IRQ 0
[ 10.330029] ACPI: PCI: Interrupt link INTF configured for IRQ 0
[ 10.330119] ACPI: PCI: Interrupt link INTG configured for IRQ 0
[ 10.330210] ACPI: PCI: Interrupt link INTH configured for IRQ 0
[ 10.330444] iommu: Default domain type: Translated
[ 10.330446] iommu: DMA domain TLB invalidation policy: lazy mode
[ 10.330624] SCSI subsystem initialized
[ 10.330718] libata version 3.00 loaded.
[ 10.330752] ACPI: bus type USB registered
[ 10.330775] usbcore: registered new interface driver usbfs
[ 10.330786] usbcore: registered new interface driver hub
[ 10.330798] usbcore: registered new device driver usb
[ 10.331125] PCI: Using ACPI for IRQ routing
[ 10.332694] PCI: pci_cache_line_size set to 64 bytes
[ 10.332745] e820: reserve RAM buffer [mem 0x0009fc00-0x0009ffff]
[ 10.332748] e820: reserve RAM buffer [mem 0x5fe4d000-0x5fffffff]
[ 10.332750] e820: reserve RAM buffer [mem 0x17f000000-0x17fffffff]
[ 10.332794] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0
[ 10.332799] hpet0: 3 comparators, 32-bit 14.318180 MHz counter
[ 10.334067] clocksource: Switched to clocksource tsc-early
[ 10.350705] VFS: Disk quotas dquot_6.6.0
[ 10.350734] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[ 10.350847] pnp: PnP ACPI init
[ 10.351137] system 00:00: [mem 0xfec10002-0xfec11001] could not be reserved
[ 10.351446] pnp: PnP ACPI: found 2 devices
[ 10.358408] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
[ 10.358544] NET: Registered PF_INET protocol family
[ 10.358686] IP idents hash table entries: 65536 (order: 7, 524288 bytes, linear)
[ 10.360256] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear)
[ 10.360271] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
[ 10.360277] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear)
[ 10.360344] TCP bind hash table entries: 32768 (order: 8, 1048576 bytes, linear)
[ 10.360670] TCP: Hash tables configured (established 32768 bind 32768)
[ 10.360739] UDP hash table entries: 2048 (order: 4, 65536 bytes, linear)
[ 10.360763] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear)
[ 10.360870] NET: Registered PF_UNIX/PF_LOCAL protocol family
[ 10.360906] pci 0000:00:15.2: BAR 15: assigned [mem 0x80000000-0x800fffff 64bit pref]
[ 10.360912] pci 0000:00:15.2: BAR 13: assigned [io 0x2000-0x2fff]
[ 10.360917] pci 0000:00:14.4: PCI bridge to [bus 01]
[ 10.360928] pci 0000:00:15.0: PCI bridge to [bus 02]
[ 10.360936] pci 0000:00:15.1: PCI bridge to [bus 03]
[ 10.360939] pci 0000:00:15.1: bridge window [mem 0xf0000000-0xf00fffff]
[ 10.360947] pci 0000:04:00.0: BAR 4: assigned [mem 0x80000000-0x80003fff 64bit pref]
[ 10.360959] pci 0000:04:00.0: BAR 2: assigned [mem 0x80004000-0x80004fff 64bit pref]
[ 10.360971] pci 0000:04:00.0: BAR 0: assigned [io 0x2000-0x20ff]
[ 10.360976] pci 0000:00:15.2: PCI bridge to [bus 04]
[ 10.361116] pci 0000:00:15.2: bridge window [io 0x2000-0x2fff]
[ 10.361122] pci 0000:00:15.2: bridge window [mem 0x80000000-0x800fffff 64bit pref]
[ 10.361128] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]
[ 10.361130] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]
[ 10.361132] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000dffff]
[ 10.361133] pci_bus 0000:00: resource 7 [mem 0x80000000-0xffffffff]
[ 10.361135] pci_bus 0000:01: resource 4 [io 0x0000-0x0cf7 window]
[ 10.361137] pci_bus 0000:01: resource 5 [io 0x0d00-0xffff window]
[ 10.361138] pci_bus 0000:01: resource 6 [mem 0x000a0000-0x000dffff]
[ 10.361140] pci_bus 0000:01: resource 7 [mem 0x80000000-0xffffffff]
[ 10.361141] pci_bus 0000:03: resource 1 [mem 0xf0000000-0xf00fffff]
[ 10.361143] pci_bus 0000:04: resource 0 [io 0x2000-0x2fff]
[ 10.361144] pci_bus 0000:04: resource 2 [mem 0x80000000-0x800fffff 64bit pref]
[ 10.361239] pci 0000:00:01.1: D0 power state depends on 0000:00:01.0
[ 10.361456] pci 0000:00:12.0: AMD USB device
[ 10.361478] pci 0000:00:12.0: AMD USB ohci handoff
[ 10.361761] pci 0000:00:12.2: AMD USB device
[ 10.361775] pci 0000:00:12.2: AMD USB ehci handoff
[ 10.361903] pci 0000:00:12.2: PME# does not work under D3, disabling it
[ 10.362049] pci 0000:00:13.0: AMD USB device
[ 10.362062] pci 0000:00:13.0: AMD USB ohci handoff
[ 10.362328] pci 0000:00:13.2: AMD USB device
[ 10.362339] pci 0000:00:13.2: AMD USB ehci handoff
[ 10.362469] pci 0000:00:13.2: PME# does not work under D3, disabling it
[ 10.362622] pci 0000:00:14.5: AMD USB device
[ 10.362634] pci 0000:00:14.5: AMD USB ohci handoff
[ 10.362915] pci 0000:00:16.0: AMD USB device
[ 10.362928] pci 0000:00:16.0: AMD USB ohci handoff
[ 10.363195] pci 0000:00:16.2: AMD USB device
[ 10.363206] pci 0000:00:16.2: AMD USB ehci handoff
[ 10.363334] pci 0000:00:16.2: PME# does not work under D3, disabling it
[ 10.363561] pci 0000:03:00.0: AMD USB xhci handoff
[ 10.363610] PCI: CLS 64 bytes, default 64
[ 10.363723] pci 0000:00:00.2: AMD-Vi: Applying erratum 746 workaround
[ 10.363810] pci 0000:00:01.0: Adding to iommu group 0
[ 10.363826] pci 0000:00:01.1: Adding to iommu group 0
[ 10.363851] pci 0000:00:11.0: Adding to iommu group 1
[ 10.363886] pci 0000:00:12.0: Adding to iommu group 2
[ 10.363903] pci 0000:00:12.2: Adding to iommu group 2
[ 10.363937] pci 0000:00:13.0: Adding to iommu group 3
[ 10.363953] pci 0000:00:13.2: Adding to iommu group 3
[ 10.363991] pci 0000:00:14.0: Adding to iommu group 4
[ 10.364009] pci 0000:00:14.2: Adding to iommu group 4
[ 10.364025] pci 0000:00:14.3: Adding to iommu group 4
[ 10.364048] pci 0000:00:14.4: Adding to iommu group 5
[ 10.364070] pci 0000:00:14.5: Adding to iommu group 6
[ 10.364104] pci 0000:00:15.0: Adding to iommu group 7
[ 10.364123] pci 0000:00:15.1: Adding to iommu group 7
[ 10.364139] pci 0000:00:15.2: Adding to iommu group 7
[ 10.364178] pci 0000:00:16.0: Adding to iommu group 8
[ 10.364194] pci 0000:00:16.2: Adding to iommu group 8
[ 10.364246] pci 0000:00:18.0: Adding to iommu group 9
[ 10.364266] pci 0000:00:18.1: Adding to iommu group 9
[ 10.364283] pci 0000:00:18.2: Adding to iommu group 9
[ 10.364303] pci 0000:00:18.3: Adding to iommu group 9
[ 10.364321] pci 0000:00:18.4: Adding to iommu group 9
[ 10.364340] pci 0000:00:18.5: Adding to iommu group 9
[ 10.364352] pci 0000:03:00.0: Adding to iommu group 7
[ 10.364360] pci 0000:04:00.0: Adding to iommu group 7
[ 10.366500] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[ 10.366505] AMD-Vi: Extended features (0x800000853, 0x0): PreF PPR GT IA
[ 10.366510] AMD-Vi: Interrupt remapping enabled
[ 10.366691] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[ 10.366693] software IO TLB: mapped [mem 0x000000005be4d000-0x000000005fe4d000] (64MB)
[ 10.366742] LVT offset 0 assigned for vector 0x400
[ 10.366763] perf: AMD IBS detected (0x000000ff)
[ 10.366771] amd_uncore: 4 amd_nb counters detected
[ 10.367574] workingset: timestamp_bits=37 max_order=20 bucket_order=0
[ 10.367603] zbud: loaded
[ 10.368066] NET: Registered PF_ALG protocol family
[ 10.368071] Key type asymmetric registered
[ 10.368073] Asymmetric key parser 'x509' registered
[ 10.368346] alg: self-tests disabled
[ 10.368439] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 251)
[ 10.368477] io scheduler mq-deadline registered
[ 10.368479] io scheduler kyber registered
[ 10.370011] pcieport 0000:00:15.0: PME: Signaling with IRQ 25
[ 10.370174] pcieport 0000:00:15.1: PME: Signaling with IRQ 26
[ 10.370247] pcieport 0000:00:15.2: enabling device (0000 -> 0003)
[ 10.370449] pcieport 0000:00:15.2: PME: Signaling with IRQ 27
[ 10.370706] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
[ 10.370767] ACPI: button: Power Button [PWRF]
[ 10.370823] ACPI: \_SB_.P000: Found 2 idle states
[ 10.370937] ACPI: \_SB_.P001: Found 2 idle states
[ 10.371827] thermal LNXTHERM:00: registered as thermal_zone0
[ 10.371830] ACPI: thermal: Thermal Zone [TZ00] (0 C)
[ 10.372146] Non-volatile memory driver v1.3
[ 10.372218] AMD-Vi: AMD IOMMUv2 loaded and initialized
[ 10.372338] ahci 0000:00:11.0: version 3.0
[ 10.372612] ahci 0000:00:11.0: AHCI 0001.0300 32 slots 8 ports 6 Gbps 0x40 impl SATA mode
[ 10.372616] ahci 0000:00:11.0: flags: 64bit ncq sntf ilck led clo pio
[ 10.374133] scsi host0: ahci
[ 10.374333] scsi host1: ahci
[ 10.374510] scsi host2: ahci
[ 10.374710] scsi host3: ahci
[ 10.374879] scsi host4: ahci
[ 10.375057] scsi host5: ahci
[ 10.375241] scsi host6: ahci
[ 10.375421] scsi host7: ahci
[ 10.375508] ata port1: DUMMY
[ 10.375510] ata port2: DUMMY
[ 10.375511] ata port3: DUMMY
[ 10.375512] ata port4: DUMMY
[ 10.375514] ata port5: DUMMY
[ 10.375515] ata port6: DUMMY
[ 10.375517] ata port7: SATA max UDMA/133 abar m2048@0xf01cc000 port 0xf01cc400 irq 19
[ 10.375519] ata port8: DUMMY
[ 10.375597] ACPI: bus type drm_connector registered
[ 10.375823] i8042: PNP: No PS/2 controller found.
[ 10.375824] i8042: Probing ports directly.
[ 10.378675] serio: i8042 KBD port at 0x60,0x64 irq 1
[ 10.378751] serio: i8042 AUX port at 0x60,0x64 irq 12
[ 10.378874] mousedev: PS/2 mouse device common for all mice
[ 10.378927] rtc_cmos 00:01: RTC can wake from S4
[ 10.379173] rtc_cmos 00:01: registered as rtc0
[ 10.379197] rtc_cmos 00:01: setting system clock to 2023-04-17T07:39:50 UTC (1681717190)
[ 10.379234] rtc_cmos 00:01: alarms up to one day, y3k, 114 bytes nvram, hpet irqs
[ 10.379268] device-mapper: uevent: version 1.0.3
[ 10.379337] device-mapper: ioctl: 4.47.0-ioctl (2022-07-28) initialised: dm-devel@redhat.com
[ 10.379496] hid: raw HID events driver (C) Jiri Kosina
[ 10.379531] usbcore: registered new interface driver usbhid
[ 10.379532] usbhid: USB HID core driver
[ 10.379638] Initializing XFRM netlink socket
[ 10.379648] NET: Registered PF_PACKET protocol family
[ 10.379650] x86/pm: family 0x15 cpu detected, MSR saving is needed during suspending.
[ 10.379817] microcode: CPU0: patch_level=0x0600111f
[ 10.379827] microcode: Microcode Update Driver: v2.2.
[ 10.379831] IPI shorthand broadcast: enabled
[ 10.379838] AVX version of gcm_enc/dec engaged.
[ 10.379854] AES CTR mode by8 optimization enabled
[ 10.380695] ata link7: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[ 10.380969] ata dev7.0: ATA-9: SanDisk SDSSDP064G, 2.0.0, max UDMA/133
[ 10.380972] ata dev7.0: 125045424 sectors, multi 1: LBA48 NCQ (depth 32)
[ 10.381178] ata dev7.0: configured for UDMA/133
[ 10.381289] scsi 6:0:0:0: Direct-Access ATA SanDisk SDSSDP06 0 PQ: 0 ANSI: 5
[ 10.381718] sd 6:0:0:0: [sda] 125045424 512-byte logical blocks: (64.0 GB/59.6 GiB)
[ 10.381730] sd 6:0:0:0: [sda] Write Protect is off
[ 10.381733] sd 6:0:0:0: [sda] Mode Sense: 00 3a 00 00
[ 10.381749] sd 6:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 10.381775] sd 6:0:0:0: [sda] Preferred minimum I/O size 512 bytes
[ 10.382547] sda: sda1 sda2 sda3
[ 10.382736] sd 6:0:0:0: [sda] Attached SCSI disk
[ 10.385344] sched_clock: Marking stable (10268007974, 116977866)->(10387423156, -2437316)
[ 10.385519] registered taskstats version 1
[ 10.385734] zswap: loaded using pool lzo/zbud
[ 10.390028] kmemleak: Kernel memory leak detector initialized (mem pool available: 15677)
[ 10.390033] debug_vm_pgtable: [debug_vm_pgtable ]: Validating architecture page table helpers
[ 10.393894] kmemleak: Automatic memory scanning thread started
[ 10.394488] Key type encrypted registered
[ 10.397453] PM: Magic number: 3:139:673
[ 10.397466] workqueue scsi_tmf_3: hash matches
[ 10.410334] EXT4-fs (sda3): mounted filesystem fe29e0dc-6303-4401-987c-8472bc1b9516 with ordered data mode. Quota mode: none.
[ 10.410378] VFS: Mounted root (ext4 filesystem) on device 8:3.
[ 10.412268] devtmpfs: mounted
[ 10.412286] After kernel_init_freeable
[ 10.416793] Freeing unused kernel image (initmem) memory: 2908K
[ 10.421291] Write protecting the kernel read-only data: 20480k
[ 10.421559] Freeing unused kernel image (rodata/data gap) memory: 836K
[ 10.458614] x86/mm: Checked W+X mappings: passed, no W+X pages found.
[ 10.458620] rodata_test: all tests were successful
[ 10.458621] After mark_readonly
[ 10.458621] After pti_finalize
[ 10.458637] rcu_end_inkernel_boot
[ 10.458644] Run /sbin/init as init process
[ 10.458646] with arguments:
[ 10.458648] /sbin/init
[ 10.458649] noisapnp
[ 10.458650] with environment:
[ 10.458650] HOME=/
[ 10.458651] TERM=linux
[ 10.458652] BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc6-00311-gde8224969f66
[ 10.637190] systemd[1]: Inserted module 'autofs4'
[ 10.663625] NET: Registered PF_INET6 protocol family
[ 10.664473] Segment Routing with IPv6
[ 10.664500] In-situ OAM (IOAM) with IPv6
[ 10.691376] systemd[1]: systemd 252.6-1 running in system mode (+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
[ 10.691387] systemd[1]: Detected architecture x86-64.
[ 10.696468] systemd[1]: Hostname set to <kodi>.
[ 11.000700] systemd[1]: Queued start job for default target graphical.target.
[ 11.011651] systemd[1]: Created slice system-getty.slice - Slice /system/getty.
[ 11.012747] systemd[1]: Created slice system-modprobe.slice - Slice /system/modprobe.
[ 11.013600] systemd[1]: Created slice user.slice - User and Session Slice.
[ 11.013784] systemd[1]: Started systemd-ask-password-console.path - Dispatch Password Requests to Console Directory Watch.
[ 11.013902] systemd[1]: Started systemd-ask-password-wall.path - Forward Password Requests to Wall Directory Watch.
[ 11.014324] systemd[1]: Set up automount proc-sys-fs-binfmt_misc.automount - Arbitrary Executable File Formats File System Automount Point.
[ 11.014366] systemd[1]: Reached target cryptsetup.target - Local Encrypted Volumes.
[ 11.014416] systemd[1]: Reached target integritysetup.target - Local Integrity Protected Volumes.
[ 11.014455] systemd[1]: Reached target paths.target - Path Units.
[ 11.014485] systemd[1]: Reached target remote-fs.target - Remote File Systems.
[ 11.014515] systemd[1]: Reached target slices.target - Slice Units.
[ 11.014554] systemd[1]: Reached target swap.target - Swaps.
[ 11.014592] systemd[1]: Reached target veritysetup.target - Local Verity Protected Volumes.
[ 11.017065] systemd[1]: Listening on systemd-coredump.socket - Process Core Dump Socket.
[ 11.017317] systemd[1]: Listening on systemd-fsckd.socket - fsck to fsckd communication Socket.
[ 11.017479] systemd[1]: Listening on systemd-initctl.socket - initctl Compatibility Named Pipe.
[ 11.017786] systemd[1]: Listening on systemd-journald-audit.socket - Journal Audit Socket.
[ 11.018059] systemd[1]: Listening on systemd-journald-dev-log.socket - Journal Socket (/dev/log).
[ 11.018337] systemd[1]: Listening on systemd-journald.socket - Journal Socket.
[ 11.018594] systemd[1]: Listening on systemd-networkd.socket - Network Service Netlink Socket.
[ 11.019420] systemd[1]: Listening on systemd-udevd-control.socket - udev Control Socket.
[ 11.019693] systemd[1]: Listening on systemd-udevd-kernel.socket - udev Kernel Socket.
[ 11.022504] systemd[1]: Mounting dev-hugepages.mount - Huge Pages File System...
[ 11.025189] systemd[1]: Mounting dev-mqueue.mount - POSIX Message Queue File System...
[ 11.029590] systemd[1]: Mounting sys-kernel-debug.mount - Kernel Debug File System...
[ 11.046849] systemd[1]: Mounting sys-kernel-tracing.mount - Kernel Trace File System...
[ 11.053730] systemd[1]: Starting kmod-static-nodes.service - Create List of Static Device Nodes...
[ 11.066015] systemd[1]: Starting modprobe@configfs.service - Load Kernel Module configfs...
[ 11.069414] systemd[1]: Starting modprobe@dm_mod.service - Load Kernel Module dm_mod...
[ 11.076547] systemd[1]: Starting modprobe@drm.service - Load Kernel Module drm...
[ 11.090852] systemd[1]: Starting modprobe@efi_pstore.service - Load Kernel Module efi_pstore...
[ 11.097815] systemd[1]: Starting modprobe@fuse.service - Load Kernel Module fuse...
[ 11.109388] systemd[1]: Starting modprobe@loop.service - Load Kernel Module loop...
[ 11.109456] systemd[1]: systemd-firstboot.service - First Boot Wizard was skipped because of an unmet condition check (ConditionFirstBoot=yes).
[ 11.109525] systemd[1]: systemd-fsck-root.service - File System Check on Root Device was skipped because of an unmet condition check (ConditionPathIsReadWrite=!/).
[ 11.109562] systemd[1]: Reached target local-fs.target - Local File Systems.
[ 11.109629] systemd[1]: apparmor.service - Load AppArmor profiles was skipped because of an unmet condition check (ConditionSecurity=apparmor).
[ 11.120810] loop: module loaded
[ 11.122003] systemd[1]: Starting systemd-binfmt.service - Set Up Additional Binary Formats...
[ 11.129598] fuse: init (API version 7.38)
[ 11.137426] systemd[1]: Starting systemd-journald.service - Journal Service...
[ 11.145752] systemd[1]: Starting systemd-random-seed.service - Load/Save Random Seed...
[ 11.157494] systemd[1]: Starting systemd-sysctl.service - Apply Kernel Variables...
[ 11.172634] systemd[1]: Starting systemd-sysusers.service - Create System Users...
[ 11.197939] systemd[1]: Starting systemd-udev-trigger.service - Coldplug All udev Devices...
[ 11.216247] systemd[1]: Mounted dev-hugepages.mount - Huge Pages File System.
[ 11.216451] systemd[1]: Mounted dev-mqueue.mount - POSIX Message Queue File System.
[ 11.216611] systemd[1]: Mounted sys-kernel-debug.mount - Kernel Debug File System.
[ 11.216772] systemd[1]: Mounted sys-kernel-tracing.mount - Kernel Trace File System.
[ 11.233481] systemd[1]: Finished kmod-static-nodes.service - Create List of Static Device Nodes.
[ 11.234279] systemd[1]: modprobe@configfs.service: Deactivated successfully.
[ 11.238519] systemd[1]: Finished modprobe@configfs.service - Load Kernel Module configfs.
[ 11.239177] systemd[1]: modprobe@dm_mod.service: Deactivated successfully.
[ 11.247227] systemd[1]: Finished modprobe@dm_mod.service - Load Kernel Module dm_mod.
[ 11.247980] systemd[1]: modprobe@drm.service: Deactivated successfully.
[ 11.253523] systemd[1]: Finished modprobe@drm.service - Load Kernel Module drm.
[ 11.254233] systemd[1]: modprobe@efi_pstore.service: Deactivated successfully.
[ 11.255540] systemd[1]: Finished modprobe@efi_pstore.service - Load Kernel Module efi_pstore.
[ 11.256109] systemd[1]: modprobe@fuse.service: Deactivated successfully.
[ 11.261747] systemd[1]: Finished modprobe@fuse.service - Load Kernel Module fuse.
[ 11.262375] systemd[1]: modprobe@loop.service: Deactivated successfully.
[ 11.265991] systemd[1]: Finished modprobe@loop.service - Load Kernel Module loop.
[ 11.267213] systemd[1]: Finished systemd-sysctl.service - Apply Kernel Variables.
[ 11.268186] systemd[1]: Finished systemd-sysusers.service - Create System Users.
[ 11.268690] systemd[1]: proc-sys-fs-binfmt_misc.automount: Got automount request for /proc/sys/fs/binfmt_misc, triggered by 133 (systemd-binfmt)
[ 11.282799] systemd[1]: Mounting proc-sys-fs-binfmt_misc.mount - Arbitrary Executable File Formats File System...
[ 11.309441] systemd[1]: Mounting sys-fs-fuse-connections.mount - FUSE Control File System...
[ 11.358583] systemd[1]: Mounting sys-kernel-config.mount - Kernel Configuration File System...
[ 11.358687] systemd[1]: systemd-pstore.service - Platform Persistent Storage Archival was skipped because of an unmet condition check (ConditionDirectoryNotEmpty=/sys/fs/pstore).
[ 11.358816] systemd[1]: systemd-repart.service - Repartition Root Disk was skipped because no trigger condition checks were met.
[ 11.377438] systemd[1]: Starting systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev...
[ 11.378820] systemd[1]: Mounted proc-sys-fs-binfmt_misc.mount - Arbitrary Executable File Formats File System.
[ 11.385019] tsc: Refined TSC clocksource calibration: 3900.223 MHz
[ 11.385026] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x70705a6472c, max_idle_ns: 881590586812 ns
[ 11.385038] clocksource: Switched to clocksource tsc
[ 11.397465] systemd[1]: Finished systemd-binfmt.service - Set Up Additional Binary Formats.
[ 11.397822] systemd[1]: Mounted sys-fs-fuse-connections.mount - FUSE Control File System.
[ 11.397985] systemd[1]: Mounted sys-kernel-config.mount - Kernel Configuration File System.
[ 11.448710] systemd[1]: Finished systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev.
[ 11.457254] systemd[1]: Starting systemd-udevd.service - Rule-based Manager for Device Events and Files...
[ 11.509082] systemd[1]: Started systemd-journald.service - Journal Service.
[ 11.569863] systemd-journald[134]: Received client request to flush runtime journal.
[ 12.053221] sd 6:0:0:0: Attached scsi generic sg0 type 0
[ 12.140997] random: crng init done
[ 12.367954] acpi_cpufreq: overriding BIOS provided _PSD data
[ 12.510747] QUIRK: Enable AMD PLL fix
[ 12.510804] ehci-pci 0000:00:12.2: EHCI Host Controller
[ 12.510833] ehci-pci 0000:00:12.2: new USB bus registered, assigned bus number 1
[ 12.510844] ehci-pci 0000:00:12.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
[ 12.510853] ehci-pci 0000:00:12.2: debug port 1
[ 12.511023] ehci-pci 0000:00:12.2: irq 17, io mem 0xf01cd000
[ 12.515767] piix4_smbus 0000:00:14.0: SMBus Host Controller at 0xb00, revision 0
[ 12.515773] piix4_smbus 0000:00:14.0: Using register 0x2e for SMBus port selection
[ 12.516241] piix4_smbus 0000:00:14.0: Auxiliary SMBus Host Controller at 0xb20
[ 12.525014] ehci-pci 0000:00:12.2: USB 2.0 started, EHCI 1.00
[ 12.525397] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 12.525400] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.525402] usb usb1: Product: EHCI Host Controller
[ 12.525404] usb usb1: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ehci_hcd
[ 12.525405] usb usb1: SerialNumber: 0000:00:12.2
[ 12.525862] hub 1-0:1.0: USB hub found
[ 12.525889] hub 1-0:1.0: 5 ports detected
[ 12.526592] ehci-pci 0000:00:13.2: EHCI Host Controller
[ 12.526612] ehci-pci 0000:00:13.2: new USB bus registered, assigned bus number 2
[ 12.526623] ehci-pci 0000:00:13.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
[ 12.526632] ehci-pci 0000:00:13.2: debug port 1
[ 12.526767] ehci-pci 0000:00:13.2: irq 17, io mem 0xf01ce000
[ 12.541014] ehci-pci 0000:00:13.2: USB 2.0 started, EHCI 1.00
[ 12.541263] usb usb2: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 12.541266] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.541268] usb usb2: Product: EHCI Host Controller
[ 12.541270] usb usb2: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ehci_hcd
[ 12.541271] usb usb2: SerialNumber: 0000:00:13.2
[ 12.541718] hub 2-0:1.0: USB hub found
[ 12.541746] hub 2-0:1.0: 5 ports detected
[ 12.542418] ehci-pci 0000:00:16.2: EHCI Host Controller
[ 12.542436] ehci-pci 0000:00:16.2: new USB bus registered, assigned bus number 3
[ 12.542447] ehci-pci 0000:00:16.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
[ 12.542456] ehci-pci 0000:00:16.2: debug port 1
[ 12.542584] ehci-pci 0000:00:16.2: irq 17, io mem 0xf01cf000
[ 12.557023] ehci-pci 0000:00:16.2: USB 2.0 started, EHCI 1.00
[ 12.557405] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 12.557408] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.557410] usb usb3: Product: EHCI Host Controller
[ 12.557411] usb usb3: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ehci_hcd
[ 12.557413] usb usb3: SerialNumber: 0000:00:16.2
[ 12.557858] hub 3-0:1.0: USB hub found
[ 12.557884] hub 3-0:1.0: 4 ports detected
[ 12.558447] ohci-pci 0000:00:12.0: OHCI PCI host controller
[ 12.558466] ohci-pci 0000:00:12.0: new USB bus registered, assigned bus number 4
[ 12.558608] ohci-pci 0000:00:12.0: irq 18, io mem 0xf01c8000
[ 12.558618] ohci-pci 0000:00:13.0: OHCI PCI host controller
[ 12.558635] ohci-pci 0000:00:13.0: new USB bus registered, assigned bus number 5
[ 12.558703] ohci-pci 0000:00:13.0: irq 18, io mem 0xf01c9000
[ 12.558710] ohci-pci 0000:00:14.5: OHCI PCI host controller
[ 12.558725] ohci-pci 0000:00:14.5: new USB bus registered, assigned bus number 6
[ 12.558793] ohci-pci 0000:00:14.5: irq 18, io mem 0xf01ca000
[ 12.558800] ohci-pci 0000:00:16.0: OHCI PCI host controller
[ 12.558814] ohci-pci 0000:00:16.0: new USB bus registered, assigned bus number 7
[ 12.558888] ohci-pci 0000:00:16.0: irq 18, io mem 0xf01cb000
[ 12.635709] usb usb7: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 12.635716] usb usb7: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.635718] usb usb7: Product: OHCI PCI host controller
[ 12.635720] usb usb7: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ohci_hcd
[ 12.635721] usb usb7: SerialNumber: 0000:00:16.0
[ 12.636145] hub 7-0:1.0: USB hub found
[ 12.636172] hub 7-0:1.0: 4 ports detected
[ 12.636849] usb usb5: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 12.636851] usb usb5: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.636853] usb usb5: Product: OHCI PCI host controller
[ 12.636854] usb usb5: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ohci_hcd
[ 12.636856] usb usb5: SerialNumber: 0000:00:13.0
[ 12.639549] hub 5-0:1.0: USB hub found
[ 12.639579] hub 5-0:1.0: 5 ports detected
[ 12.640465] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 12.640468] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.640470] usb usb4: Product: OHCI PCI host controller
[ 12.640471] usb usb4: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ohci_hcd
[ 12.640473] usb usb4: SerialNumber: 0000:00:12.0
[ 12.640848] hub 4-0:1.0: USB hub found
[ 12.640874] hub 4-0:1.0: 5 ports detected
[ 12.644079] usb usb6: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 12.644084] usb usb6: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.644086] usb usb6: Product: OHCI PCI host controller
[ 12.644088] usb usb6: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ohci_hcd
[ 12.644089] usb usb6: SerialNumber: 0000:00:14.5
[ 12.644484] hub 6-0:1.0: USB hub found
[ 12.644509] hub 6-0:1.0: 2 ports detected
[ 12.658738] r8169 0000:04:00.0: enabling device (0000 -> 0003)
[ 12.697540] xhci_hcd 0000:03:00.0: xHCI Host Controller
[ 12.697566] xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 8
[ 12.709878] r8169 0000:04:00.0 eth0: RTL8168f/8111f, 08:60:6e:74:7a:51, XID 480, IRQ 28
[ 12.709886] r8169 0000:04:00.0 eth0: jumbo features [frames: 9194 bytes, tx checksumming: ko]
[ 12.735756] 1
[ 12.735806] 2
[ 12.736276] snd_hda_intel 0000:00:01.1: Force to non-snoop mode
[ 12.736282] 3
[ 12.736283] 4
[ 12.736284] 5
[ 12.736284] 7
[ 12.736288] 8
[ 12.736288] 9
[ 12.740797] 1
[ 12.740846] 2
[ 12.741376] 3
[ 12.741377] 4
[ 12.741378] 5
[ 12.741379] 7
[ 12.741382] 8
[ 12.741383] 9
[ 12.769260] input: HDA ATI HDMI HDMI/DP,pcm=3 as /devices/pci0000:00/0000:00:01.1/sound/card0/input1
[ 12.769555] input: HDA ATI HDMI HDMI/DP,pcm=7 as /devices/pci0000:00/0000:00:01.1/sound/card0/input2
[ 12.784439] snd_hda_codec_realtek hdaudioC1D0: ALC892: SKU not ready 0x00000100
[ 12.785682] snd_hda_codec_realtek hdaudioC1D0: autoconfig for ALC892: line_outs=4 (0x14/0x16/0x15/0x17/0x0) type:line
[ 12.785688] snd_hda_codec_realtek hdaudioC1D0: speaker_outs=0 (0x0/0x0/0x0/0x0/0x0)
[ 12.785691] snd_hda_codec_realtek hdaudioC1D0: hp_outs=1 (0x1b/0x0/0x0/0x0/0x0)
[ 12.785693] snd_hda_codec_realtek hdaudioC1D0: mono: mono_out=0x0
[ 12.785694] snd_hda_codec_realtek hdaudioC1D0: dig-out=0x1e/0x0
[ 12.785696] snd_hda_codec_realtek hdaudioC1D0: inputs:
[ 12.785697] snd_hda_codec_realtek hdaudioC1D0: Rear Mic=0x18
[ 12.785699] snd_hda_codec_realtek hdaudioC1D0: Front Mic=0x19
[ 12.785700] snd_hda_codec_realtek hdaudioC1D0: Line=0x1a
[ 12.785701] snd_hda_codec_realtek hdaudioC1D0: CD=0x1c
[ 12.807528] xhci_hcd 0000:03:00.0: hcc params 0x0200f180 hci version 0x96 quirks 0x0000000000080010
[ 12.813178] input: HD-Audio Generic Rear Mic as /devices/pci0000:00/0000:00:14.2/sound/card1/input3
[ 12.813514] input: HD-Audio Generic Front Mic as /devices/pci0000:00/0000:00:14.2/sound/card1/input4
[ 12.813782] input: HD-Audio Generic Line as /devices/pci0000:00/0000:00:14.2/sound/card1/input5
[ 12.814048] input: HD-Audio Generic Line Out Front as /devices/pci0000:00/0000:00:14.2/sound/card1/input6
[ 12.814317] input: HD-Audio Generic Line Out Surround as /devices/pci0000:00/0000:00:14.2/sound/card1/input7
[ 12.814575] input: HD-Audio Generic Line Out CLFE as /devices/pci0000:00/0000:00:14.2/sound/card1/input8
[ 12.814833] input: HD-Audio Generic Line Out Side as /devices/pci0000:00/0000:00:14.2/sound/card1/input9
[ 12.815087] input: HD-Audio Generic Front Headphone as /devices/pci0000:00/0000:00:14.2/sound/card1/input10
[ 12.816233] xhci_hcd 0000:03:00.0: xHCI Host Controller
[ 12.816249] xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 9
[ 12.816260] xhci_hcd 0000:03:00.0: Host supports USB 3.0 SuperSpeed
[ 12.830985] usb usb8: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 12.830993] usb usb8: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.830995] usb usb8: Product: xHCI Host Controller
[ 12.830996] usb usb8: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 xhci-hcd
[ 12.830998] usb usb8: SerialNumber: 0000:03:00.0
[ 12.835543] hub 8-0:1.0: USB hub found
[ 12.842296] hub 8-0:1.0: 2 ports detected
[ 12.850842] usb usb9: We don't know the algorithms for LPM for this host, disabling LPM.
[ 12.850983] usb usb9: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 6.03
[ 12.850986] usb usb9: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.850988] usb usb9: Product: xHCI Host Controller
[ 12.850989] usb usb9: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 xhci-hcd
[ 12.850991] usb usb9: SerialNumber: 0000:03:00.0
[ 12.856278] r8169 0000:04:00.0 enp4s0: renamed from eth0
[ 12.862771] hub 9-0:1.0: USB hub found
[ 12.873601] hub 9-0:1.0: 2 ports detected
[ 13.025026] usb 4-1: new low-speed USB device number 2 using ohci-pci
[ 13.103377] r8169 0000:04:00.0: Direct firmware load for rtl_nic/rtl8168f-1.fw failed with error -2
[ 13.103389] r8169 0000:04:00.0: Unable to load firmware rtl_nic/rtl8168f-1.fw (-2)
[ 13.103907] RTL8211E Gigabit Ethernet r8169-0-400:00: attached PHY driver (mii_bus:phy_addr=r8169-0-400:00, irq=MAC)
[ 13.175505] r8169 0000:04:00.0 enp4s0: Link is Down
[ 13.179870] [drm] radeon kernel modesetting enabled.
[ 13.181586] [drm] initializing kernel modesetting (ARUBA 0x1002:0x9996 0x1002:0x9996 0x00).
[ 13.181652] ATOM BIOS: 113
[ 13.181757] radeon 0000:00:01.0: VRAM: 512M 0x0000000000000000 - 0x000000001FFFFFFF (512M used)
[ 13.181761] radeon 0000:00:01.0: GTT: 1024M 0x0000000020000000 - 0x000000005FFFFFFF
[ 13.181769] [drm] Detected VRAM RAM=512M, BAR=256M
[ 13.181770] [drm] RAM width 64bits DDR
[ 13.181947] [drm] radeon: 512M of VRAM memory ready
[ 13.181952] [drm] radeon: 1024M of GTT memory ready.
[ 13.181994] [drm] Loading ARUBA Microcode
[ 13.190350] [drm] Internal thermal controller without fan control
[ 13.191100] [drm] radeon: dpm initialized
[ 13.196015] [drm] Found VCE firmware/feedback version 50.0.1 / 17!
[ 13.196073] [drm] GART: num cpu pages 262144, num gpu pages 262144
[ 13.234337] [drm] GART: Restore entries: num cpu pages 262144, num gpu pages 262144
[ 13.237934] [drm] GART: Done restoring entries
[ 13.237938] [drm] PCIE GART of 1024M enabled (table at 0x00000000001D6000).
[ 13.238177] radeon 0000:00:01.0: WB enabled
[ 13.238180] radeon 0000:00:01.0: fence driver on ring 0 use gpu addr 0x0000000020000c00
[ 13.238558] radeon 0000:00:01.0: fence driver on ring 5 use gpu addr 0x0000000000075a18
[ 13.246045] usb 4-1: New USB device found, idVendor=413c, idProduct=2106, bcdDevice= 1.01
[ 13.246050] usb 4-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 13.246052] usb 4-1: Product: Dell QuietKey Keyboard
[ 13.246054] usb 4-1: Manufacturer: DELL
[ 13.253819] input: DELL Dell QuietKey Keyboard as /devices/pci0000:00/0000:00:12.0/usb4/4-1/4-1:1.0/0003:413C:2106.0001/input/input11
[ 13.260936] radeon 0000:00:01.0: fence driver on ring 6 use gpu addr 0x0000000020000c18
[ 13.260940] radeon 0000:00:01.0: fence driver on ring 7 use gpu addr 0x0000000020000c1c
[ 13.260942] radeon 0000:00:01.0: fence driver on ring 1 use gpu addr 0x0000000020000c04
[ 13.260944] radeon 0000:00:01.0: fence driver on ring 2 use gpu addr 0x0000000020000c08
[ 13.260945] radeon 0000:00:01.0: fence driver on ring 3 use gpu addr 0x0000000020000c0c
[ 13.260947] radeon 0000:00:01.0: fence driver on ring 4 use gpu addr 0x0000000020000c10
[ 13.262964] radeon 0000:00:01.0: radeon: MSI limited to 32-bit
[ 13.263151] radeon 0000:00:01.0: radeon: using MSI.
[ 13.263222] [drm] radeon: irq initialized.
[ 13.281648] [drm] ring test on 0 succeeded in 3 usecs
[ 13.281658] [drm] ring test on 3 succeeded in 4 usecs
[ 13.281665] [drm] ring test on 4 succeeded in 4 usecs
[ 13.295659] [drm] ring test on 5 succeeded in 2 usecs
[ 13.297656] [drm] UVD initialized successfully.
[ 13.313663] hid-generic 0003:413C:2106.0001: input,hidraw0: USB HID v1.10 Keyboard [DELL Dell QuietKey Keyboard] on usb-0000:00:12.0-1/input0
[ 13.446953] [drm] ring test on 6 succeeded in 18 usecs
[ 13.446967] [drm] ring test on 7 succeeded in 3 usecs
[ 13.446968] [drm] VCE initialized successfully.
[ 13.447122] snd_hda_intel 0000:00:01.1: bound 0000:00:01.0 (ops radeon_audio_component_bind_ops [radeon])
[ 13.447293] [drm] ib test on ring 0 succeeded in 0 usecs
[ 13.447346] [drm] ib test on ring 3 succeeded in 0 usecs
[ 13.447396] [drm] ib test on ring 4 succeeded in 0 usecs
[ 13.465099] [drm] ib test on ring 5 succeeded
[ 13.481132] [drm] ib test on ring 6 succeeded in 1 usecs
[ 13.497085] [drm] ib test on ring 7 succeeded in 1 usecs
[ 13.500056] [drm] Radeon Display Connectors
[ 13.500060] [drm] Connector 0:
[ 13.500061] [drm] DP-1
[ 13.500062] [drm] HPD1
[ 13.500062] [drm] DDC: 0x6530 0x6530 0x6534 0x6534 0x6538 0x6538 0x653c 0x653c
[ 13.500065] [drm] Encoders:
[ 13.500065] [drm] DFP1: INTERNAL_UNIPHY2
[ 13.500066] [drm] Connector 1:
[ 13.500067] [drm] VGA-1
[ 13.500068] [drm] HPD2
[ 13.500069] [drm] DDC: 0x6540 0x6540 0x6544 0x6544 0x6548 0x6548 0x654c 0x654c
[ 13.500070] [drm] Encoders:
[ 13.500071] [drm] CRT1: INTERNAL_UNIPHY2
[ 13.500072] [drm] CRT1: NUTMEG
[ 13.500072] [drm] Connector 2:
[ 13.500073] [drm] HDMI-A-1
[ 13.500074] [drm] HPD3
[ 13.500075] [drm] DDC: 0x6550 0x6550 0x6554 0x6554 0x6558 0x6558 0x655c 0x655c
[ 13.500076] [drm] Encoders:
[ 13.500077] [drm] DFP2: INTERNAL_UNIPHY
[ 13.741081] usb 4-2: new low-speed USB device number 3 using ohci-pci
[ 13.779818] [drm] fb mappable at 0xE03E9000
[ 13.779826] [drm] vram apper at 0xE0000000
[ 13.779828] [drm] size 5242880
[ 13.779830] [drm] fb depth is 24
[ 13.779832] [drm] pitch is 5120
[ 13.780398] fbcon: radeondrmfb (fb0) is primary device
[ 13.936158] usb 4-2: New USB device found, idVendor=046d, idProduct=c016, bcdDevice= 3.40
[ 13.936167] usb 4-2: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 13.936170] usb 4-2: Product: Optical USB Mouse
[ 13.936173] usb 4-2: Manufacturer: Logitech
[ 13.945345] input: Logitech Optical USB Mouse as /devices/pci0000:00/0000:00:12.0/usb4/4-2/4-2:1.0/0003:046D:C016.0002/input/input12
[ 13.946178] hid-generic 0003:046D:C016.0002: input,hidraw1: USB HID v1.10 Mouse [Logitech Optical USB Mouse] on usb-0000:00:12.0-2/input0
[ 13.968191] Console: switching to colour frame buffer device 160x64
[ 13.973460] radeon 0000:00:01.0: [drm] fb0: radeondrmfb frame buffer device
[ 13.981574] [drm] Initialized radeon 2.50.0 20080528 for 0000:00:01.0 on minor 0
[ 15.876613] r8169 0000:04:00.0 enp4s0: Link is Up - 1Gbps/Full - flow control rx/tx
[ 15.876628] IPv6: ADDRCONF(NETDEV_CHANGE): enp4s0: link becomes ready
[ 16.713266] [drm] amdgpu kernel modesetting enabled.
[ 17.110008] memfd_create() without MFD_EXEC nor MFD_NOEXEC_SEAL, pid=249 'systemd'
[-- Attachment #3: Type: text/plain, Size: 176 bytes --]
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
2023-04-17 11:19 ` Paul Menzel
(?)
@ 2023-04-17 11:24 ` Paul Menzel
-1 siblings, 0 replies; 236+ messages in thread
From: Paul Menzel @ 2023-04-17 11:24 UTC (permalink / raw)
To: Thomas Gleixner
Cc: linux-kernel, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
[Correct David’s address]
Am 17.04.23 um 13:19 schrieb Paul Menzel:
> Dear Thomas,
>
>
> Am 15.04.23 um 01:44 schrieb Thomas Gleixner:
>
>> This is a complete rework of the parallel bringup patch series (V17)
>>
>>
>> https://lore.kernel.org/lkml/20230328195758.1049469-1-usama.arif@bytedance.com
>>
>> to address the issues which were discovered in review:
>
> […]
>
> Thank you very much for your rework.
>
> I tested this on the ASUS F2A85-M PRO, and get a delay of ten seconds.
>
> ```
> […]
> [ 0.258193] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD
> Graphics (family: 0x15, model: 0x13, stepping: 0x1)
> […]
> [ 0.259329] smp: Bringing up secondary CPUs ...
> [ 0.259527] x86: Booting SMP configuration:
> [ 0.259528] .... node #0, CPUs: #1
> [ 0.261007] After schedule_preempt_disabled
> [ 10.260990] CPU1 failed to report alive state
> [ 10.261070] smp: Brought up 1 node, 1 CPU
> [ 10.261073] smpboot: Max logical packages: 2
> [ 10.261074] smpboot: Total of 1 processors activated (7800.54 BogoMIPS)
> [ 10.261601] devtmpfs: initialized
> [ 10.261697] x86/mm: Memory block size: 128MB
> ```
>
> This delay has been there with v6.3-rc6-46-gde4664485abbc and some
> custom (printk) patches on top and merging dwmw2/parallel-6.2-rc3-v16
> into it. I only tested this. I think dwmw2/parallel-6.2-v17 failed to
> build for me, when trying to merge it into Linus’ master version at that
> time. I didn’t come around to report it, and you posted your rework, so
> I am replying here.
>
> I am going to try your branch directly in the next days, but just wanted
> to report back already.
>
>
> Kind regards,
>
> Paul
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-17 11:24 ` Paul Menzel
0 siblings, 0 replies; 236+ messages in thread
From: Paul Menzel @ 2023-04-17 11:24 UTC (permalink / raw)
To: Thomas Gleixner
Cc: linux-kernel, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
[Correct David’s address]
Am 17.04.23 um 13:19 schrieb Paul Menzel:
> Dear Thomas,
>
>
> Am 15.04.23 um 01:44 schrieb Thomas Gleixner:
>
>> This is a complete rework of the parallel bringup patch series (V17)
>>
>>
>> https://lore.kernel.org/lkml/20230328195758.1049469-1-usama.arif@bytedance.com
>>
>> to address the issues which were discovered in review:
>
> […]
>
> Thank you very much for your rework.
>
> I tested this on the ASUS F2A85-M PRO, and get a delay of ten seconds.
>
> ```
> […]
> [ 0.258193] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD
> Graphics (family: 0x15, model: 0x13, stepping: 0x1)
> […]
> [ 0.259329] smp: Bringing up secondary CPUs ...
> [ 0.259527] x86: Booting SMP configuration:
> [ 0.259528] .... node #0, CPUs: #1
> [ 0.261007] After schedule_preempt_disabled
> [ 10.260990] CPU1 failed to report alive state
> [ 10.261070] smp: Brought up 1 node, 1 CPU
> [ 10.261073] smpboot: Max logical packages: 2
> [ 10.261074] smpboot: Total of 1 processors activated (7800.54 BogoMIPS)
> [ 10.261601] devtmpfs: initialized
> [ 10.261697] x86/mm: Memory block size: 128MB
> ```
>
> This delay has been there with v6.3-rc6-46-gde4664485abbc and some
> custom (printk) patches on top and merging dwmw2/parallel-6.2-rc3-v16
> into it. I only tested this. I think dwmw2/parallel-6.2-v17 failed to
> build for me, when trying to merge it into Linus’ master version at that
> time. I didn’t come around to report it, and you posted your rework, so
> I am replying here.
>
> I am going to try your branch directly in the next days, but just wanted
> to report back already.
>
>
> Kind regards,
>
> Paul
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-17 11:24 ` Paul Menzel
0 siblings, 0 replies; 236+ messages in thread
From: Paul Menzel @ 2023-04-17 11:24 UTC (permalink / raw)
To: Thomas Gleixner
Cc: linux-kernel, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
[Correct David’s address]
Am 17.04.23 um 13:19 schrieb Paul Menzel:
> Dear Thomas,
>
>
> Am 15.04.23 um 01:44 schrieb Thomas Gleixner:
>
>> This is a complete rework of the parallel bringup patch series (V17)
>>
>>
>> https://lore.kernel.org/lkml/20230328195758.1049469-1-usama.arif@bytedance.com
>>
>> to address the issues which were discovered in review:
>
> […]
>
> Thank you very much for your rework.
>
> I tested this on the ASUS F2A85-M PRO, and get a delay of ten seconds.
>
> ```
> […]
> [ 0.258193] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD
> Graphics (family: 0x15, model: 0x13, stepping: 0x1)
> […]
> [ 0.259329] smp: Bringing up secondary CPUs ...
> [ 0.259527] x86: Booting SMP configuration:
> [ 0.259528] .... node #0, CPUs: #1
> [ 0.261007] After schedule_preempt_disabled
> [ 10.260990] CPU1 failed to report alive state
> [ 10.261070] smp: Brought up 1 node, 1 CPU
> [ 10.261073] smpboot: Max logical packages: 2
> [ 10.261074] smpboot: Total of 1 processors activated (7800.54 BogoMIPS)
> [ 10.261601] devtmpfs: initialized
> [ 10.261697] x86/mm: Memory block size: 128MB
> ```
>
> This delay has been there with v6.3-rc6-46-gde4664485abbc and some
> custom (printk) patches on top and merging dwmw2/parallel-6.2-rc3-v16
> into it. I only tested this. I think dwmw2/parallel-6.2-v17 failed to
> build for me, when trying to merge it into Linus’ master version at that
> time. I didn’t come around to report it, and you posted your rework, so
> I am replying here.
>
> I am going to try your branch directly in the next days, but just wanted
> to report back already.
>
>
> Kind regards,
>
> Paul
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
2023-04-17 11:19 ` Paul Menzel
(?)
@ 2023-04-17 14:48 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-17 14:48 UTC (permalink / raw)
To: Paul Menzel
Cc: linux-kernel, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Paul!
On Mon, Apr 17 2023 at 13:19, Paul Menzel wrote:
> Am 15.04.23 um 01:44 schrieb Thomas Gleixner:
> [ 0.258193] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD
> Graphics (family: 0x15, model: 0x13, stepping: 0x1)
> […]
> [ 0.259329] smp: Bringing up secondary CPUs ...
> [ 0.259527] x86: Booting SMP configuration:
> [ 0.259528] .... node #0, CPUs: #1
> [ 0.261007] After schedule_preempt_disabled
> [ 10.260990] CPU1 failed to report alive state
Weird. CPU1 fails to come up and report that it has reached the
synchronization point.
Does it work when you add cpuhp.parallel=off on the kernel command line?
Thanks,
tglx
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-17 14:48 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-17 14:48 UTC (permalink / raw)
To: Paul Menzel
Cc: linux-kernel, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Paul!
On Mon, Apr 17 2023 at 13:19, Paul Menzel wrote:
> Am 15.04.23 um 01:44 schrieb Thomas Gleixner:
> [ 0.258193] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD
> Graphics (family: 0x15, model: 0x13, stepping: 0x1)
> […]
> [ 0.259329] smp: Bringing up secondary CPUs ...
> [ 0.259527] x86: Booting SMP configuration:
> [ 0.259528] .... node #0, CPUs: #1
> [ 0.261007] After schedule_preempt_disabled
> [ 10.260990] CPU1 failed to report alive state
Weird. CPU1 fails to come up and report that it has reached the
synchronization point.
Does it work when you add cpuhp.parallel=off on the kernel command line?
Thanks,
tglx
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-17 14:48 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-17 14:48 UTC (permalink / raw)
To: Paul Menzel
Cc: linux-kernel, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Paul!
On Mon, Apr 17 2023 at 13:19, Paul Menzel wrote:
> Am 15.04.23 um 01:44 schrieb Thomas Gleixner:
> [ 0.258193] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD
> Graphics (family: 0x15, model: 0x13, stepping: 0x1)
> […]
> [ 0.259329] smp: Bringing up secondary CPUs ...
> [ 0.259527] x86: Booting SMP configuration:
> [ 0.259528] .... node #0, CPUs: #1
> [ 0.261007] After schedule_preempt_disabled
> [ 10.260990] CPU1 failed to report alive state
Weird. CPU1 fails to come up and report that it has reached the
synchronization point.
Does it work when you add cpuhp.parallel=off on the kernel command line?
Thanks,
tglx
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 22/37] arm64: smp: Switch to hotplug core state synchronization
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-17 15:50 ` Mark Rutland
-1 siblings, 0 replies; 236+ messages in thread
From: Mark Rutland @ 2023-04-17 15:50 UTC (permalink / raw)
To: Thomas Gleixner
Cc: LKML, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Catalin Marinas, Will Deacon,
linux-arm-kernel, David Woodhouse, Usama Arif, Juergen Gross,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Sabin Rapan
On Sat, Apr 15, 2023 at 01:44:49AM +0200, Thomas Gleixner wrote:
> Switch to the CPU hotplug core state tracking and synchronization
> mechanim. No functional change intended.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will@kernel.org>
> Cc: linux-arm-kernel@lists.infradead.org
I gave this a spin on arm64 (in a 64-vCPU VM on an M1 host), and it seems to
work fine with a bunch of vCPUs being hotplugged off and on again randomly.
FWIW:
Tested-by: Mark Rutland <mark.rutland@arm.com>
I also hacked the code to have the dying CPU spin forever before the call to
cpuhp_ap_report_dead(). In that case I see a warning, and that we don't call
arch_cpuhp_cleanup_dead_cpu(), and that the CPU is marked as offline (per
/sys/devices/system/cpu/$N/online).
As a tangent/aside, we might need to improve that for confidential compute
architectures, and we might want to generically track cpus which might still be
using kernel text/data. On arm64 we ensure that via our cpu_kill() callback
(which'll use PSCI CPU_AFFINITY_INFO), but I'm not sure if TDX and/or SEV-SNP
have a similar mechanism.
Otherwise, a malicious hypervisor can pause a vCPU just before it leaves the
kernel (e.g. immediately after the arch_cpuhp_cleanup_dead_cpu() call), wait
for a kexec (or resuse of stack memroy), and unpause the vCPU to cause things
to blow up.
Thanks,
Mark.
> ---
> arch/arm64/Kconfig | 1 +
> arch/arm64/include/asm/smp.h | 2 +-
> arch/arm64/kernel/smp.c | 14 +++++---------
> 3 files changed, 7 insertions(+), 10 deletions(-)
>
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -216,6 +216,7 @@ config ARM64
> select HAVE_KPROBES
> select HAVE_KRETPROBES
> select HAVE_GENERIC_VDSO
> + select HOTPLUG_CORE_SYNC_DEAD if HOTPLUG_CPU
> select IRQ_DOMAIN
> select IRQ_FORCED_THREADING
> select KASAN_VMALLOC if KASAN
> --- a/arch/arm64/include/asm/smp.h
> +++ b/arch/arm64/include/asm/smp.h
> @@ -99,7 +99,7 @@ static inline void arch_send_wakeup_ipi_
>
> extern int __cpu_disable(void);
>
> -extern void __cpu_die(unsigned int cpu);
> +static inline void __cpu_die(unsigned int cpu) { }
> extern void cpu_die(void);
> extern void cpu_die_early(void);
>
> --- a/arch/arm64/kernel/smp.c
> +++ b/arch/arm64/kernel/smp.c
> @@ -333,17 +333,13 @@ static int op_cpu_kill(unsigned int cpu)
> }
>
> /*
> - * called on the thread which is asking for a CPU to be shutdown -
> - * waits until shutdown has completed, or it is timed out.
> + * Called on the thread which is asking for a CPU to be shutdown after the
> + * shutdown completed.
> */
> -void __cpu_die(unsigned int cpu)
> +void arch_cpuhp_cleanup_dead_cpu(unsigned int cpu)
> {
> int err;
>
> - if (!cpu_wait_death(cpu, 5)) {
> - pr_crit("CPU%u: cpu didn't die\n", cpu);
> - return;
> - }
> pr_debug("CPU%u: shutdown\n", cpu);
>
> /*
> @@ -370,8 +366,8 @@ void cpu_die(void)
>
> local_daif_mask();
>
> - /* Tell __cpu_die() that this CPU is now safe to dispose of */
> - (void)cpu_report_death();
> + /* Tell cpuhp_bp_sync_dead() that this CPU is now safe to dispose of */
> + cpuhp_ap_report_dead();
>
> /*
> * Actually shutdown the CPU. This must never fail. The specific hotplug
>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 22/37] arm64: smp: Switch to hotplug core state synchronization
@ 2023-04-17 15:50 ` Mark Rutland
0 siblings, 0 replies; 236+ messages in thread
From: Mark Rutland @ 2023-04-17 15:50 UTC (permalink / raw)
To: Thomas Gleixner
Cc: LKML, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Catalin Marinas, Will Deacon,
linux-arm-kernel, David Woodhouse, Usama Arif, Juergen Gross,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Sabin Rapan
On Sat, Apr 15, 2023 at 01:44:49AM +0200, Thomas Gleixner wrote:
> Switch to the CPU hotplug core state tracking and synchronization
> mechanim. No functional change intended.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will@kernel.org>
> Cc: linux-arm-kernel@lists.infradead.org
I gave this a spin on arm64 (in a 64-vCPU VM on an M1 host), and it seems to
work fine with a bunch of vCPUs being hotplugged off and on again randomly.
FWIW:
Tested-by: Mark Rutland <mark.rutland@arm.com>
I also hacked the code to have the dying CPU spin forever before the call to
cpuhp_ap_report_dead(). In that case I see a warning, and that we don't call
arch_cpuhp_cleanup_dead_cpu(), and that the CPU is marked as offline (per
/sys/devices/system/cpu/$N/online).
As a tangent/aside, we might need to improve that for confidential compute
architectures, and we might want to generically track cpus which might still be
using kernel text/data. On arm64 we ensure that via our cpu_kill() callback
(which'll use PSCI CPU_AFFINITY_INFO), but I'm not sure if TDX and/or SEV-SNP
have a similar mechanism.
Otherwise, a malicious hypervisor can pause a vCPU just before it leaves the
kernel (e.g. immediately after the arch_cpuhp_cleanup_dead_cpu() call), wait
for a kexec (or resuse of stack memroy), and unpause the vCPU to cause things
to blow up.
Thanks,
Mark.
> ---
> arch/arm64/Kconfig | 1 +
> arch/arm64/include/asm/smp.h | 2 +-
> arch/arm64/kernel/smp.c | 14 +++++---------
> 3 files changed, 7 insertions(+), 10 deletions(-)
>
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -216,6 +216,7 @@ config ARM64
> select HAVE_KPROBES
> select HAVE_KRETPROBES
> select HAVE_GENERIC_VDSO
> + select HOTPLUG_CORE_SYNC_DEAD if HOTPLUG_CPU
> select IRQ_DOMAIN
> select IRQ_FORCED_THREADING
> select KASAN_VMALLOC if KASAN
> --- a/arch/arm64/include/asm/smp.h
> +++ b/arch/arm64/include/asm/smp.h
> @@ -99,7 +99,7 @@ static inline void arch_send_wakeup_ipi_
>
> extern int __cpu_disable(void);
>
> -extern void __cpu_die(unsigned int cpu);
> +static inline void __cpu_die(unsigned int cpu) { }
> extern void cpu_die(void);
> extern void cpu_die_early(void);
>
> --- a/arch/arm64/kernel/smp.c
> +++ b/arch/arm64/kernel/smp.c
> @@ -333,17 +333,13 @@ static int op_cpu_kill(unsigned int cpu)
> }
>
> /*
> - * called on the thread which is asking for a CPU to be shutdown -
> - * waits until shutdown has completed, or it is timed out.
> + * Called on the thread which is asking for a CPU to be shutdown after the
> + * shutdown completed.
> */
> -void __cpu_die(unsigned int cpu)
> +void arch_cpuhp_cleanup_dead_cpu(unsigned int cpu)
> {
> int err;
>
> - if (!cpu_wait_death(cpu, 5)) {
> - pr_crit("CPU%u: cpu didn't die\n", cpu);
> - return;
> - }
> pr_debug("CPU%u: shutdown\n", cpu);
>
> /*
> @@ -370,8 +366,8 @@ void cpu_die(void)
>
> local_daif_mask();
>
> - /* Tell __cpu_die() that this CPU is now safe to dispose of */
> - (void)cpu_report_death();
> + /* Tell cpuhp_bp_sync_dead() that this CPU is now safe to dispose of */
> + cpuhp_ap_report_dead();
>
> /*
> * Actually shutdown the CPU. This must never fail. The specific hotplug
>
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 22/37] arm64: smp: Switch to hotplug core state synchronization
@ 2023-04-17 15:50 ` Mark Rutland
0 siblings, 0 replies; 236+ messages in thread
From: Mark Rutland @ 2023-04-17 15:50 UTC (permalink / raw)
To: Thomas Gleixner
Cc: LKML, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Catalin Marinas, Will Deacon,
linux-arm-kernel, David Woodhouse, Usama Arif, Juergen Gross,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Sabin Rapan
On Sat, Apr 15, 2023 at 01:44:49AM +0200, Thomas Gleixner wrote:
> Switch to the CPU hotplug core state tracking and synchronization
> mechanim. No functional change intended.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will@kernel.org>
> Cc: linux-arm-kernel@lists.infradead.org
I gave this a spin on arm64 (in a 64-vCPU VM on an M1 host), and it seems to
work fine with a bunch of vCPUs being hotplugged off and on again randomly.
FWIW:
Tested-by: Mark Rutland <mark.rutland@arm.com>
I also hacked the code to have the dying CPU spin forever before the call to
cpuhp_ap_report_dead(). In that case I see a warning, and that we don't call
arch_cpuhp_cleanup_dead_cpu(), and that the CPU is marked as offline (per
/sys/devices/system/cpu/$N/online).
As a tangent/aside, we might need to improve that for confidential compute
architectures, and we might want to generically track cpus which might still be
using kernel text/data. On arm64 we ensure that via our cpu_kill() callback
(which'll use PSCI CPU_AFFINITY_INFO), but I'm not sure if TDX and/or SEV-SNP
have a similar mechanism.
Otherwise, a malicious hypervisor can pause a vCPU just before it leaves the
kernel (e.g. immediately after the arch_cpuhp_cleanup_dead_cpu() call), wait
for a kexec (or resuse of stack memroy), and unpause the vCPU to cause things
to blow up.
Thanks,
Mark.
> ---
> arch/arm64/Kconfig | 1 +
> arch/arm64/include/asm/smp.h | 2 +-
> arch/arm64/kernel/smp.c | 14 +++++---------
> 3 files changed, 7 insertions(+), 10 deletions(-)
>
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -216,6 +216,7 @@ config ARM64
> select HAVE_KPROBES
> select HAVE_KRETPROBES
> select HAVE_GENERIC_VDSO
> + select HOTPLUG_CORE_SYNC_DEAD if HOTPLUG_CPU
> select IRQ_DOMAIN
> select IRQ_FORCED_THREADING
> select KASAN_VMALLOC if KASAN
> --- a/arch/arm64/include/asm/smp.h
> +++ b/arch/arm64/include/asm/smp.h
> @@ -99,7 +99,7 @@ static inline void arch_send_wakeup_ipi_
>
> extern int __cpu_disable(void);
>
> -extern void __cpu_die(unsigned int cpu);
> +static inline void __cpu_die(unsigned int cpu) { }
> extern void cpu_die(void);
> extern void cpu_die_early(void);
>
> --- a/arch/arm64/kernel/smp.c
> +++ b/arch/arm64/kernel/smp.c
> @@ -333,17 +333,13 @@ static int op_cpu_kill(unsigned int cpu)
> }
>
> /*
> - * called on the thread which is asking for a CPU to be shutdown -
> - * waits until shutdown has completed, or it is timed out.
> + * Called on the thread which is asking for a CPU to be shutdown after the
> + * shutdown completed.
> */
> -void __cpu_die(unsigned int cpu)
> +void arch_cpuhp_cleanup_dead_cpu(unsigned int cpu)
> {
> int err;
>
> - if (!cpu_wait_death(cpu, 5)) {
> - pr_crit("CPU%u: cpu didn't die\n", cpu);
> - return;
> - }
> pr_debug("CPU%u: shutdown\n", cpu);
>
> /*
> @@ -370,8 +366,8 @@ void cpu_die(void)
>
> local_daif_mask();
>
> - /* Tell __cpu_die() that this CPU is now safe to dispose of */
> - (void)cpu_report_death();
> + /* Tell cpuhp_bp_sync_dead() that this CPU is now safe to dispose of */
> + cpuhp_ap_report_dead();
>
> /*
> * Actually shutdown the CPU. This must never fail. The specific hotplug
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
2023-04-17 14:48 ` Thomas Gleixner
(?)
@ 2023-04-17 17:40 ` Paul Menzel
-1 siblings, 0 replies; 236+ messages in thread
From: Paul Menzel @ 2023-04-17 17:40 UTC (permalink / raw)
To: Thomas Gleixner
Cc: linux-kernel, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
[-- Attachment #1: Type: text/plain, Size: 1062 bytes --]
Dear Thomas,
Am 17.04.23 um 16:48 schrieb Thomas Gleixner:
> On Mon, Apr 17 2023 at 13:19, Paul Menzel wrote:
>> Am 15.04.23 um 01:44 schrieb Thomas Gleixner:
>> [ 0.258193] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD
>> Graphics (family: 0x15, model: 0x13, stepping: 0x1)
>> […]
>> [ 0.259329] smp: Bringing up secondary CPUs ...
>> [ 0.259527] x86: Booting SMP configuration:
>> [ 0.259528] .... node #0, CPUs: #1
>> [ 0.261007] After schedule_preempt_disabled
>> [ 10.260990] CPU1 failed to report alive state
>
> Weird. CPU1 fails to come up and report that it has reached the
> synchronization point.
>
> Does it work when you add cpuhp.parallel=off on the kernel command line?
Yes, the ten seconds delay is gone with `cpuhp.parallel=off`.
There was a patch set in the past, that worked on that device. I think
up to v4 it did *not* work at all and hung [1]. I need some days to
collect the results again.
Kind regards,
Paul
[1]:
https://lore.kernel.org/lkml/ab28d2ce-4a9c-387d-9eda-558045a0c35b@molgen.mpg.de/
[-- Attachment #2: kodi-linux-6.3-rc6-smp-tglx-cpuhp.paralleloff.txt --]
[-- Type: text/plain, Size: 61806 bytes --]
[ 0.000000] Linux version 6.3.0-rc6-00311-gde8224969f66 (root@bf16f3646a84) (gcc (Debian 11.2.0-12) 11.2.0, GNU ld (GNU Binutils for Debian) 2.40) #446 SMP PREEMPT_DYNAMIC Sat Apr 15 14:12:29 UTC 2023
[ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc6-00311-gde8224969f66 root=/dev/sda3 rw quiet noisapnp cryptomgr.notests ipv6.disable_ipv6=1 selinux=0 cpuhp.parallel=off
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256
[ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format.
[ 0.000000] signal: max sigframe size: 1776
[ 0.000000] BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
[ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005fe4cfff] usable
[ 0.000000] BIOS-e820: [mem 0x000000005fe4d000-0x000000007fffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000f8000000-0x00000000fbffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000fec10000-0x00000000fec10fff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000017effffff] usable
[ 0.000000] NX (Execute Disable) protection: active
[ 0.000000] SMBIOS 3.0.0 present.
[ 0.000000] DMI: ASUS F2A85-M_PRO/F2A85-M_PRO, BIOS 4.18-9-gb640ed51b2 04/17/2023
[ 0.000000] tsc: Fast TSC calibration using PIT
[ 0.000000] tsc: Initial usec timer 9249065
[ 0.000000] tsc: Detected 3899.954 MHz processor
[ 0.000755] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
[ 0.000759] e820: remove [mem 0x000a0000-0x000fffff] usable
[ 0.000763] last_pfn = 0x17f000 max_arch_pfn = 0x400000000
[ 0.000768] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT
[ 0.000938] last_pfn = 0x5fe4d max_arch_pfn = 0x400000000
[ 0.004000] Using GB pages for direct mapping
[ 0.004000] ACPI: Early table checksum verification disabled
[ 0.004000] ACPI: RSDP 0x00000000000F6830 000024 (v02 COREv4)
[ 0.004000] ACPI: XSDT 0x000000005FE5A0E0 000074 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: FACP 0x000000005FE5BBC0 000114 (v06 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: DSDT 0x000000005FE5A280 00193A (v02 COREv4 COREBOOT 00010001 INTL 20200925)
[ 0.004000] ACPI: FACS 0x000000005FE5A240 000040
[ 0.004000] ACPI: FACS 0x000000005FE5A240 000040
[ 0.004000] ACPI: SSDT 0x000000005FE5BCE0 00008A (v02 COREv4 COREBOOT 0000002A CORE 20200925)
[ 0.004000] ACPI: MCFG 0x000000005FE5BD70 00003C (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: APIC 0x000000005FE5BDB0 000062 (v03 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: HPET 0x000000005FE5BE20 000038 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: HEST 0x000000005FE5BE60 0001D0 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: IVRS 0x000000005FE5C030 000070 (v02 AMD AMDIOMMU 00000001 AMD 00000000)
[ 0.004000] ACPI: SSDT 0x000000005FE5C0A0 00051F (v02 AMD ALIB 00000001 MSFT 04000000)
[ 0.004000] ACPI: SSDT 0x000000005FE5C5C0 0006B2 (v01 AMD POWERNOW 00000001 AMD 00000001)
[ 0.004000] ACPI: VFCT 0x000000005FE5CC80 00F269 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: Reserving FACP table memory at [mem 0x5fe5bbc0-0x5fe5bcd3]
[ 0.004000] ACPI: Reserving DSDT table memory at [mem 0x5fe5a280-0x5fe5bbb9]
[ 0.004000] ACPI: Reserving FACS table memory at [mem 0x5fe5a240-0x5fe5a27f]
[ 0.004000] ACPI: Reserving FACS table memory at [mem 0x5fe5a240-0x5fe5a27f]
[ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe5bce0-0x5fe5bd69]
[ 0.004000] ACPI: Reserving MCFG table memory at [mem 0x5fe5bd70-0x5fe5bdab]
[ 0.004000] ACPI: Reserving APIC table memory at [mem 0x5fe5bdb0-0x5fe5be11]
[ 0.004000] ACPI: Reserving HPET table memory at [mem 0x5fe5be20-0x5fe5be57]
[ 0.004000] ACPI: Reserving HEST table memory at [mem 0x5fe5be60-0x5fe5c02f]
[ 0.004000] ACPI: Reserving IVRS table memory at [mem 0x5fe5c030-0x5fe5c09f]
[ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe5c0a0-0x5fe5c5be]
[ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe5c5c0-0x5fe5cc71]
[ 0.004000] ACPI: Reserving VFCT table memory at [mem 0x5fe5cc80-0x5fe6bee8]
[ 0.004000] No NUMA configuration found
[ 0.004000] Faking a node at [mem 0x0000000000000000-0x000000017effffff]
[ 0.004000] NODE_DATA(0) allocated [mem 0x17efe9000-0x17effffff]
[ 0.004000] Zone ranges:
[ 0.004000] DMA [mem 0x0000000000001000-0x0000000000ffffff]
[ 0.004000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]
[ 0.004000] Normal [mem 0x0000000100000000-0x000000017effffff]
[ 0.004000] Device empty
[ 0.004000] Movable zone start for each node
[ 0.004000] Early memory node ranges
[ 0.004000] node 0: [mem 0x0000000000001000-0x000000000009efff]
[ 0.004000] node 0: [mem 0x0000000000100000-0x000000005fe4cfff]
[ 0.004000] node 0: [mem 0x0000000100000000-0x000000017effffff]
[ 0.004000] Initmem setup node 0 [mem 0x0000000000001000-0x000000017effffff]
[ 0.004000] On node 0, zone DMA: 1 pages in unavailable ranges
[ 0.004000] On node 0, zone DMA: 97 pages in unavailable ranges
[ 0.004000] On node 0, zone Normal: 435 pages in unavailable ranges
[ 0.004000] On node 0, zone Normal: 4096 pages in unavailable ranges
[ 0.004000] ACPI: PM-Timer IO Port: 0x818
[ 0.004000] ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1])
[ 0.004000] IOAPIC[0]: apic_id 4, version 33, address 0xfec00000, GSI 0-23
[ 0.004000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[ 0.004000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 low level)
[ 0.004000] ACPI: Using ACPI (MADT) for SMP configuration information
[ 0.004000] ACPI: HPET id: 0x10228210 base: 0xfed00000
[ 0.004000] smpboot: Allowing 2 CPUs, 0 hotplug CPUs
[ 0.004000] smpboot: smpboot: XXX end of prefill_possible_map
[ 0.004000] After prefill_possible_map
[ 0.004000] After init_cpu_to_node
[ 0.004000] After init_gi_nodes
[ 0.004000] After io_apic_init_mappings
[ 0.004000] After x86_init.hyper.guest_late_init
[ 0.004000] [mem 0x80000000-0xf7ffffff] available for PCI devices
[ 0.004000] After e820
[ 0.004000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns
[ 0.004000] After unwind_init
[ 0.004000] After setup_arch
[ 0.004000] After setup_command_line
[ 0.004000] After setup_nr_cpu_ids
[ 0.004000] setup_percpu: NR_CPUS:64 nr_cpumask_bits:2 nr_cpu_ids:2 nr_node_ids:1
[ 0.004000] percpu: Embedded 55 pages/cpu s188328 r8192 d28760 u1048576
[ 0.004000] pcpu-alloc: s188328 r8192 d28760 u1048576 alloc=1*2097152
[ 0.004000] pcpu-alloc: [0] 0 1
[ 0.004000] After setup_per_cpu_areas
[ 0.004000] After smp_perpare_boot_cpu
[ 0.004000] After boot_cpu_hotplug_init
[ 0.004000] Fallback order for Node 0: 0
[ 0.004000] Built 1 zonelists, mobility grouping on. Total pages: 898451
[ 0.004000] Policy zone: Normal
[ 0.004000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc6-00311-gde8224969f66 root=/dev/sda3 rw quiet noisapnp cryptomgr.notests ipv6.disable_ipv6=1 selinux=0 cpuhp.parallel=off
[ 0.004000] Unknown kernel command line parameters "noisapnp BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc6-00311-gde8224969f66", will be passed to user space.
[ 0.004000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[ 0.004000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[ 0.004000] mem auto-init: stack:off, heap alloc:off, heap free:off
[ 0.004000] stackdepot: allocating hash table via alloc_large_system_hash
[ 0.004000] stackdepot hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[ 0.004000] software IO TLB: area num 2.
[ 0.004000] Memory: 3477168K/3651500K available (14336K kernel code, 2340K rwdata, 5308K rodata, 2908K init, 11060K bss, 174072K reserved, 0K cma-reserved)
[ 0.004000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
[ 0.004000] After mm_init
[ 0.004000] After poking_init
[ 0.004000] ftrace: allocating 38664 entries in 152 pages
[ 0.004000] ftrace: allocated 152 pages with 3 groups
[ 0.004000] Dynamic Preempt: full
[ 0.004000] After sched_init
[ 0.004000] rcu: Preemptible hierarchical RCU implementation.
[ 0.004000] rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=2.
[ 0.004000] Trampoline variant of Tasks RCU enabled.
[ 0.004000] Rude variant of Tasks RCU enabled.
[ 0.004000] Tracing variant of Tasks RCU enabled.
[ 0.004000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[ 0.004000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2
[ 0.004000] After rcu_init
[ 0.004000] NR_IRQS: 4352, nr_irqs: 440, preallocated irqs: 16
[ 0.004000] rcu: srcu_init: Setting srcu_struct sizes based on contention.
[ 0.004000] After random_init()
[ 0.004000] After boot_init_stack_canary
[ 0.004000] spurious 8259A interrupt: IRQ7.
[ 0.004000] Console: colour VGA+ 80x25
[ 0.004000] printk: console [tty0] enabled
[ 0.004000] ACPI: Core revision 20221020
[ 0.004000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 133484873504 ns
[ 0.004000] APIC: Switch to symmetric I/O mode setup
[ 0.004000] AMD-Vi: Using global IVHD EFR:0x0, EFR2:0x0
[ 0.004000] APIC: Done
[ 0.004000] Before apic_bsb_setup
[ 0.004000] check_timer begin
[ 0.004000] check_timer after local_irq_disable
[ 0.004000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[ 0.004000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x706e603bb55, max_idle_ns: 881590819133 ns
[ 0.145156] Calibrating delay loop (skipped), value calculated using timer frequency.. 7799.90 BogoMIPS (lpj=15599816)
[ 0.145160] pid_max: default: 32768 minimum: 301
[ 0.145254] LSM: initializing lsm=capability
[ 0.145348] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[ 0.145365] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[ 0.145738] Bit 30 in CPUID ECX not set.
[ 0.145762] Last level iTLB entries: 4KB 512, 2MB 1024, 4MB 512
[ 0.145764] Last level dTLB entries: 4KB 1024, 2MB 1024, 4MB 512, 1GB 0
[ 0.145769] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization
[ 0.145772] Spectre V2 : Mitigation: Retpolines
[ 0.145773] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch
[ 0.145774] Spectre V2 : Spectre v2 / SpectreRSB : Filling RSB on VMEXIT
[ 0.145774] Spectre V2 : Enabling Speculation Barrier for firmware calls
[ 0.145775] RETBleed: Mitigation: untrained return thunk
[ 0.145777] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier
[ 0.145779] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl
[ 0.150244] Freeing SMP alternatives memory: 32K
[ 0.150249] After check_bugs
[ 0.150250] After acpi_subsystem_init
[ 0.150251] After arch_post_acpi_subsys_init
[ 0.150252] After rcu_scheduler_starting
[ 0.150324] After find_task_by_pid_ns and PF_NO_SETAFFINITY
[ 0.150329] After numa_default_policy
[ 0.150349] After rcu_read_lock
[ 0.150350] After rcu_read_unlock
[ 0.150351] After kthreadd_done
[ 0.150363] smpboot: Start of smp_prepare_cpus_common
[ 0.150365] smpboot: smpboot: zalloc 0
[ 0.150366] smpboot: smpboot: zalloc 1
[ 0.150367] smpboot: smpboot: After set_sched_topology()
[ 0.150369] smpboot: smpboot: After smp_sanity_check()
[ 0.150369] smpboot: smpboot: Before x86_init.timers.setup_percpu_clockev()
[ 0.258381] smpboot: smpboot: After x86_init.timers.setup_percpu_clockev()
[ 0.258382] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD Graphics (family: 0x15, model: 0x13, stepping: 0x1)
[ 0.258615] cblist_init_generic: Setting adjustable number of callback queues.
[ 0.258618] cblist_init_generic: Setting shift to 1 and lim to 1.
[ 0.258648] cblist_init_generic: Setting shift to 1 and lim to 1.
[ 0.258675] cblist_init_generic: Setting shift to 1 and lim to 1.
[ 0.258703] Performance Events: Fam15h core perfctr, AMD PMU driver.
[ 0.258726] ... version: 0
[ 0.258727] ... bit width: 48
[ 0.258728] ... generic registers: 6
[ 0.258729] ... value mask: 0000ffffffffffff
[ 0.258730] ... max period: 00007fffffffffff
[ 0.258731] ... fixed-purpose events: 0
[ 0.258732] ... event mask: 000000000000003f
[ 0.258852] rcu: Hierarchical SRCU implementation.
[ 0.258853] rcu: Max phase no-delay instances is 1000.
[ 0.259441] NMI watchdog: Enabled. Permanently consumes one hw-PMU counter.
[ 0.259515] smp: Bringing up secondary CPUs ...
[ 0.259714] x86: Booting SMP configuration:
[ 0.259715] .... node #0, CPUs: #1
[ 0.121151] Bit 30 in CPUID ECX not set.
[ 0.259844] After schedule_preempt_disabled
[ 0.259849] smp: Brought up 1 node, 2 CPUs
[ 0.259849] smpboot: Max logical packages: 1
[ 0.259849] smpboot: Total of 2 processors activated (15599.81 BogoMIPS)
[ 0.261311] devtmpfs: initialized
[ 0.261311] x86/mm: Memory block size: 128MB
[ 0.262220] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[ 0.262220] futex hash table entries: 512 (order: 3, 32768 bytes, linear)
[ 0.262220] pinctrl core: initialized pinctrl subsystem
[ 0.262220] PM: RTC time: 17:24:09, date: 2023-04-17
[ 0.262220] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[ 0.262434] audit: initializing netlink subsys (disabled)
[ 0.262453] audit: type=2000 audit(1681752249.140:1): state=initialized audit_enabled=0 res=1
[ 0.262453] thermal_sys: Registered thermal governor 'fair_share'
[ 0.262453] thermal_sys: Registered thermal governor 'bang_bang'
[ 0.262453] thermal_sys: Registered thermal governor 'step_wise'
[ 0.262453] thermal_sys: Registered thermal governor 'user_space'
[ 0.262453] cpuidle: using governor ladder
[ 0.262453] cpuidle: using governor menu
[ 0.262453] PCI: MMCONFIG for domain 0000 [bus 00-3f] at [mem 0xf8000000-0xfbffffff] (base 0xf8000000)
[ 0.262453] PCI: MMCONFIG at [mem 0xf8000000-0xfbffffff] reserved as E820 entry
[ 0.262453] PCI: Using configuration type 1 for base access
[ 0.262453] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.
[ 0.273235] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages
[ 0.273235] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page
[ 0.273235] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages
[ 0.273235] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page
[ 0.275428] cryptd: max_cpu_qlen set to 1000
[ 0.275428] ACPI: Added _OSI(Module Device)
[ 0.275428] ACPI: Added _OSI(Processor Device)
[ 0.275428] ACPI: Added _OSI(3.0 _SCP Extensions)
[ 0.275428] ACPI: Added _OSI(Processor Aggregator Device)
[ 0.280879] ACPI: DSDT successfully acquired and loaded
[ 0.281154] ACPI: 4 ACPI AML tables successfully acquired and loaded
[ 0.281649] ACPI: Interpreter enabled
[ 0.281669] ACPI: PM: (supports S0 S1 S3 S5)
[ 0.281671] ACPI: Using IOAPIC for interrupt routing
[ 0.281724] HEST: Table parsing has been initialized.
[ 0.281743] GHES: Failed to enable APEI firmware first mode.
[ 0.281745] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[ 0.281746] PCI: Ignoring E820 reservations for host bridge windows
[ 0.282003] ACPI: Enabled 8 GPEs in block 00 to 1F
[ 0.286735] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[ 0.286746] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]
[ 0.286838] acpi PNP0A03:00: _OSC: OS now controls [PME AER PCIeCapability LTR]
[ 0.286852] acpi PNP0A03:00: [Firmware Info]: MMCONFIG for domain 0000 [bus 00-3f] only partially covers this bridge
[ 0.286942] acpi PNP0A03:00: host bridge window expanded to [io 0x0000-0x0cf7 window]; [io 0x03b0-0x03df window] ignored
[ 0.287197] PCI host bridge to bus 0000:00
[ 0.287199] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]
[ 0.287201] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]
[ 0.287203] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000dffff]
[ 0.287205] pci_bus 0000:00: root bus resource [mem 0x80000000-0xffffffff]
[ 0.287210] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 0.287234] pci 0000:00:00.0: [1022:1410] type 00 class 0x060000
[ 0.287388] pci 0000:00:00.2: [1022:1419] type 00 class 0x080600
[ 0.287479] pci 0000:00:01.0: [1002:9996] type 00 class 0x030000
[ 0.287488] pci 0000:00:01.0: reg 0x10: [mem 0xe0000000-0xefffffff pref]
[ 0.287493] pci 0000:00:01.0: reg 0x14: [io 0x1000-0x10ff]
[ 0.287498] pci 0000:00:01.0: reg 0x18: [mem 0xf0180000-0xf01bffff]
[ 0.287514] pci 0000:00:01.0: enabling Extended Tags
[ 0.287524] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]
[ 0.287541] pci 0000:00:01.0: supports D1 D2
[ 0.287611] pci 0000:00:01.1: [1002:9902] type 00 class 0x040300
[ 0.287618] pci 0000:00:01.1: reg 0x10: [mem 0xf01c0000-0xf01c3fff]
[ 0.287640] pci 0000:00:01.1: enabling Extended Tags
[ 0.287663] pci 0000:00:01.1: supports D1 D2
[ 0.287750] pci 0000:00:11.0: [1022:7801] type 00 class 0x010601
[ 0.287763] pci 0000:00:11.0: reg 0x10: [io 0x1410-0x1417]
[ 0.287771] pci 0000:00:11.0: reg 0x14: [io 0x1420-0x1423]
[ 0.287778] pci 0000:00:11.0: reg 0x18: [io 0x1418-0x141f]
[ 0.287785] pci 0000:00:11.0: reg 0x1c: [io 0x1424-0x1427]
[ 0.287793] pci 0000:00:11.0: reg 0x20: [io 0x1400-0x140f]
[ 0.287800] pci 0000:00:11.0: reg 0x24: [mem 0xf01cc000-0xf01cc7ff]
[ 0.287954] pci 0000:00:12.0: [1022:7807] type 00 class 0x0c0310
[ 0.287967] pci 0000:00:12.0: reg 0x10: [mem 0xf01c8000-0xf01c8fff]
[ 0.288152] pci 0000:00:12.2: [1022:7808] type 00 class 0x0c0320
[ 0.288165] pci 0000:00:12.2: reg 0x10: [mem 0xf01cd000-0xf01cd0ff]
[ 0.288230] pci 0000:00:12.2: supports D1 D2
[ 0.288231] pci 0000:00:12.2: PME# supported from D0 D1 D2 D3hot
[ 0.288233] pci 0000:00:12.2: pme_poll = true
[ 0.288234] pci 0000:00:12.2: after device_set_wakeup_capable()
[ 0.288237] pci 0000:00:12.2: after pci_pme_active()
[ 0.288373] pci 0000:00:13.0: [1022:7807] type 00 class 0x0c0310
[ 0.288387] pci 0000:00:13.0: reg 0x10: [mem 0xf01c9000-0xf01c9fff]
[ 0.288570] pci 0000:00:13.2: [1022:7808] type 00 class 0x0c0320
[ 0.288583] pci 0000:00:13.2: reg 0x10: [mem 0xf01ce000-0xf01ce0ff]
[ 0.288650] pci 0000:00:13.2: supports D1 D2
[ 0.288652] pci 0000:00:13.2: PME# supported from D0 D1 D2 D3hot
[ 0.288653] pci 0000:00:13.2: pme_poll = true
[ 0.288654] pci 0000:00:13.2: after device_set_wakeup_capable()
[ 0.288657] pci 0000:00:13.2: after pci_pme_active()
[ 0.288798] pci 0000:00:14.0: [1022:780b] type 00 class 0x0c0500
[ 0.288980] pci 0000:00:14.2: [1022:780d] type 00 class 0x040300
[ 0.288997] pci 0000:00:14.2: reg 0x10: [mem 0xf01c4000-0xf01c7fff 64bit]
[ 0.289052] pci 0000:00:14.2: PME# supported from D0 D3hot D3cold
[ 0.289054] pci 0000:00:14.2: pme_poll = true
[ 0.289055] pci 0000:00:14.2: after device_set_wakeup_capable()
[ 0.289058] pci 0000:00:14.2: after pci_pme_active()
[ 0.290131] pci 0000:00:14.3: [1022:780e] type 00 class 0x060100
[ 0.290335] pci 0000:00:14.4: [1022:780f] type 01 class 0x060401
[ 0.290490] pci 0000:00:14.5: [1022:7809] type 00 class 0x0c0310
[ 0.290504] pci 0000:00:14.5: reg 0x10: [mem 0xf01ca000-0xf01cafff]
[ 0.290677] pci 0000:00:15.0: [1022:43a0] type 01 class 0x060400
[ 0.290705] pci 0000:00:15.0: enabling Extended Tags
[ 0.290746] pci 0000:00:15.0: supports D1 D2
[ 0.290907] pci 0000:00:15.1: [1022:43a1] type 01 class 0x060400
[ 0.290939] pci 0000:00:15.1: enabling Extended Tags
[ 0.290978] pci 0000:00:15.1: supports D1 D2
[ 0.291146] pci 0000:00:15.2: [1022:43a2] type 01 class 0x060400
[ 0.291174] pci 0000:00:15.2: enabling Extended Tags
[ 0.291214] pci 0000:00:15.2: supports D1 D2
[ 0.291289] pci 0000:00:16.0: [1022:7807] type 00 class 0x0c0310
[ 0.291303] pci 0000:00:16.0: reg 0x10: [mem 0xf01cb000-0xf01cbfff]
[ 0.291473] pci 0000:00:16.2: [1022:7808] type 00 class 0x0c0320
[ 0.291486] pci 0000:00:16.2: reg 0x10: [mem 0xf01cf000-0xf01cf0ff]
[ 0.291551] pci 0000:00:16.2: supports D1 D2
[ 0.291552] pci 0000:00:16.2: PME# supported from D0 D1 D2 D3hot
[ 0.291554] pci 0000:00:16.2: pme_poll = true
[ 0.291555] pci 0000:00:16.2: after device_set_wakeup_capable()
[ 0.291558] pci 0000:00:16.2: after pci_pme_active()
[ 0.291684] pci 0000:00:18.0: [1022:1400] type 00 class 0x060000
[ 0.291748] pci 0000:00:18.1: [1022:1401] type 00 class 0x060000
[ 0.291806] pci 0000:00:18.2: [1022:1402] type 00 class 0x060000
[ 0.291866] pci 0000:00:18.3: [1022:1403] type 00 class 0x060000
[ 0.291998] pci 0000:00:18.4: [1022:1404] type 00 class 0x060000
[ 0.292064] pci 0000:00:18.5: [1022:1405] type 00 class 0x060000
[ 0.292136] pci_bus 0000:01: extended config space not accessible
[ 0.292200] pci 0000:00:14.4: PCI bridge to [bus 01] (subtractive decode)
[ 0.292208] pci 0000:00:14.4: bridge window [io 0x0000-0x0cf7 window] (subtractive decode)
[ 0.292211] pci 0000:00:14.4: bridge window [io 0x0d00-0xffff window] (subtractive decode)
[ 0.292214] pci 0000:00:14.4: bridge window [mem 0x000a0000-0x000dffff] (subtractive decode)
[ 0.292216] pci 0000:00:14.4: bridge window [mem 0x80000000-0xffffffff] (subtractive decode)
[ 0.292261] pci 0000:00:15.0: PCI bridge to [bus 02]
[ 0.292345] pci 0000:03:00.0: [1b21:1042] type 00 class 0x0c0330
[ 0.292382] pci 0000:03:00.0: reg 0x10: [mem 0xf0000000-0xf0007fff 64bit]
[ 0.292559] pci 0000:03:00.0: PME# supported from D3hot D3cold
[ 0.292560] pci 0000:03:00.0: pme_poll = true
[ 0.292562] pci 0000:03:00.0: after device_set_wakeup_capable()
[ 0.292567] pci 0000:03:00.0: after pci_pme_active()
[ 0.292605] pci 0000:03:00.0: 2.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s PCIe x1 link at 0000:00:15.1 (capable of 4.000 Gb/s with 5.0 GT/s PCIe x1 link)
[ 0.305215] pci 0000:00:15.1: PCI bridge to [bus 03]
[ 0.305228] pci 0000:00:15.1: bridge window [mem 0xf0000000-0xf00fffff]
[ 0.305237] pci 0000:00:15.2: bridge configuration invalid ([bus 00-00]), reconfiguring
[ 0.305359] pci 0000:04:00.0: [10ec:8168] type 00 class 0x020000
[ 0.305378] pci 0000:04:00.0: reg 0x10: [io 0x0000-0x00ff]
[ 0.305399] pci 0000:04:00.0: reg 0x18: [mem 0x00000000-0x00000fff 64bit pref]
[ 0.305413] pci 0000:04:00.0: reg 0x20: [mem 0x00000000-0x00003fff 64bit pref]
[ 0.305520] pci 0000:04:00.0: supports D1 D2
[ 0.305522] pci 0000:04:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[ 0.305524] pci 0000:04:00.0: pme_poll = true
[ 0.305525] pci 0000:04:00.0: after device_set_wakeup_capable()
[ 0.305529] pci 0000:04:00.0: after pci_pme_active()
[ 0.321217] pci 0000:00:15.2: PCI bridge to [bus 04-ff]
[ 0.321228] pci 0000:00:15.2: bridge window [io 0x0000-0x0fff]
[ 0.321232] pci 0000:00:15.2: bridge window [mem 0x00000000-0x000fffff]
[ 0.321237] pci 0000:00:15.2: bridge window [mem 0x00000000-0x000fffff 64bit pref]
[ 0.321240] pci_bus 0000:04: busn_res: [bus 04-ff] end is updated to 04
[ 0.321759] ACPI: PCI: Interrupt link INTA configured for IRQ 0
[ 0.321855] ACPI: PCI: Interrupt link INTB configured for IRQ 0
[ 0.321949] ACPI: PCI: Interrupt link INTC configured for IRQ 0
[ 0.322041] ACPI: PCI: Interrupt link INTD configured for IRQ 0
[ 0.322134] ACPI: PCI: Interrupt link INTE configured for IRQ 0
[ 0.322232] ACPI: PCI: Interrupt link INTF configured for IRQ 0
[ 0.322325] ACPI: PCI: Interrupt link INTG configured for IRQ 0
[ 0.322418] ACPI: PCI: Interrupt link INTH configured for IRQ 0
[ 0.322640] iommu: Default domain type: Translated
[ 0.322642] iommu: DMA domain TLB invalidation policy: lazy mode
[ 0.322839] SCSI subsystem initialized
[ 0.325159] libata version 3.00 loaded.
[ 0.325159] ACPI: bus type USB registered
[ 0.325159] usbcore: registered new interface driver usbfs
[ 0.325159] usbcore: registered new interface driver hub
[ 0.325159] usbcore: registered new device driver usb
[ 0.325159] PCI: Using ACPI for IRQ routing
[ 0.325159] PCI: pci_cache_line_size set to 64 bytes
[ 0.325159] e820: reserve RAM buffer [mem 0x0009fc00-0x0009ffff]
[ 0.325159] e820: reserve RAM buffer [mem 0x5fe4d000-0x5fffffff]
[ 0.325159] e820: reserve RAM buffer [mem 0x17f000000-0x17fffffff]
[ 0.325159] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0
[ 0.325159] hpet0: 3 comparators, 32-bit 14.318180 MHz counter
[ 0.330232] clocksource: Switched to clocksource tsc-early
[ 0.330474] VFS: Disk quotas dquot_6.6.0
[ 0.330500] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[ 0.330620] pnp: PnP ACPI init
[ 0.330969] system 00:00: [mem 0xfec10002-0xfec11001] could not be reserved
[ 0.331362] pnp: PnP ACPI: found 2 devices
[ 0.337986] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
[ 0.338186] NET: Registered PF_INET protocol family
[ 0.338359] IP idents hash table entries: 65536 (order: 7, 524288 bytes, linear)
[ 0.340153] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear)
[ 0.340169] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
[ 0.340177] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear)
[ 0.340251] TCP bind hash table entries: 32768 (order: 8, 1048576 bytes, linear)
[ 0.340614] TCP: Hash tables configured (established 32768 bind 32768)
[ 0.340685] UDP hash table entries: 2048 (order: 4, 65536 bytes, linear)
[ 0.340711] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear)
[ 0.340827] NET: Registered PF_UNIX/PF_LOCAL protocol family
[ 0.340862] pci 0000:00:15.2: BAR 15: assigned [mem 0x80000000-0x800fffff 64bit pref]
[ 0.340869] pci 0000:00:15.2: BAR 13: assigned [io 0x2000-0x2fff]
[ 0.340874] pci 0000:00:14.4: PCI bridge to [bus 01]
[ 0.340886] pci 0000:00:15.0: PCI bridge to [bus 02]
[ 0.340894] pci 0000:00:15.1: PCI bridge to [bus 03]
[ 0.340898] pci 0000:00:15.1: bridge window [mem 0xf0000000-0xf00fffff]
[ 0.340908] pci 0000:04:00.0: BAR 4: assigned [mem 0x80000000-0x80003fff 64bit pref]
[ 0.340922] pci 0000:04:00.0: BAR 2: assigned [mem 0x80004000-0x80004fff 64bit pref]
[ 0.340934] pci 0000:04:00.0: BAR 0: assigned [io 0x2000-0x20ff]
[ 0.340941] pci 0000:00:15.2: PCI bridge to [bus 04]
[ 0.340943] pci 0000:00:15.2: bridge window [io 0x2000-0x2fff]
[ 0.340949] pci 0000:00:15.2: bridge window [mem 0x80000000-0x800fffff 64bit pref]
[ 0.340956] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]
[ 0.340958] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]
[ 0.340960] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000dffff]
[ 0.340964] pci_bus 0000:00: resource 7 [mem 0x80000000-0xffffffff]
[ 0.340966] pci_bus 0000:01: resource 4 [io 0x0000-0x0cf7 window]
[ 0.340968] pci_bus 0000:01: resource 5 [io 0x0d00-0xffff window]
[ 0.340970] pci_bus 0000:01: resource 6 [mem 0x000a0000-0x000dffff]
[ 0.340973] pci_bus 0000:01: resource 7 [mem 0x80000000-0xffffffff]
[ 0.340975] pci_bus 0000:03: resource 1 [mem 0xf0000000-0xf00fffff]
[ 0.340977] pci_bus 0000:04: resource 0 [io 0x2000-0x2fff]
[ 0.340979] pci_bus 0000:04: resource 2 [mem 0x80000000-0x800fffff 64bit pref]
[ 0.341104] pci 0000:00:01.1: D0 power state depends on 0000:00:01.0
[ 0.341412] pci 0000:00:12.0: AMD USB device
[ 0.341436] pci 0000:00:12.0: AMD USB ohci handoff
[ 0.341833] pci 0000:00:12.2: AMD USB device
[ 0.341847] pci 0000:00:12.2: AMD USB ehci handoff
[ 0.342032] pci 0000:00:12.2: PME# does not work under D3, disabling it
[ 0.342235] pci 0000:00:13.0: AMD USB device
[ 0.342250] pci 0000:00:13.0: AMD USB ohci handoff
[ 0.342633] pci 0000:00:13.2: AMD USB device
[ 0.342650] pci 0000:00:13.2: AMD USB ehci handoff
[ 0.342830] pci 0000:00:13.2: PME# does not work under D3, disabling it
[ 0.343041] pci 0000:00:14.5: AMD USB device
[ 0.343057] pci 0000:00:14.5: AMD USB ohci handoff
[ 0.343577] pci 0000:00:16.0: AMD USB device
[ 0.343598] pci 0000:00:16.0: AMD USB ohci handoff
[ 0.343993] pci 0000:00:16.2: AMD USB device
[ 0.344008] pci 0000:00:16.2: AMD USB ehci handoff
[ 0.344202] pci 0000:00:16.2: PME# does not work under D3, disabling it
[ 0.344510] pci 0000:03:00.0: AMD USB xhci handoff
[ 0.344577] PCI: CLS 64 bytes, default 64
[ 0.344678] pci 0000:00:00.2: AMD-Vi: Applying erratum 746 workaround
[ 0.344790] pci 0000:00:01.0: Adding to iommu group 0
[ 0.344813] pci 0000:00:01.1: Adding to iommu group 0
[ 0.344844] pci 0000:00:11.0: Adding to iommu group 1
[ 0.344885] pci 0000:00:12.0: Adding to iommu group 2
[ 0.344905] pci 0000:00:12.2: Adding to iommu group 2
[ 0.344949] pci 0000:00:13.0: Adding to iommu group 3
[ 0.344970] pci 0000:00:13.2: Adding to iommu group 3
[ 0.345020] pci 0000:00:14.0: Adding to iommu group 4
[ 0.345043] pci 0000:00:14.2: Adding to iommu group 4
[ 0.345066] pci 0000:00:14.3: Adding to iommu group 4
[ 0.345102] pci 0000:00:14.4: Adding to iommu group 5
[ 0.345128] pci 0000:00:14.5: Adding to iommu group 6
[ 0.345172] pci 0000:00:15.0: Adding to iommu group 7
[ 0.345198] pci 0000:00:15.1: Adding to iommu group 7
[ 0.345219] pci 0000:00:15.2: Adding to iommu group 7
[ 0.345261] pci 0000:00:16.0: Adding to iommu group 8
[ 0.345285] pci 0000:00:16.2: Adding to iommu group 8
[ 0.345349] pci 0000:00:18.0: Adding to iommu group 9
[ 0.345374] pci 0000:00:18.1: Adding to iommu group 9
[ 0.345400] pci 0000:00:18.2: Adding to iommu group 9
[ 0.345428] pci 0000:00:18.3: Adding to iommu group 9
[ 0.345451] pci 0000:00:18.4: Adding to iommu group 9
[ 0.345479] pci 0000:00:18.5: Adding to iommu group 9
[ 0.345489] pci 0000:03:00.0: Adding to iommu group 7
[ 0.345499] pci 0000:04:00.0: Adding to iommu group 7
[ 0.348265] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[ 0.348272] AMD-Vi: Extended features (0x800000853, 0x0): PreF PPR GT IA
[ 0.348279] AMD-Vi: Interrupt remapping enabled
[ 0.348484] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[ 0.348486] software IO TLB: mapped [mem 0x000000005be4d000-0x000000005fe4d000] (64MB)
[ 0.348547] LVT offset 0 assigned for vector 0x400
[ 0.348595] perf: AMD IBS detected (0x000000ff)
[ 0.348602] amd_uncore: 4 amd_nb counters detected
[ 0.352225] workingset: timestamp_bits=37 max_order=20 bucket_order=0
[ 0.352259] zbud: loaded
[ 0.352752] NET: Registered PF_ALG protocol family
[ 0.352758] Key type asymmetric registered
[ 0.352759] Asymmetric key parser 'x509' registered
[ 0.353113] alg: self-tests disabled
[ 0.353209] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 251)
[ 0.353263] io scheduler mq-deadline registered
[ 0.353265] io scheduler kyber registered
[ 0.354479] pcieport 0000:00:15.0: PME: Signaling with IRQ 25
[ 0.354644] pcieport 0000:00:15.1: PME: Signaling with IRQ 26
[ 0.354712] pcieport 0000:00:15.2: enabling device (0000 -> 0003)
[ 0.354941] pcieport 0000:00:15.2: PME: Signaling with IRQ 27
[ 0.355211] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
[ 0.355298] ACPI: button: Power Button [PWRF]
[ 0.355363] ACPI: \_SB_.P000: Found 2 idle states
[ 0.355495] ACPI: \_SB_.P001: Found 2 idle states
[ 0.356442] thermal LNXTHERM:00: registered as thermal_zone0
[ 0.356445] ACPI: thermal: Thermal Zone [TZ00] (21 C)
[ 0.356797] Non-volatile memory driver v1.3
[ 0.356864] AMD-Vi: AMD IOMMUv2 loaded and initialized
[ 0.357065] ahci 0000:00:11.0: version 3.0
[ 0.357356] ahci 0000:00:11.0: AHCI 0001.0300 32 slots 8 ports 6 Gbps 0x40 impl SATA mode
[ 0.357360] ahci 0000:00:11.0: flags: 64bit ncq sntf ilck led clo pio
[ 0.358785] scsi host0: ahci
[ 0.358999] scsi host1: ahci
[ 0.359215] scsi host2: ahci
[ 0.359424] scsi host3: ahci
[ 0.359624] scsi host4: ahci
[ 0.359836] scsi host5: ahci
[ 0.360048] scsi host6: ahci
[ 0.360243] scsi host7: ahci
[ 0.360336] ata port1: DUMMY
[ 0.360338] ata port2: DUMMY
[ 0.360339] ata port3: DUMMY
[ 0.360341] ata port4: DUMMY
[ 0.360342] ata port5: DUMMY
[ 0.360343] ata port6: DUMMY
[ 0.360345] ata port7: SATA max UDMA/133 abar m2048@0xf01cc000 port 0xf01cc400 irq 19
[ 0.360347] ata port8: DUMMY
[ 0.360431] ACPI: bus type drm_connector registered
[ 0.360734] i8042: PNP: No PS/2 controller found.
[ 0.360736] i8042: Probing ports directly.
[ 0.363078] serio: i8042 KBD port at 0x60,0x64 irq 1
[ 0.363185] serio: i8042 AUX port at 0x60,0x64 irq 12
[ 0.363347] mousedev: PS/2 mouse device common for all mice
[ 0.363420] rtc_cmos 00:01: RTC can wake from S4
[ 0.363924] rtc_cmos 00:01: registered as rtc0
[ 0.363950] rtc_cmos 00:01: setting system clock to 2023-04-17T17:24:09 UTC (1681752249)
[ 0.364011] rtc_cmos 00:01: alarms up to one day, y3k, 114 bytes nvram, hpet irqs
[ 0.364048] device-mapper: uevent: version 1.0.3
[ 0.364136] device-mapper: ioctl: 4.47.0-ioctl (2022-07-28) initialised: dm-devel@redhat.com
[ 0.364301] hid: raw HID events driver (C) Jiri Kosina
[ 0.364335] usbcore: registered new interface driver usbhid
[ 0.364336] usbhid: USB HID core driver
[ 0.364427] Initializing XFRM netlink socket
[ 0.364436] NET: Registered PF_PACKET protocol family
[ 0.364438] x86/pm: family 0x15 cpu detected, MSR saving is needed during suspending.
[ 0.364652] microcode: CPU1: patch_level=0x0600111f
[ 0.364652] microcode: CPU0: patch_level=0x0600111f
[ 0.364664] microcode: Microcode Update Driver: v2.2.
[ 0.364668] IPI shorthand broadcast: enabled
[ 0.364678] AVX version of gcm_enc/dec engaged.
[ 0.364710] AES CTR mode by8 optimization enabled
[ 0.368578] sched_clock: Marking stable (249972593, 117151517)->(369867515, -2743405)
[ 0.368823] registered taskstats version 1
[ 0.369080] zswap: loaded using pool lzo/zbud
[ 0.371663] ata link7: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[ 0.371939] ata dev7.0: ATA-9: SanDisk SDSSDP064G, 2.0.0, max UDMA/133
[ 0.371942] ata dev7.0: 125045424 sectors, multi 1: LBA48 NCQ (depth 32)
[ 0.372148] ata dev7.0: configured for UDMA/133
[ 0.372283] scsi 6:0:0:0: Direct-Access ATA SanDisk SDSSDP06 0 PQ: 0 ANSI: 5
[ 0.373496] kmemleak: Kernel memory leak detector initialized (mem pool available: 15679)
[ 0.373501] debug_vm_pgtable: [debug_vm_pgtable ]: Validating architecture page table helpers
[ 0.373959] sd 6:0:0:0: [sda] 125045424 512-byte logical blocks: (64.0 GB/59.6 GiB)
[ 0.373982] sd 6:0:0:0: [sda] Write Protect is off
[ 0.373987] sd 6:0:0:0: [sda] Mode Sense: 00 3a 00 00
[ 0.374015] sd 6:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 0.374059] sd 6:0:0:0: [sda] Preferred minimum I/O size 512 bytes
[ 0.380407] kmemleak: Automatic memory scanning thread started
[ 0.380633] sda: sda1 sda2 sda3
[ 0.381120] Key type encrypted registered
[ 0.381248] sd 6:0:0:0: [sda] Attached SCSI disk
[ 0.384294] PM: Magic number: 3:443:441
[ 0.396721] EXT4-fs (sda3): mounted filesystem fe29e0dc-6303-4401-987c-8472bc1b9516 with ordered data mode. Quota mode: none.
[ 0.396766] VFS: Mounted root (ext4 filesystem) on device 8:3.
[ 0.398650] devtmpfs: mounted
[ 0.398668] After kernel_init_freeable
[ 0.403170] Freeing unused kernel image (initmem) memory: 2908K
[ 0.415639] Write protecting the kernel read-only data: 20480k
[ 0.415922] Freeing unused kernel image (rodata/data gap) memory: 836K
[ 0.453024] x86/mm: Checked W+X mappings: passed, no W+X pages found.
[ 0.453030] rodata_test: all tests were successful
[ 0.453031] After mark_readonly
[ 0.453031] After pti_finalize
[ 0.453045] rcu_end_inkernel_boot
[ 0.453054] Run /sbin/init as init process
[ 0.453055] with arguments:
[ 0.453057] /sbin/init
[ 0.453058] noisapnp
[ 0.453059] with environment:
[ 0.453060] HOME=/
[ 0.453060] TERM=linux
[ 0.453061] BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc6-00311-gde8224969f66
[ 0.629518] systemd[1]: Inserted module 'autofs4'
[ 0.656601] NET: Registered PF_INET6 protocol family
[ 0.657495] Segment Routing with IPv6
[ 0.657523] In-situ OAM (IOAM) with IPv6
[ 0.683745] systemd[1]: systemd 252.6-1 running in system mode (+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
[ 0.683756] systemd[1]: Detected architecture x86-64.
[ 0.688485] systemd[1]: Hostname set to <kodi>.
[ 0.959071] systemd[1]: Queued start job for default target graphical.target.
[ 0.982198] systemd[1]: Created slice system-getty.slice - Slice /system/getty.
[ 0.983286] systemd[1]: Created slice system-modprobe.slice - Slice /system/modprobe.
[ 0.984123] systemd[1]: Created slice user.slice - User and Session Slice.
[ 0.984302] systemd[1]: Started systemd-ask-password-console.path - Dispatch Password Requests to Console Directory Watch.
[ 0.984417] systemd[1]: Started systemd-ask-password-wall.path - Forward Password Requests to Wall Directory Watch.
[ 0.984855] systemd[1]: Set up automount proc-sys-fs-binfmt_misc.automount - Arbitrary Executable File Formats File System Automount Point.
[ 0.984894] systemd[1]: Reached target cryptsetup.target - Local Encrypted Volumes.
[ 0.984934] systemd[1]: Reached target integritysetup.target - Local Integrity Protected Volumes.
[ 0.984972] systemd[1]: Reached target paths.target - Path Units.
[ 0.985006] systemd[1]: Reached target remote-fs.target - Remote File Systems.
[ 0.985038] systemd[1]: Reached target slices.target - Slice Units.
[ 0.985078] systemd[1]: Reached target swap.target - Swaps.
[ 0.985116] systemd[1]: Reached target veritysetup.target - Local Verity Protected Volumes.
[ 0.987550] systemd[1]: Listening on systemd-coredump.socket - Process Core Dump Socket.
[ 0.987802] systemd[1]: Listening on systemd-fsckd.socket - fsck to fsckd communication Socket.
[ 0.987975] systemd[1]: Listening on systemd-initctl.socket - initctl Compatibility Named Pipe.
[ 0.988274] systemd[1]: Listening on systemd-journald-audit.socket - Journal Audit Socket.
[ 0.988543] systemd[1]: Listening on systemd-journald-dev-log.socket - Journal Socket (/dev/log).
[ 0.988822] systemd[1]: Listening on systemd-journald.socket - Journal Socket.
[ 0.989072] systemd[1]: Listening on systemd-networkd.socket - Network Service Netlink Socket.
[ 0.989946] systemd[1]: Listening on systemd-udevd-control.socket - udev Control Socket.
[ 0.990210] systemd[1]: Listening on systemd-udevd-kernel.socket - udev Kernel Socket.
[ 0.992904] systemd[1]: Mounting dev-hugepages.mount - Huge Pages File System...
[ 0.995944] systemd[1]: Mounting dev-mqueue.mount - POSIX Message Queue File System...
[ 0.999142] systemd[1]: Mounting sys-kernel-debug.mount - Kernel Debug File System...
[ 1.002682] systemd[1]: Mounting sys-kernel-tracing.mount - Kernel Trace File System...
[ 1.007256] systemd[1]: Starting kmod-static-nodes.service - Create List of Static Device Nodes...
[ 1.014276] systemd[1]: Starting modprobe@configfs.service - Load Kernel Module configfs...
[ 1.017399] systemd[1]: Starting modprobe@dm_mod.service - Load Kernel Module dm_mod...
[ 1.020657] systemd[1]: Starting modprobe@drm.service - Load Kernel Module drm...
[ 1.024006] systemd[1]: Starting modprobe@efi_pstore.service - Load Kernel Module efi_pstore...
[ 1.027278] systemd[1]: Starting modprobe@fuse.service - Load Kernel Module fuse...
[ 1.036066] systemd[1]: Starting modprobe@loop.service - Load Kernel Module loop...
[ 1.036151] systemd[1]: systemd-firstboot.service - First Boot Wizard was skipped because of an unmet condition check (ConditionFirstBoot=yes).
[ 1.036237] systemd[1]: systemd-fsck-root.service - File System Check on Root Device was skipped because of an unmet condition check (ConditionPathIsReadWrite=!/).
[ 1.036279] systemd[1]: Reached target local-fs.target - Local File Systems.
[ 1.036362] systemd[1]: apparmor.service - Load AppArmor profiles was skipped because of an unmet condition check (ConditionSecurity=apparmor).
[ 1.041356] loop: module loaded
[ 1.043749] systemd[1]: Starting systemd-binfmt.service - Set Up Additional Binary Formats...
[ 1.048983] systemd[1]: Starting systemd-journald.service - Journal Service...
[ 1.050740] fuse: init (API version 7.38)
[ 1.052824] systemd[1]: Starting systemd-random-seed.service - Load/Save Random Seed...
[ 1.063924] systemd[1]: Starting systemd-sysctl.service - Apply Kernel Variables...
[ 1.068061] systemd[1]: Starting systemd-sysusers.service - Create System Users...
[ 1.071328] systemd[1]: Starting systemd-udev-trigger.service - Coldplug All udev Devices...
[ 1.093517] systemd[1]: Mounted dev-hugepages.mount - Huge Pages File System.
[ 1.094134] systemd[1]: Mounted dev-mqueue.mount - POSIX Message Queue File System.
[ 1.094447] systemd[1]: Mounted sys-kernel-debug.mount - Kernel Debug File System.
[ 1.094985] systemd[1]: Mounted sys-kernel-tracing.mount - Kernel Trace File System.
[ 1.095974] systemd[1]: Finished kmod-static-nodes.service - Create List of Static Device Nodes.
[ 1.096888] systemd[1]: modprobe@configfs.service: Deactivated successfully.
[ 1.097257] systemd[1]: Finished modprobe@configfs.service - Load Kernel Module configfs.
[ 1.098001] systemd[1]: modprobe@dm_mod.service: Deactivated successfully.
[ 1.098339] systemd[1]: Finished modprobe@dm_mod.service - Load Kernel Module dm_mod.
[ 1.099077] systemd[1]: modprobe@drm.service: Deactivated successfully.
[ 1.099446] systemd[1]: Finished modprobe@drm.service - Load Kernel Module drm.
[ 1.100518] systemd[1]: modprobe@efi_pstore.service: Deactivated successfully.
[ 1.100846] systemd[1]: Finished modprobe@efi_pstore.service - Load Kernel Module efi_pstore.
[ 1.101749] systemd[1]: modprobe@fuse.service: Deactivated successfully.
[ 1.102064] systemd[1]: Finished modprobe@fuse.service - Load Kernel Module fuse.
[ 1.102810] systemd[1]: modprobe@loop.service: Deactivated successfully.
[ 1.103142] systemd[1]: Finished modprobe@loop.service - Load Kernel Module loop.
[ 1.103830] systemd[1]: proc-sys-fs-binfmt_misc.automount: Got automount request for /proc/sys/fs/binfmt_misc, triggered by 138 (systemd-binfmt)
[ 1.123242] systemd[1]: Mounting proc-sys-fs-binfmt_misc.mount - Arbitrary Executable File Formats File System...
[ 1.140028] systemd[1]: Mounting sys-fs-fuse-connections.mount - FUSE Control File System...
[ 1.148223] systemd[1]: Mounting sys-kernel-config.mount - Kernel Configuration File System...
[ 1.148346] systemd[1]: systemd-pstore.service - Platform Persistent Storage Archival was skipped because of an unmet condition check (ConditionDirectoryNotEmpty=/sys/fs/pstore).
[ 1.148515] systemd[1]: systemd-repart.service - Repartition Root Disk was skipped because no trigger condition checks were met.
[ 1.159909] systemd[1]: Finished systemd-sysctl.service - Apply Kernel Variables.
[ 1.184529] systemd[1]: Finished systemd-sysusers.service - Create System Users.
[ 1.186000] systemd[1]: Mounted proc-sys-fs-binfmt_misc.mount - Arbitrary Executable File Formats File System.
[ 1.186357] systemd[1]: Mounted sys-fs-fuse-connections.mount - FUSE Control File System.
[ 1.186600] systemd[1]: Mounted sys-kernel-config.mount - Kernel Configuration File System.
[ 1.209823] systemd[1]: Starting systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev...
[ 1.210512] systemd[1]: Finished systemd-binfmt.service - Set Up Additional Binary Formats.
[ 1.254273] systemd[1]: Finished systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev.
[ 1.273469] systemd[1]: Starting systemd-udevd.service - Rule-based Manager for Device Events and Files...
[ 1.313286] systemd[1]: Started systemd-journald.service - Journal Service.
[ 1.359636] systemd-journald[139]: Received client request to flush runtime journal.
[ 1.369173] tsc: Refined TSC clocksource calibration: 3900.223 MHz
[ 1.369184] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x70705a6472c, max_idle_ns: 881590586812 ns
[ 1.369200] clocksource: Switched to clocksource tsc
[ 1.594559] sd 6:0:0:0: Attached scsi generic sg0 type 0
[ 1.731704] acpi_cpufreq: overriding BIOS provided _PSD data
[ 1.953174] random: crng init done
[ 1.992654] QUIRK: Enable AMD PLL fix
[ 1.992706] ehci-pci 0000:00:12.2: EHCI Host Controller
[ 1.992738] ehci-pci 0000:00:12.2: new USB bus registered, assigned bus number 1
[ 1.992751] ehci-pci 0000:00:12.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
[ 1.992760] ehci-pci 0000:00:12.2: debug port 1
[ 1.992971] ehci-pci 0000:00:12.2: irq 17, io mem 0xf01cd000
[ 2.005197] ehci-pci 0000:00:12.2: USB 2.0 started, EHCI 1.00
[ 2.005468] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 2.005471] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.005474] usb usb1: Product: EHCI Host Controller
[ 2.005476] usb usb1: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ehci_hcd
[ 2.005478] usb usb1: SerialNumber: 0000:00:12.2
[ 2.006279] hub 1-0:1.0: USB hub found
[ 2.006321] hub 1-0:1.0: 5 ports detected
[ 2.007319] ehci-pci 0000:00:13.2: EHCI Host Controller
[ 2.007345] ehci-pci 0000:00:13.2: new USB bus registered, assigned bus number 2
[ 2.007360] ehci-pci 0000:00:13.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
[ 2.007369] ehci-pci 0000:00:13.2: debug port 1
[ 2.007523] ehci-pci 0000:00:13.2: irq 17, io mem 0xf01ce000
[ 2.016141] piix4_smbus 0000:00:14.0: SMBus Host Controller at 0xb00, revision 0
[ 2.016147] piix4_smbus 0000:00:14.0: Using register 0x2e for SMBus port selection
[ 2.016788] piix4_smbus 0000:00:14.0: Auxiliary SMBus Host Controller at 0xb20
[ 2.021184] ehci-pci 0000:00:13.2: USB 2.0 started, EHCI 1.00
[ 2.021523] usb usb2: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 2.021527] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.021529] usb usb2: Product: EHCI Host Controller
[ 2.021531] usb usb2: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ehci_hcd
[ 2.021533] usb usb2: SerialNumber: 0000:00:13.2
[ 2.032086] hub 2-0:1.0: USB hub found
[ 2.032171] hub 2-0:1.0: 5 ports detected
[ 2.033420] ehci-pci 0000:00:16.2: EHCI Host Controller
[ 2.033455] ehci-pci 0000:00:16.2: new USB bus registered, assigned bus number 3
[ 2.033470] ehci-pci 0000:00:16.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
[ 2.033479] ehci-pci 0000:00:16.2: debug port 1
[ 2.033641] ehci-pci 0000:00:16.2: irq 17, io mem 0xf01cf000
[ 2.049177] ehci-pci 0000:00:16.2: USB 2.0 started, EHCI 1.00
[ 2.049454] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 2.049459] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.049461] usb usb3: Product: EHCI Host Controller
[ 2.049463] usb usb3: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ehci_hcd
[ 2.049465] usb usb3: SerialNumber: 0000:00:16.2
[ 2.050025] hub 3-0:1.0: USB hub found
[ 2.050070] hub 3-0:1.0: 4 ports detected
[ 2.050881] ohci-pci 0000:00:16.0: OHCI PCI host controller
[ 2.050910] ohci-pci 0000:00:16.0: new USB bus registered, assigned bus number 4
[ 2.050999] ohci-pci 0000:00:12.0: OHCI PCI host controller
[ 2.051021] ohci-pci 0000:00:12.0: new USB bus registered, assigned bus number 5
[ 2.051110] ohci-pci 0000:00:16.0: irq 18, io mem 0xf01cb000
[ 2.051111] ohci-pci 0000:00:12.0: irq 18, io mem 0xf01c8000
[ 2.051120] ohci-pci 0000:00:13.0: OHCI PCI host controller
[ 2.051141] ohci-pci 0000:00:13.0: new USB bus registered, assigned bus number 6
[ 2.051240] ohci-pci 0000:00:13.0: irq 18, io mem 0xf01c9000
[ 2.051250] ohci-pci 0000:00:14.5: OHCI PCI host controller
[ 2.051268] ohci-pci 0000:00:14.5: new USB bus registered, assigned bus number 7
[ 2.051356] ohci-pci 0000:00:14.5: irq 18, io mem 0xf01ca000
[ 2.113579] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 2.113586] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.113589] usb usb4: Product: OHCI PCI host controller
[ 2.113591] usb usb4: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ohci_hcd
[ 2.113593] usb usb4: SerialNumber: 0000:00:16.0
[ 2.114096] hub 4-0:1.0: USB hub found
[ 2.114127] hub 4-0:1.0: 4 ports detected
[ 2.126194] usb usb5: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 2.126202] usb usb5: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.126204] usb usb5: Product: OHCI PCI host controller
[ 2.126207] usb usb5: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ohci_hcd
[ 2.126209] usb usb5: SerialNumber: 0000:00:12.0
[ 2.127341] hub 5-0:1.0: USB hub found
[ 2.127380] hub 5-0:1.0: 5 ports detected
[ 2.128522] usb usb6: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 2.128528] usb usb6: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.128531] usb usb6: Product: OHCI PCI host controller
[ 2.128533] usb usb6: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ohci_hcd
[ 2.128535] usb usb6: SerialNumber: 0000:00:13.0
[ 2.129049] hub 6-0:1.0: USB hub found
[ 2.129082] hub 6-0:1.0: 5 ports detected
[ 2.130311] usb usb7: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 2.130316] usb usb7: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.130319] usb usb7: Product: OHCI PCI host controller
[ 2.130321] usb usb7: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ohci_hcd
[ 2.130323] usb usb7: SerialNumber: 0000:00:14.5
[ 2.130855] hub 7-0:1.0: USB hub found
[ 2.130895] hub 7-0:1.0: 2 ports detected
[ 2.148137] 1
[ 2.148196] 2
[ 2.148820] snd_hda_intel 0000:00:01.1: Force to non-snoop mode
[ 2.148832] 3
[ 2.148833] 4
[ 2.148834] 5
[ 2.148835] 7
[ 2.148837] 8
[ 2.148837] 9
[ 2.148929] 1
[ 2.148979] 2
[ 2.149637] 3
[ 2.149641] 4
[ 2.149642] 5
[ 2.149643] 7
[ 2.149645] 8
[ 2.149646] 9
[ 2.166674] xhci_hcd 0000:03:00.0: xHCI Host Controller
[ 2.166707] xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 8
[ 2.206853] input: HDA ATI HDMI HDMI/DP,pcm=3 as /devices/pci0000:00/0000:00:01.1/sound/card0/input1
[ 2.208133] input: HDA ATI HDMI HDMI/DP,pcm=7 as /devices/pci0000:00/0000:00:01.1/sound/card0/input2
[ 2.221615] snd_hda_codec_realtek hdaudioC1D0: ALC892: SKU not ready 0x00000100
[ 2.222377] snd_hda_codec_realtek hdaudioC1D0: autoconfig for ALC892: line_outs=4 (0x14/0x16/0x15/0x17/0x0) type:line
[ 2.222384] snd_hda_codec_realtek hdaudioC1D0: speaker_outs=0 (0x0/0x0/0x0/0x0/0x0)
[ 2.222387] snd_hda_codec_realtek hdaudioC1D0: hp_outs=1 (0x1b/0x0/0x0/0x0/0x0)
[ 2.222390] snd_hda_codec_realtek hdaudioC1D0: mono: mono_out=0x0
[ 2.222392] snd_hda_codec_realtek hdaudioC1D0: dig-out=0x1e/0x0
[ 2.222393] snd_hda_codec_realtek hdaudioC1D0: inputs:
[ 2.222395] snd_hda_codec_realtek hdaudioC1D0: Rear Mic=0x18
[ 2.222398] snd_hda_codec_realtek hdaudioC1D0: Front Mic=0x19
[ 2.222399] snd_hda_codec_realtek hdaudioC1D0: Line=0x1a
[ 2.222401] snd_hda_codec_realtek hdaudioC1D0: CD=0x1c
[ 2.231811] xhci_hcd 0000:03:00.0: hcc params 0x0200f180 hci version 0x96 quirks 0x0000000000080010
[ 2.235245] r8169 0000:04:00.0: enabling device (0000 -> 0003)
[ 2.265051] xhci_hcd 0000:03:00.0: xHCI Host Controller
[ 2.265076] xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 9
[ 2.265092] xhci_hcd 0000:03:00.0: Host supports USB 3.0 SuperSpeed
[ 2.266951] usb usb8: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 2.266958] usb usb8: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.266961] usb usb8: Product: xHCI Host Controller
[ 2.266963] usb usb8: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 xhci-hcd
[ 2.266965] usb usb8: SerialNumber: 0000:03:00.0
[ 2.267014] input: HD-Audio Generic Rear Mic as /devices/pci0000:00/0000:00:14.2/sound/card1/input3
[ 2.267354] input: HD-Audio Generic Front Mic as /devices/pci0000:00/0000:00:14.2/sound/card1/input4
[ 2.267750] input: HD-Audio Generic Line as /devices/pci0000:00/0000:00:14.2/sound/card1/input5
[ 2.268089] input: HD-Audio Generic Line Out Front as /devices/pci0000:00/0000:00:14.2/sound/card1/input6
[ 2.268437] input: HD-Audio Generic Line Out Surround as /devices/pci0000:00/0000:00:14.2/sound/card1/input7
[ 2.268783] input: HD-Audio Generic Line Out CLFE as /devices/pci0000:00/0000:00:14.2/sound/card1/input8
[ 2.269132] input: HD-Audio Generic Line Out Side as /devices/pci0000:00/0000:00:14.2/sound/card1/input9
[ 2.269714] hub 8-0:1.0: USB hub found
[ 2.269815] hub 8-0:1.0: 2 ports detected
[ 2.269956] input: HD-Audio Generic Front Headphone as /devices/pci0000:00/0000:00:14.2/sound/card1/input10
[ 2.272826] usb usb9: We don't know the algorithms for LPM for this host, disabling LPM.
[ 2.273006] usb usb9: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 6.03
[ 2.273010] usb usb9: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.273012] usb usb9: Product: xHCI Host Controller
[ 2.273014] usb usb9: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 xhci-hcd
[ 2.273016] usb usb9: SerialNumber: 0000:03:00.0
[ 2.275521] hub 9-0:1.0: USB hub found
[ 2.275649] hub 9-0:1.0: 2 ports detected
[ 2.295318] r8169 0000:04:00.0 eth0: RTL8168f/8111f, 08:60:6e:74:7a:51, XID 480, IRQ 32
[ 2.295327] r8169 0000:04:00.0 eth0: jumbo features [frames: 9194 bytes, tx checksumming: ko]
[ 2.311845] r8169 0000:04:00.0 enp4s0: renamed from eth0
[ 2.421403] r8169 0000:04:00.0: Direct firmware load for rtl_nic/rtl8168f-1.fw failed with error -2
[ 2.421415] r8169 0000:04:00.0: Unable to load firmware rtl_nic/rtl8168f-1.fw (-2)
[ 2.421903] RTL8211E Gigabit Ethernet r8169-0-400:00: attached PHY driver (mii_bus:phy_addr=r8169-0-400:00, irq=MAC)
[ 2.465163] usb 5-1: new low-speed USB device number 2 using ohci-pci
[ 2.472735] [drm] radeon kernel modesetting enabled.
[ 2.474200] [drm] initializing kernel modesetting (ARUBA 0x1002:0x9996 0x1002:0x9996 0x00).
[ 2.474271] ATOM BIOS: 113
[ 2.474372] radeon 0000:00:01.0: VRAM: 512M 0x0000000000000000 - 0x000000001FFFFFFF (512M used)
[ 2.474376] radeon 0000:00:01.0: GTT: 1024M 0x0000000020000000 - 0x000000005FFFFFFF
[ 2.474384] [drm] Detected VRAM RAM=512M, BAR=256M
[ 2.474385] [drm] RAM width 64bits DDR
[ 2.474589] [drm] radeon: 512M of VRAM memory ready
[ 2.474595] [drm] radeon: 1024M of GTT memory ready.
[ 2.474642] [drm] Loading ARUBA Microcode
[ 2.481608] [drm] Internal thermal controller without fan control
[ 2.481974] [drm] radeon: dpm initialized
[ 2.486664] [drm] Found VCE firmware/feedback version 50.0.1 / 17!
[ 2.486711] [drm] GART: num cpu pages 262144, num gpu pages 262144
[ 2.492402] r8169 0000:04:00.0 enp4s0: Link is Down
[ 2.526396] [drm] GART: Restore entries: num cpu pages 262144, num gpu pages 262144
[ 2.529843] [drm] GART: Done restoring entries
[ 2.529847] [drm] PCIE GART of 1024M enabled (table at 0x00000000001D6000).
[ 2.530087] radeon 0000:00:01.0: WB enabled
[ 2.530090] radeon 0000:00:01.0: fence driver on ring 0 use gpu addr 0x0000000020000c00
[ 2.530468] radeon 0000:00:01.0: fence driver on ring 5 use gpu addr 0x0000000000075a18
[ 2.550526] radeon 0000:00:01.0: fence driver on ring 6 use gpu addr 0x0000000020000c18
[ 2.550531] radeon 0000:00:01.0: fence driver on ring 7 use gpu addr 0x0000000020000c1c
[ 2.550533] radeon 0000:00:01.0: fence driver on ring 1 use gpu addr 0x0000000020000c04
[ 2.550535] radeon 0000:00:01.0: fence driver on ring 2 use gpu addr 0x0000000020000c08
[ 2.550536] radeon 0000:00:01.0: fence driver on ring 3 use gpu addr 0x0000000020000c0c
[ 2.550538] radeon 0000:00:01.0: fence driver on ring 4 use gpu addr 0x0000000020000c10
[ 2.550809] radeon 0000:00:01.0: radeon: MSI limited to 32-bit
[ 2.551003] radeon 0000:00:01.0: radeon: using MSI.
[ 2.551071] [drm] radeon: irq initialized.
[ 2.569579] [drm] ring test on 0 succeeded in 3 usecs
[ 2.569589] [drm] ring test on 3 succeeded in 4 usecs
[ 2.569596] [drm] ring test on 4 succeeded in 4 usecs
[ 2.583595] [drm] ring test on 5 succeeded in 2 usecs
[ 2.585593] [drm] UVD initialized successfully.
[ 2.670213] usb 5-1: New USB device found, idVendor=413c, idProduct=2106, bcdDevice= 1.01
[ 2.670218] usb 5-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 2.670220] usb 5-1: Product: Dell QuietKey Keyboard
[ 2.670222] usb 5-1: Manufacturer: DELL
[ 2.678027] input: DELL Dell QuietKey Keyboard as /devices/pci0000:00/0000:00:12.0/usb5/5-1/5-1:1.0/0003:413C:2106.0001/input/input11
[ 2.694994] [drm] ring test on 6 succeeded in 18 usecs
[ 2.695006] [drm] ring test on 7 succeeded in 3 usecs
[ 2.695007] [drm] VCE initialized successfully.
[ 2.695174] snd_hda_intel 0000:00:01.1: bound 0000:00:01.0 (ops radeon_audio_component_bind_ops [radeon])
[ 2.695346] [drm] ib test on ring 0 succeeded in 0 usecs
[ 2.695400] [drm] ib test on ring 3 succeeded in 0 usecs
[ 2.695449] [drm] ib test on ring 4 succeeded in 0 usecs
[ 2.713280] [drm] ib test on ring 5 succeeded
[ 2.729270] [drm] ib test on ring 6 succeeded in 1 usecs
[ 2.737964] hid-generic 0003:413C:2106.0001: input,hidraw0: USB HID v1.10 Keyboard [DELL Dell QuietKey Keyboard] on usb-0000:00:12.0-1/input0
[ 2.745270] [drm] ib test on ring 7 succeeded in 1 usecs
[ 2.748783] [drm] Radeon Display Connectors
[ 2.748789] [drm] Connector 0:
[ 2.748790] [drm] DP-1
[ 2.748791] [drm] HPD1
[ 2.748792] [drm] DDC: 0x6530 0x6530 0x6534 0x6534 0x6538 0x6538 0x653c 0x653c
[ 2.748795] [drm] Encoders:
[ 2.748796] [drm] DFP1: INTERNAL_UNIPHY2
[ 2.748797] [drm] Connector 1:
[ 2.748798] [drm] VGA-1
[ 2.748799] [drm] HPD2
[ 2.748800] [drm] DDC: 0x6540 0x6540 0x6544 0x6544 0x6548 0x6548 0x654c 0x654c
[ 2.748802] [drm] Encoders:
[ 2.748803] [drm] CRT1: INTERNAL_UNIPHY2
[ 2.748804] [drm] CRT1: NUTMEG
[ 2.748805] [drm] Connector 2:
[ 2.748806] [drm] HDMI-A-1
[ 2.748807] [drm] HPD3
[ 2.748808] [drm] DDC: 0x6550 0x6550 0x6554 0x6554 0x6558 0x6558 0x655c 0x655c
[ 2.748811] [drm] Encoders:
[ 2.748811] [drm] DFP2: INTERNAL_UNIPHY
[ 3.019999] [drm] fb mappable at 0xE03E9000
[ 3.020006] [drm] vram apper at 0xE0000000
[ 3.020009] [drm] size 5242880
[ 3.020011] [drm] fb depth is 24
[ 3.020012] [drm] pitch is 5120
[ 3.020508] fbcon: radeondrmfb (fb0) is primary device
[ 3.201239] usb 5-2: new low-speed USB device number 3 using ohci-pci
[ 3.215988] Console: switching to colour frame buffer device 160x64
[ 3.217767] radeon 0000:00:01.0: [drm] fb0: radeondrmfb frame buffer device
[ 3.237462] [drm] Initialized radeon 2.50.0 20080528 for 0000:00:01.0 on minor 0
[ 3.397485] usb 5-2: New USB device found, idVendor=046d, idProduct=c016, bcdDevice= 3.40
[ 3.397493] usb 5-2: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 3.397495] usb 5-2: Product: Optical USB Mouse
[ 3.397497] usb 5-2: Manufacturer: Logitech
[ 3.406006] input: Logitech Optical USB Mouse as /devices/pci0000:00/0000:00:12.0/usb5/5-2/5-2:1.0/0003:046D:C016.0002/input/input12
[ 3.406790] hid-generic 0003:046D:C016.0002: input,hidraw1: USB HID v1.10 Mouse [Logitech Optical USB Mouse] on usb-0000:00:12.0-2/input0
[ 5.100538] r8169 0000:04:00.0 enp4s0: Link is Up - 1Gbps/Full - flow control rx/tx
[ 5.100555] IPv6: ADDRCONF(NETDEV_CHANGE): enp4s0: link becomes ready
[ 5.945432] [drm] amdgpu kernel modesetting enabled.
[ 10.066361] memfd_create() without MFD_EXEC nor MFD_NOEXEC_SEAL, pid=258 'systemd'
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-17 17:40 ` Paul Menzel
0 siblings, 0 replies; 236+ messages in thread
From: Paul Menzel @ 2023-04-17 17:40 UTC (permalink / raw)
To: Thomas Gleixner
Cc: linux-kernel, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
[-- Attachment #1: Type: text/plain, Size: 1062 bytes --]
Dear Thomas,
Am 17.04.23 um 16:48 schrieb Thomas Gleixner:
> On Mon, Apr 17 2023 at 13:19, Paul Menzel wrote:
>> Am 15.04.23 um 01:44 schrieb Thomas Gleixner:
>> [ 0.258193] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD
>> Graphics (family: 0x15, model: 0x13, stepping: 0x1)
>> […]
>> [ 0.259329] smp: Bringing up secondary CPUs ...
>> [ 0.259527] x86: Booting SMP configuration:
>> [ 0.259528] .... node #0, CPUs: #1
>> [ 0.261007] After schedule_preempt_disabled
>> [ 10.260990] CPU1 failed to report alive state
>
> Weird. CPU1 fails to come up and report that it has reached the
> synchronization point.
>
> Does it work when you add cpuhp.parallel=off on the kernel command line?
Yes, the ten seconds delay is gone with `cpuhp.parallel=off`.
There was a patch set in the past, that worked on that device. I think
up to v4 it did *not* work at all and hung [1]. I need some days to
collect the results again.
Kind regards,
Paul
[1]:
https://lore.kernel.org/lkml/ab28d2ce-4a9c-387d-9eda-558045a0c35b@molgen.mpg.de/
[-- Attachment #2: kodi-linux-6.3-rc6-smp-tglx-cpuhp.paralleloff.txt --]
[-- Type: text/plain, Size: 61806 bytes --]
[ 0.000000] Linux version 6.3.0-rc6-00311-gde8224969f66 (root@bf16f3646a84) (gcc (Debian 11.2.0-12) 11.2.0, GNU ld (GNU Binutils for Debian) 2.40) #446 SMP PREEMPT_DYNAMIC Sat Apr 15 14:12:29 UTC 2023
[ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc6-00311-gde8224969f66 root=/dev/sda3 rw quiet noisapnp cryptomgr.notests ipv6.disable_ipv6=1 selinux=0 cpuhp.parallel=off
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256
[ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format.
[ 0.000000] signal: max sigframe size: 1776
[ 0.000000] BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
[ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005fe4cfff] usable
[ 0.000000] BIOS-e820: [mem 0x000000005fe4d000-0x000000007fffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000f8000000-0x00000000fbffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000fec10000-0x00000000fec10fff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000017effffff] usable
[ 0.000000] NX (Execute Disable) protection: active
[ 0.000000] SMBIOS 3.0.0 present.
[ 0.000000] DMI: ASUS F2A85-M_PRO/F2A85-M_PRO, BIOS 4.18-9-gb640ed51b2 04/17/2023
[ 0.000000] tsc: Fast TSC calibration using PIT
[ 0.000000] tsc: Initial usec timer 9249065
[ 0.000000] tsc: Detected 3899.954 MHz processor
[ 0.000755] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
[ 0.000759] e820: remove [mem 0x000a0000-0x000fffff] usable
[ 0.000763] last_pfn = 0x17f000 max_arch_pfn = 0x400000000
[ 0.000768] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT
[ 0.000938] last_pfn = 0x5fe4d max_arch_pfn = 0x400000000
[ 0.004000] Using GB pages for direct mapping
[ 0.004000] ACPI: Early table checksum verification disabled
[ 0.004000] ACPI: RSDP 0x00000000000F6830 000024 (v02 COREv4)
[ 0.004000] ACPI: XSDT 0x000000005FE5A0E0 000074 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: FACP 0x000000005FE5BBC0 000114 (v06 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: DSDT 0x000000005FE5A280 00193A (v02 COREv4 COREBOOT 00010001 INTL 20200925)
[ 0.004000] ACPI: FACS 0x000000005FE5A240 000040
[ 0.004000] ACPI: FACS 0x000000005FE5A240 000040
[ 0.004000] ACPI: SSDT 0x000000005FE5BCE0 00008A (v02 COREv4 COREBOOT 0000002A CORE 20200925)
[ 0.004000] ACPI: MCFG 0x000000005FE5BD70 00003C (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: APIC 0x000000005FE5BDB0 000062 (v03 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: HPET 0x000000005FE5BE20 000038 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: HEST 0x000000005FE5BE60 0001D0 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: IVRS 0x000000005FE5C030 000070 (v02 AMD AMDIOMMU 00000001 AMD 00000000)
[ 0.004000] ACPI: SSDT 0x000000005FE5C0A0 00051F (v02 AMD ALIB 00000001 MSFT 04000000)
[ 0.004000] ACPI: SSDT 0x000000005FE5C5C0 0006B2 (v01 AMD POWERNOW 00000001 AMD 00000001)
[ 0.004000] ACPI: VFCT 0x000000005FE5CC80 00F269 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: Reserving FACP table memory at [mem 0x5fe5bbc0-0x5fe5bcd3]
[ 0.004000] ACPI: Reserving DSDT table memory at [mem 0x5fe5a280-0x5fe5bbb9]
[ 0.004000] ACPI: Reserving FACS table memory at [mem 0x5fe5a240-0x5fe5a27f]
[ 0.004000] ACPI: Reserving FACS table memory at [mem 0x5fe5a240-0x5fe5a27f]
[ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe5bce0-0x5fe5bd69]
[ 0.004000] ACPI: Reserving MCFG table memory at [mem 0x5fe5bd70-0x5fe5bdab]
[ 0.004000] ACPI: Reserving APIC table memory at [mem 0x5fe5bdb0-0x5fe5be11]
[ 0.004000] ACPI: Reserving HPET table memory at [mem 0x5fe5be20-0x5fe5be57]
[ 0.004000] ACPI: Reserving HEST table memory at [mem 0x5fe5be60-0x5fe5c02f]
[ 0.004000] ACPI: Reserving IVRS table memory at [mem 0x5fe5c030-0x5fe5c09f]
[ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe5c0a0-0x5fe5c5be]
[ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe5c5c0-0x5fe5cc71]
[ 0.004000] ACPI: Reserving VFCT table memory at [mem 0x5fe5cc80-0x5fe6bee8]
[ 0.004000] No NUMA configuration found
[ 0.004000] Faking a node at [mem 0x0000000000000000-0x000000017effffff]
[ 0.004000] NODE_DATA(0) allocated [mem 0x17efe9000-0x17effffff]
[ 0.004000] Zone ranges:
[ 0.004000] DMA [mem 0x0000000000001000-0x0000000000ffffff]
[ 0.004000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]
[ 0.004000] Normal [mem 0x0000000100000000-0x000000017effffff]
[ 0.004000] Device empty
[ 0.004000] Movable zone start for each node
[ 0.004000] Early memory node ranges
[ 0.004000] node 0: [mem 0x0000000000001000-0x000000000009efff]
[ 0.004000] node 0: [mem 0x0000000000100000-0x000000005fe4cfff]
[ 0.004000] node 0: [mem 0x0000000100000000-0x000000017effffff]
[ 0.004000] Initmem setup node 0 [mem 0x0000000000001000-0x000000017effffff]
[ 0.004000] On node 0, zone DMA: 1 pages in unavailable ranges
[ 0.004000] On node 0, zone DMA: 97 pages in unavailable ranges
[ 0.004000] On node 0, zone Normal: 435 pages in unavailable ranges
[ 0.004000] On node 0, zone Normal: 4096 pages in unavailable ranges
[ 0.004000] ACPI: PM-Timer IO Port: 0x818
[ 0.004000] ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1])
[ 0.004000] IOAPIC[0]: apic_id 4, version 33, address 0xfec00000, GSI 0-23
[ 0.004000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[ 0.004000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 low level)
[ 0.004000] ACPI: Using ACPI (MADT) for SMP configuration information
[ 0.004000] ACPI: HPET id: 0x10228210 base: 0xfed00000
[ 0.004000] smpboot: Allowing 2 CPUs, 0 hotplug CPUs
[ 0.004000] smpboot: smpboot: XXX end of prefill_possible_map
[ 0.004000] After prefill_possible_map
[ 0.004000] After init_cpu_to_node
[ 0.004000] After init_gi_nodes
[ 0.004000] After io_apic_init_mappings
[ 0.004000] After x86_init.hyper.guest_late_init
[ 0.004000] [mem 0x80000000-0xf7ffffff] available for PCI devices
[ 0.004000] After e820
[ 0.004000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns
[ 0.004000] After unwind_init
[ 0.004000] After setup_arch
[ 0.004000] After setup_command_line
[ 0.004000] After setup_nr_cpu_ids
[ 0.004000] setup_percpu: NR_CPUS:64 nr_cpumask_bits:2 nr_cpu_ids:2 nr_node_ids:1
[ 0.004000] percpu: Embedded 55 pages/cpu s188328 r8192 d28760 u1048576
[ 0.004000] pcpu-alloc: s188328 r8192 d28760 u1048576 alloc=1*2097152
[ 0.004000] pcpu-alloc: [0] 0 1
[ 0.004000] After setup_per_cpu_areas
[ 0.004000] After smp_perpare_boot_cpu
[ 0.004000] After boot_cpu_hotplug_init
[ 0.004000] Fallback order for Node 0: 0
[ 0.004000] Built 1 zonelists, mobility grouping on. Total pages: 898451
[ 0.004000] Policy zone: Normal
[ 0.004000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc6-00311-gde8224969f66 root=/dev/sda3 rw quiet noisapnp cryptomgr.notests ipv6.disable_ipv6=1 selinux=0 cpuhp.parallel=off
[ 0.004000] Unknown kernel command line parameters "noisapnp BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc6-00311-gde8224969f66", will be passed to user space.
[ 0.004000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[ 0.004000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[ 0.004000] mem auto-init: stack:off, heap alloc:off, heap free:off
[ 0.004000] stackdepot: allocating hash table via alloc_large_system_hash
[ 0.004000] stackdepot hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[ 0.004000] software IO TLB: area num 2.
[ 0.004000] Memory: 3477168K/3651500K available (14336K kernel code, 2340K rwdata, 5308K rodata, 2908K init, 11060K bss, 174072K reserved, 0K cma-reserved)
[ 0.004000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
[ 0.004000] After mm_init
[ 0.004000] After poking_init
[ 0.004000] ftrace: allocating 38664 entries in 152 pages
[ 0.004000] ftrace: allocated 152 pages with 3 groups
[ 0.004000] Dynamic Preempt: full
[ 0.004000] After sched_init
[ 0.004000] rcu: Preemptible hierarchical RCU implementation.
[ 0.004000] rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=2.
[ 0.004000] Trampoline variant of Tasks RCU enabled.
[ 0.004000] Rude variant of Tasks RCU enabled.
[ 0.004000] Tracing variant of Tasks RCU enabled.
[ 0.004000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[ 0.004000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2
[ 0.004000] After rcu_init
[ 0.004000] NR_IRQS: 4352, nr_irqs: 440, preallocated irqs: 16
[ 0.004000] rcu: srcu_init: Setting srcu_struct sizes based on contention.
[ 0.004000] After random_init()
[ 0.004000] After boot_init_stack_canary
[ 0.004000] spurious 8259A interrupt: IRQ7.
[ 0.004000] Console: colour VGA+ 80x25
[ 0.004000] printk: console [tty0] enabled
[ 0.004000] ACPI: Core revision 20221020
[ 0.004000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 133484873504 ns
[ 0.004000] APIC: Switch to symmetric I/O mode setup
[ 0.004000] AMD-Vi: Using global IVHD EFR:0x0, EFR2:0x0
[ 0.004000] APIC: Done
[ 0.004000] Before apic_bsb_setup
[ 0.004000] check_timer begin
[ 0.004000] check_timer after local_irq_disable
[ 0.004000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[ 0.004000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x706e603bb55, max_idle_ns: 881590819133 ns
[ 0.145156] Calibrating delay loop (skipped), value calculated using timer frequency.. 7799.90 BogoMIPS (lpj=15599816)
[ 0.145160] pid_max: default: 32768 minimum: 301
[ 0.145254] LSM: initializing lsm=capability
[ 0.145348] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[ 0.145365] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[ 0.145738] Bit 30 in CPUID ECX not set.
[ 0.145762] Last level iTLB entries: 4KB 512, 2MB 1024, 4MB 512
[ 0.145764] Last level dTLB entries: 4KB 1024, 2MB 1024, 4MB 512, 1GB 0
[ 0.145769] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization
[ 0.145772] Spectre V2 : Mitigation: Retpolines
[ 0.145773] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch
[ 0.145774] Spectre V2 : Spectre v2 / SpectreRSB : Filling RSB on VMEXIT
[ 0.145774] Spectre V2 : Enabling Speculation Barrier for firmware calls
[ 0.145775] RETBleed: Mitigation: untrained return thunk
[ 0.145777] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier
[ 0.145779] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl
[ 0.150244] Freeing SMP alternatives memory: 32K
[ 0.150249] After check_bugs
[ 0.150250] After acpi_subsystem_init
[ 0.150251] After arch_post_acpi_subsys_init
[ 0.150252] After rcu_scheduler_starting
[ 0.150324] After find_task_by_pid_ns and PF_NO_SETAFFINITY
[ 0.150329] After numa_default_policy
[ 0.150349] After rcu_read_lock
[ 0.150350] After rcu_read_unlock
[ 0.150351] After kthreadd_done
[ 0.150363] smpboot: Start of smp_prepare_cpus_common
[ 0.150365] smpboot: smpboot: zalloc 0
[ 0.150366] smpboot: smpboot: zalloc 1
[ 0.150367] smpboot: smpboot: After set_sched_topology()
[ 0.150369] smpboot: smpboot: After smp_sanity_check()
[ 0.150369] smpboot: smpboot: Before x86_init.timers.setup_percpu_clockev()
[ 0.258381] smpboot: smpboot: After x86_init.timers.setup_percpu_clockev()
[ 0.258382] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD Graphics (family: 0x15, model: 0x13, stepping: 0x1)
[ 0.258615] cblist_init_generic: Setting adjustable number of callback queues.
[ 0.258618] cblist_init_generic: Setting shift to 1 and lim to 1.
[ 0.258648] cblist_init_generic: Setting shift to 1 and lim to 1.
[ 0.258675] cblist_init_generic: Setting shift to 1 and lim to 1.
[ 0.258703] Performance Events: Fam15h core perfctr, AMD PMU driver.
[ 0.258726] ... version: 0
[ 0.258727] ... bit width: 48
[ 0.258728] ... generic registers: 6
[ 0.258729] ... value mask: 0000ffffffffffff
[ 0.258730] ... max period: 00007fffffffffff
[ 0.258731] ... fixed-purpose events: 0
[ 0.258732] ... event mask: 000000000000003f
[ 0.258852] rcu: Hierarchical SRCU implementation.
[ 0.258853] rcu: Max phase no-delay instances is 1000.
[ 0.259441] NMI watchdog: Enabled. Permanently consumes one hw-PMU counter.
[ 0.259515] smp: Bringing up secondary CPUs ...
[ 0.259714] x86: Booting SMP configuration:
[ 0.259715] .... node #0, CPUs: #1
[ 0.121151] Bit 30 in CPUID ECX not set.
[ 0.259844] After schedule_preempt_disabled
[ 0.259849] smp: Brought up 1 node, 2 CPUs
[ 0.259849] smpboot: Max logical packages: 1
[ 0.259849] smpboot: Total of 2 processors activated (15599.81 BogoMIPS)
[ 0.261311] devtmpfs: initialized
[ 0.261311] x86/mm: Memory block size: 128MB
[ 0.262220] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[ 0.262220] futex hash table entries: 512 (order: 3, 32768 bytes, linear)
[ 0.262220] pinctrl core: initialized pinctrl subsystem
[ 0.262220] PM: RTC time: 17:24:09, date: 2023-04-17
[ 0.262220] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[ 0.262434] audit: initializing netlink subsys (disabled)
[ 0.262453] audit: type=2000 audit(1681752249.140:1): state=initialized audit_enabled=0 res=1
[ 0.262453] thermal_sys: Registered thermal governor 'fair_share'
[ 0.262453] thermal_sys: Registered thermal governor 'bang_bang'
[ 0.262453] thermal_sys: Registered thermal governor 'step_wise'
[ 0.262453] thermal_sys: Registered thermal governor 'user_space'
[ 0.262453] cpuidle: using governor ladder
[ 0.262453] cpuidle: using governor menu
[ 0.262453] PCI: MMCONFIG for domain 0000 [bus 00-3f] at [mem 0xf8000000-0xfbffffff] (base 0xf8000000)
[ 0.262453] PCI: MMCONFIG at [mem 0xf8000000-0xfbffffff] reserved as E820 entry
[ 0.262453] PCI: Using configuration type 1 for base access
[ 0.262453] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.
[ 0.273235] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages
[ 0.273235] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page
[ 0.273235] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages
[ 0.273235] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page
[ 0.275428] cryptd: max_cpu_qlen set to 1000
[ 0.275428] ACPI: Added _OSI(Module Device)
[ 0.275428] ACPI: Added _OSI(Processor Device)
[ 0.275428] ACPI: Added _OSI(3.0 _SCP Extensions)
[ 0.275428] ACPI: Added _OSI(Processor Aggregator Device)
[ 0.280879] ACPI: DSDT successfully acquired and loaded
[ 0.281154] ACPI: 4 ACPI AML tables successfully acquired and loaded
[ 0.281649] ACPI: Interpreter enabled
[ 0.281669] ACPI: PM: (supports S0 S1 S3 S5)
[ 0.281671] ACPI: Using IOAPIC for interrupt routing
[ 0.281724] HEST: Table parsing has been initialized.
[ 0.281743] GHES: Failed to enable APEI firmware first mode.
[ 0.281745] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[ 0.281746] PCI: Ignoring E820 reservations for host bridge windows
[ 0.282003] ACPI: Enabled 8 GPEs in block 00 to 1F
[ 0.286735] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[ 0.286746] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]
[ 0.286838] acpi PNP0A03:00: _OSC: OS now controls [PME AER PCIeCapability LTR]
[ 0.286852] acpi PNP0A03:00: [Firmware Info]: MMCONFIG for domain 0000 [bus 00-3f] only partially covers this bridge
[ 0.286942] acpi PNP0A03:00: host bridge window expanded to [io 0x0000-0x0cf7 window]; [io 0x03b0-0x03df window] ignored
[ 0.287197] PCI host bridge to bus 0000:00
[ 0.287199] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]
[ 0.287201] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]
[ 0.287203] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000dffff]
[ 0.287205] pci_bus 0000:00: root bus resource [mem 0x80000000-0xffffffff]
[ 0.287210] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 0.287234] pci 0000:00:00.0: [1022:1410] type 00 class 0x060000
[ 0.287388] pci 0000:00:00.2: [1022:1419] type 00 class 0x080600
[ 0.287479] pci 0000:00:01.0: [1002:9996] type 00 class 0x030000
[ 0.287488] pci 0000:00:01.0: reg 0x10: [mem 0xe0000000-0xefffffff pref]
[ 0.287493] pci 0000:00:01.0: reg 0x14: [io 0x1000-0x10ff]
[ 0.287498] pci 0000:00:01.0: reg 0x18: [mem 0xf0180000-0xf01bffff]
[ 0.287514] pci 0000:00:01.0: enabling Extended Tags
[ 0.287524] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]
[ 0.287541] pci 0000:00:01.0: supports D1 D2
[ 0.287611] pci 0000:00:01.1: [1002:9902] type 00 class 0x040300
[ 0.287618] pci 0000:00:01.1: reg 0x10: [mem 0xf01c0000-0xf01c3fff]
[ 0.287640] pci 0000:00:01.1: enabling Extended Tags
[ 0.287663] pci 0000:00:01.1: supports D1 D2
[ 0.287750] pci 0000:00:11.0: [1022:7801] type 00 class 0x010601
[ 0.287763] pci 0000:00:11.0: reg 0x10: [io 0x1410-0x1417]
[ 0.287771] pci 0000:00:11.0: reg 0x14: [io 0x1420-0x1423]
[ 0.287778] pci 0000:00:11.0: reg 0x18: [io 0x1418-0x141f]
[ 0.287785] pci 0000:00:11.0: reg 0x1c: [io 0x1424-0x1427]
[ 0.287793] pci 0000:00:11.0: reg 0x20: [io 0x1400-0x140f]
[ 0.287800] pci 0000:00:11.0: reg 0x24: [mem 0xf01cc000-0xf01cc7ff]
[ 0.287954] pci 0000:00:12.0: [1022:7807] type 00 class 0x0c0310
[ 0.287967] pci 0000:00:12.0: reg 0x10: [mem 0xf01c8000-0xf01c8fff]
[ 0.288152] pci 0000:00:12.2: [1022:7808] type 00 class 0x0c0320
[ 0.288165] pci 0000:00:12.2: reg 0x10: [mem 0xf01cd000-0xf01cd0ff]
[ 0.288230] pci 0000:00:12.2: supports D1 D2
[ 0.288231] pci 0000:00:12.2: PME# supported from D0 D1 D2 D3hot
[ 0.288233] pci 0000:00:12.2: pme_poll = true
[ 0.288234] pci 0000:00:12.2: after device_set_wakeup_capable()
[ 0.288237] pci 0000:00:12.2: after pci_pme_active()
[ 0.288373] pci 0000:00:13.0: [1022:7807] type 00 class 0x0c0310
[ 0.288387] pci 0000:00:13.0: reg 0x10: [mem 0xf01c9000-0xf01c9fff]
[ 0.288570] pci 0000:00:13.2: [1022:7808] type 00 class 0x0c0320
[ 0.288583] pci 0000:00:13.2: reg 0x10: [mem 0xf01ce000-0xf01ce0ff]
[ 0.288650] pci 0000:00:13.2: supports D1 D2
[ 0.288652] pci 0000:00:13.2: PME# supported from D0 D1 D2 D3hot
[ 0.288653] pci 0000:00:13.2: pme_poll = true
[ 0.288654] pci 0000:00:13.2: after device_set_wakeup_capable()
[ 0.288657] pci 0000:00:13.2: after pci_pme_active()
[ 0.288798] pci 0000:00:14.0: [1022:780b] type 00 class 0x0c0500
[ 0.288980] pci 0000:00:14.2: [1022:780d] type 00 class 0x040300
[ 0.288997] pci 0000:00:14.2: reg 0x10: [mem 0xf01c4000-0xf01c7fff 64bit]
[ 0.289052] pci 0000:00:14.2: PME# supported from D0 D3hot D3cold
[ 0.289054] pci 0000:00:14.2: pme_poll = true
[ 0.289055] pci 0000:00:14.2: after device_set_wakeup_capable()
[ 0.289058] pci 0000:00:14.2: after pci_pme_active()
[ 0.290131] pci 0000:00:14.3: [1022:780e] type 00 class 0x060100
[ 0.290335] pci 0000:00:14.4: [1022:780f] type 01 class 0x060401
[ 0.290490] pci 0000:00:14.5: [1022:7809] type 00 class 0x0c0310
[ 0.290504] pci 0000:00:14.5: reg 0x10: [mem 0xf01ca000-0xf01cafff]
[ 0.290677] pci 0000:00:15.0: [1022:43a0] type 01 class 0x060400
[ 0.290705] pci 0000:00:15.0: enabling Extended Tags
[ 0.290746] pci 0000:00:15.0: supports D1 D2
[ 0.290907] pci 0000:00:15.1: [1022:43a1] type 01 class 0x060400
[ 0.290939] pci 0000:00:15.1: enabling Extended Tags
[ 0.290978] pci 0000:00:15.1: supports D1 D2
[ 0.291146] pci 0000:00:15.2: [1022:43a2] type 01 class 0x060400
[ 0.291174] pci 0000:00:15.2: enabling Extended Tags
[ 0.291214] pci 0000:00:15.2: supports D1 D2
[ 0.291289] pci 0000:00:16.0: [1022:7807] type 00 class 0x0c0310
[ 0.291303] pci 0000:00:16.0: reg 0x10: [mem 0xf01cb000-0xf01cbfff]
[ 0.291473] pci 0000:00:16.2: [1022:7808] type 00 class 0x0c0320
[ 0.291486] pci 0000:00:16.2: reg 0x10: [mem 0xf01cf000-0xf01cf0ff]
[ 0.291551] pci 0000:00:16.2: supports D1 D2
[ 0.291552] pci 0000:00:16.2: PME# supported from D0 D1 D2 D3hot
[ 0.291554] pci 0000:00:16.2: pme_poll = true
[ 0.291555] pci 0000:00:16.2: after device_set_wakeup_capable()
[ 0.291558] pci 0000:00:16.2: after pci_pme_active()
[ 0.291684] pci 0000:00:18.0: [1022:1400] type 00 class 0x060000
[ 0.291748] pci 0000:00:18.1: [1022:1401] type 00 class 0x060000
[ 0.291806] pci 0000:00:18.2: [1022:1402] type 00 class 0x060000
[ 0.291866] pci 0000:00:18.3: [1022:1403] type 00 class 0x060000
[ 0.291998] pci 0000:00:18.4: [1022:1404] type 00 class 0x060000
[ 0.292064] pci 0000:00:18.5: [1022:1405] type 00 class 0x060000
[ 0.292136] pci_bus 0000:01: extended config space not accessible
[ 0.292200] pci 0000:00:14.4: PCI bridge to [bus 01] (subtractive decode)
[ 0.292208] pci 0000:00:14.4: bridge window [io 0x0000-0x0cf7 window] (subtractive decode)
[ 0.292211] pci 0000:00:14.4: bridge window [io 0x0d00-0xffff window] (subtractive decode)
[ 0.292214] pci 0000:00:14.4: bridge window [mem 0x000a0000-0x000dffff] (subtractive decode)
[ 0.292216] pci 0000:00:14.4: bridge window [mem 0x80000000-0xffffffff] (subtractive decode)
[ 0.292261] pci 0000:00:15.0: PCI bridge to [bus 02]
[ 0.292345] pci 0000:03:00.0: [1b21:1042] type 00 class 0x0c0330
[ 0.292382] pci 0000:03:00.0: reg 0x10: [mem 0xf0000000-0xf0007fff 64bit]
[ 0.292559] pci 0000:03:00.0: PME# supported from D3hot D3cold
[ 0.292560] pci 0000:03:00.0: pme_poll = true
[ 0.292562] pci 0000:03:00.0: after device_set_wakeup_capable()
[ 0.292567] pci 0000:03:00.0: after pci_pme_active()
[ 0.292605] pci 0000:03:00.0: 2.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s PCIe x1 link at 0000:00:15.1 (capable of 4.000 Gb/s with 5.0 GT/s PCIe x1 link)
[ 0.305215] pci 0000:00:15.1: PCI bridge to [bus 03]
[ 0.305228] pci 0000:00:15.1: bridge window [mem 0xf0000000-0xf00fffff]
[ 0.305237] pci 0000:00:15.2: bridge configuration invalid ([bus 00-00]), reconfiguring
[ 0.305359] pci 0000:04:00.0: [10ec:8168] type 00 class 0x020000
[ 0.305378] pci 0000:04:00.0: reg 0x10: [io 0x0000-0x00ff]
[ 0.305399] pci 0000:04:00.0: reg 0x18: [mem 0x00000000-0x00000fff 64bit pref]
[ 0.305413] pci 0000:04:00.0: reg 0x20: [mem 0x00000000-0x00003fff 64bit pref]
[ 0.305520] pci 0000:04:00.0: supports D1 D2
[ 0.305522] pci 0000:04:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[ 0.305524] pci 0000:04:00.0: pme_poll = true
[ 0.305525] pci 0000:04:00.0: after device_set_wakeup_capable()
[ 0.305529] pci 0000:04:00.0: after pci_pme_active()
[ 0.321217] pci 0000:00:15.2: PCI bridge to [bus 04-ff]
[ 0.321228] pci 0000:00:15.2: bridge window [io 0x0000-0x0fff]
[ 0.321232] pci 0000:00:15.2: bridge window [mem 0x00000000-0x000fffff]
[ 0.321237] pci 0000:00:15.2: bridge window [mem 0x00000000-0x000fffff 64bit pref]
[ 0.321240] pci_bus 0000:04: busn_res: [bus 04-ff] end is updated to 04
[ 0.321759] ACPI: PCI: Interrupt link INTA configured for IRQ 0
[ 0.321855] ACPI: PCI: Interrupt link INTB configured for IRQ 0
[ 0.321949] ACPI: PCI: Interrupt link INTC configured for IRQ 0
[ 0.322041] ACPI: PCI: Interrupt link INTD configured for IRQ 0
[ 0.322134] ACPI: PCI: Interrupt link INTE configured for IRQ 0
[ 0.322232] ACPI: PCI: Interrupt link INTF configured for IRQ 0
[ 0.322325] ACPI: PCI: Interrupt link INTG configured for IRQ 0
[ 0.322418] ACPI: PCI: Interrupt link INTH configured for IRQ 0
[ 0.322640] iommu: Default domain type: Translated
[ 0.322642] iommu: DMA domain TLB invalidation policy: lazy mode
[ 0.322839] SCSI subsystem initialized
[ 0.325159] libata version 3.00 loaded.
[ 0.325159] ACPI: bus type USB registered
[ 0.325159] usbcore: registered new interface driver usbfs
[ 0.325159] usbcore: registered new interface driver hub
[ 0.325159] usbcore: registered new device driver usb
[ 0.325159] PCI: Using ACPI for IRQ routing
[ 0.325159] PCI: pci_cache_line_size set to 64 bytes
[ 0.325159] e820: reserve RAM buffer [mem 0x0009fc00-0x0009ffff]
[ 0.325159] e820: reserve RAM buffer [mem 0x5fe4d000-0x5fffffff]
[ 0.325159] e820: reserve RAM buffer [mem 0x17f000000-0x17fffffff]
[ 0.325159] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0
[ 0.325159] hpet0: 3 comparators, 32-bit 14.318180 MHz counter
[ 0.330232] clocksource: Switched to clocksource tsc-early
[ 0.330474] VFS: Disk quotas dquot_6.6.0
[ 0.330500] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[ 0.330620] pnp: PnP ACPI init
[ 0.330969] system 00:00: [mem 0xfec10002-0xfec11001] could not be reserved
[ 0.331362] pnp: PnP ACPI: found 2 devices
[ 0.337986] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
[ 0.338186] NET: Registered PF_INET protocol family
[ 0.338359] IP idents hash table entries: 65536 (order: 7, 524288 bytes, linear)
[ 0.340153] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear)
[ 0.340169] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
[ 0.340177] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear)
[ 0.340251] TCP bind hash table entries: 32768 (order: 8, 1048576 bytes, linear)
[ 0.340614] TCP: Hash tables configured (established 32768 bind 32768)
[ 0.340685] UDP hash table entries: 2048 (order: 4, 65536 bytes, linear)
[ 0.340711] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear)
[ 0.340827] NET: Registered PF_UNIX/PF_LOCAL protocol family
[ 0.340862] pci 0000:00:15.2: BAR 15: assigned [mem 0x80000000-0x800fffff 64bit pref]
[ 0.340869] pci 0000:00:15.2: BAR 13: assigned [io 0x2000-0x2fff]
[ 0.340874] pci 0000:00:14.4: PCI bridge to [bus 01]
[ 0.340886] pci 0000:00:15.0: PCI bridge to [bus 02]
[ 0.340894] pci 0000:00:15.1: PCI bridge to [bus 03]
[ 0.340898] pci 0000:00:15.1: bridge window [mem 0xf0000000-0xf00fffff]
[ 0.340908] pci 0000:04:00.0: BAR 4: assigned [mem 0x80000000-0x80003fff 64bit pref]
[ 0.340922] pci 0000:04:00.0: BAR 2: assigned [mem 0x80004000-0x80004fff 64bit pref]
[ 0.340934] pci 0000:04:00.0: BAR 0: assigned [io 0x2000-0x20ff]
[ 0.340941] pci 0000:00:15.2: PCI bridge to [bus 04]
[ 0.340943] pci 0000:00:15.2: bridge window [io 0x2000-0x2fff]
[ 0.340949] pci 0000:00:15.2: bridge window [mem 0x80000000-0x800fffff 64bit pref]
[ 0.340956] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]
[ 0.340958] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]
[ 0.340960] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000dffff]
[ 0.340964] pci_bus 0000:00: resource 7 [mem 0x80000000-0xffffffff]
[ 0.340966] pci_bus 0000:01: resource 4 [io 0x0000-0x0cf7 window]
[ 0.340968] pci_bus 0000:01: resource 5 [io 0x0d00-0xffff window]
[ 0.340970] pci_bus 0000:01: resource 6 [mem 0x000a0000-0x000dffff]
[ 0.340973] pci_bus 0000:01: resource 7 [mem 0x80000000-0xffffffff]
[ 0.340975] pci_bus 0000:03: resource 1 [mem 0xf0000000-0xf00fffff]
[ 0.340977] pci_bus 0000:04: resource 0 [io 0x2000-0x2fff]
[ 0.340979] pci_bus 0000:04: resource 2 [mem 0x80000000-0x800fffff 64bit pref]
[ 0.341104] pci 0000:00:01.1: D0 power state depends on 0000:00:01.0
[ 0.341412] pci 0000:00:12.0: AMD USB device
[ 0.341436] pci 0000:00:12.0: AMD USB ohci handoff
[ 0.341833] pci 0000:00:12.2: AMD USB device
[ 0.341847] pci 0000:00:12.2: AMD USB ehci handoff
[ 0.342032] pci 0000:00:12.2: PME# does not work under D3, disabling it
[ 0.342235] pci 0000:00:13.0: AMD USB device
[ 0.342250] pci 0000:00:13.0: AMD USB ohci handoff
[ 0.342633] pci 0000:00:13.2: AMD USB device
[ 0.342650] pci 0000:00:13.2: AMD USB ehci handoff
[ 0.342830] pci 0000:00:13.2: PME# does not work under D3, disabling it
[ 0.343041] pci 0000:00:14.5: AMD USB device
[ 0.343057] pci 0000:00:14.5: AMD USB ohci handoff
[ 0.343577] pci 0000:00:16.0: AMD USB device
[ 0.343598] pci 0000:00:16.0: AMD USB ohci handoff
[ 0.343993] pci 0000:00:16.2: AMD USB device
[ 0.344008] pci 0000:00:16.2: AMD USB ehci handoff
[ 0.344202] pci 0000:00:16.2: PME# does not work under D3, disabling it
[ 0.344510] pci 0000:03:00.0: AMD USB xhci handoff
[ 0.344577] PCI: CLS 64 bytes, default 64
[ 0.344678] pci 0000:00:00.2: AMD-Vi: Applying erratum 746 workaround
[ 0.344790] pci 0000:00:01.0: Adding to iommu group 0
[ 0.344813] pci 0000:00:01.1: Adding to iommu group 0
[ 0.344844] pci 0000:00:11.0: Adding to iommu group 1
[ 0.344885] pci 0000:00:12.0: Adding to iommu group 2
[ 0.344905] pci 0000:00:12.2: Adding to iommu group 2
[ 0.344949] pci 0000:00:13.0: Adding to iommu group 3
[ 0.344970] pci 0000:00:13.2: Adding to iommu group 3
[ 0.345020] pci 0000:00:14.0: Adding to iommu group 4
[ 0.345043] pci 0000:00:14.2: Adding to iommu group 4
[ 0.345066] pci 0000:00:14.3: Adding to iommu group 4
[ 0.345102] pci 0000:00:14.4: Adding to iommu group 5
[ 0.345128] pci 0000:00:14.5: Adding to iommu group 6
[ 0.345172] pci 0000:00:15.0: Adding to iommu group 7
[ 0.345198] pci 0000:00:15.1: Adding to iommu group 7
[ 0.345219] pci 0000:00:15.2: Adding to iommu group 7
[ 0.345261] pci 0000:00:16.0: Adding to iommu group 8
[ 0.345285] pci 0000:00:16.2: Adding to iommu group 8
[ 0.345349] pci 0000:00:18.0: Adding to iommu group 9
[ 0.345374] pci 0000:00:18.1: Adding to iommu group 9
[ 0.345400] pci 0000:00:18.2: Adding to iommu group 9
[ 0.345428] pci 0000:00:18.3: Adding to iommu group 9
[ 0.345451] pci 0000:00:18.4: Adding to iommu group 9
[ 0.345479] pci 0000:00:18.5: Adding to iommu group 9
[ 0.345489] pci 0000:03:00.0: Adding to iommu group 7
[ 0.345499] pci 0000:04:00.0: Adding to iommu group 7
[ 0.348265] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[ 0.348272] AMD-Vi: Extended features (0x800000853, 0x0): PreF PPR GT IA
[ 0.348279] AMD-Vi: Interrupt remapping enabled
[ 0.348484] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[ 0.348486] software IO TLB: mapped [mem 0x000000005be4d000-0x000000005fe4d000] (64MB)
[ 0.348547] LVT offset 0 assigned for vector 0x400
[ 0.348595] perf: AMD IBS detected (0x000000ff)
[ 0.348602] amd_uncore: 4 amd_nb counters detected
[ 0.352225] workingset: timestamp_bits=37 max_order=20 bucket_order=0
[ 0.352259] zbud: loaded
[ 0.352752] NET: Registered PF_ALG protocol family
[ 0.352758] Key type asymmetric registered
[ 0.352759] Asymmetric key parser 'x509' registered
[ 0.353113] alg: self-tests disabled
[ 0.353209] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 251)
[ 0.353263] io scheduler mq-deadline registered
[ 0.353265] io scheduler kyber registered
[ 0.354479] pcieport 0000:00:15.0: PME: Signaling with IRQ 25
[ 0.354644] pcieport 0000:00:15.1: PME: Signaling with IRQ 26
[ 0.354712] pcieport 0000:00:15.2: enabling device (0000 -> 0003)
[ 0.354941] pcieport 0000:00:15.2: PME: Signaling with IRQ 27
[ 0.355211] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
[ 0.355298] ACPI: button: Power Button [PWRF]
[ 0.355363] ACPI: \_SB_.P000: Found 2 idle states
[ 0.355495] ACPI: \_SB_.P001: Found 2 idle states
[ 0.356442] thermal LNXTHERM:00: registered as thermal_zone0
[ 0.356445] ACPI: thermal: Thermal Zone [TZ00] (21 C)
[ 0.356797] Non-volatile memory driver v1.3
[ 0.356864] AMD-Vi: AMD IOMMUv2 loaded and initialized
[ 0.357065] ahci 0000:00:11.0: version 3.0
[ 0.357356] ahci 0000:00:11.0: AHCI 0001.0300 32 slots 8 ports 6 Gbps 0x40 impl SATA mode
[ 0.357360] ahci 0000:00:11.0: flags: 64bit ncq sntf ilck led clo pio
[ 0.358785] scsi host0: ahci
[ 0.358999] scsi host1: ahci
[ 0.359215] scsi host2: ahci
[ 0.359424] scsi host3: ahci
[ 0.359624] scsi host4: ahci
[ 0.359836] scsi host5: ahci
[ 0.360048] scsi host6: ahci
[ 0.360243] scsi host7: ahci
[ 0.360336] ata port1: DUMMY
[ 0.360338] ata port2: DUMMY
[ 0.360339] ata port3: DUMMY
[ 0.360341] ata port4: DUMMY
[ 0.360342] ata port5: DUMMY
[ 0.360343] ata port6: DUMMY
[ 0.360345] ata port7: SATA max UDMA/133 abar m2048@0xf01cc000 port 0xf01cc400 irq 19
[ 0.360347] ata port8: DUMMY
[ 0.360431] ACPI: bus type drm_connector registered
[ 0.360734] i8042: PNP: No PS/2 controller found.
[ 0.360736] i8042: Probing ports directly.
[ 0.363078] serio: i8042 KBD port at 0x60,0x64 irq 1
[ 0.363185] serio: i8042 AUX port at 0x60,0x64 irq 12
[ 0.363347] mousedev: PS/2 mouse device common for all mice
[ 0.363420] rtc_cmos 00:01: RTC can wake from S4
[ 0.363924] rtc_cmos 00:01: registered as rtc0
[ 0.363950] rtc_cmos 00:01: setting system clock to 2023-04-17T17:24:09 UTC (1681752249)
[ 0.364011] rtc_cmos 00:01: alarms up to one day, y3k, 114 bytes nvram, hpet irqs
[ 0.364048] device-mapper: uevent: version 1.0.3
[ 0.364136] device-mapper: ioctl: 4.47.0-ioctl (2022-07-28) initialised: dm-devel@redhat.com
[ 0.364301] hid: raw HID events driver (C) Jiri Kosina
[ 0.364335] usbcore: registered new interface driver usbhid
[ 0.364336] usbhid: USB HID core driver
[ 0.364427] Initializing XFRM netlink socket
[ 0.364436] NET: Registered PF_PACKET protocol family
[ 0.364438] x86/pm: family 0x15 cpu detected, MSR saving is needed during suspending.
[ 0.364652] microcode: CPU1: patch_level=0x0600111f
[ 0.364652] microcode: CPU0: patch_level=0x0600111f
[ 0.364664] microcode: Microcode Update Driver: v2.2.
[ 0.364668] IPI shorthand broadcast: enabled
[ 0.364678] AVX version of gcm_enc/dec engaged.
[ 0.364710] AES CTR mode by8 optimization enabled
[ 0.368578] sched_clock: Marking stable (249972593, 117151517)->(369867515, -2743405)
[ 0.368823] registered taskstats version 1
[ 0.369080] zswap: loaded using pool lzo/zbud
[ 0.371663] ata link7: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[ 0.371939] ata dev7.0: ATA-9: SanDisk SDSSDP064G, 2.0.0, max UDMA/133
[ 0.371942] ata dev7.0: 125045424 sectors, multi 1: LBA48 NCQ (depth 32)
[ 0.372148] ata dev7.0: configured for UDMA/133
[ 0.372283] scsi 6:0:0:0: Direct-Access ATA SanDisk SDSSDP06 0 PQ: 0 ANSI: 5
[ 0.373496] kmemleak: Kernel memory leak detector initialized (mem pool available: 15679)
[ 0.373501] debug_vm_pgtable: [debug_vm_pgtable ]: Validating architecture page table helpers
[ 0.373959] sd 6:0:0:0: [sda] 125045424 512-byte logical blocks: (64.0 GB/59.6 GiB)
[ 0.373982] sd 6:0:0:0: [sda] Write Protect is off
[ 0.373987] sd 6:0:0:0: [sda] Mode Sense: 00 3a 00 00
[ 0.374015] sd 6:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 0.374059] sd 6:0:0:0: [sda] Preferred minimum I/O size 512 bytes
[ 0.380407] kmemleak: Automatic memory scanning thread started
[ 0.380633] sda: sda1 sda2 sda3
[ 0.381120] Key type encrypted registered
[ 0.381248] sd 6:0:0:0: [sda] Attached SCSI disk
[ 0.384294] PM: Magic number: 3:443:441
[ 0.396721] EXT4-fs (sda3): mounted filesystem fe29e0dc-6303-4401-987c-8472bc1b9516 with ordered data mode. Quota mode: none.
[ 0.396766] VFS: Mounted root (ext4 filesystem) on device 8:3.
[ 0.398650] devtmpfs: mounted
[ 0.398668] After kernel_init_freeable
[ 0.403170] Freeing unused kernel image (initmem) memory: 2908K
[ 0.415639] Write protecting the kernel read-only data: 20480k
[ 0.415922] Freeing unused kernel image (rodata/data gap) memory: 836K
[ 0.453024] x86/mm: Checked W+X mappings: passed, no W+X pages found.
[ 0.453030] rodata_test: all tests were successful
[ 0.453031] After mark_readonly
[ 0.453031] After pti_finalize
[ 0.453045] rcu_end_inkernel_boot
[ 0.453054] Run /sbin/init as init process
[ 0.453055] with arguments:
[ 0.453057] /sbin/init
[ 0.453058] noisapnp
[ 0.453059] with environment:
[ 0.453060] HOME=/
[ 0.453060] TERM=linux
[ 0.453061] BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc6-00311-gde8224969f66
[ 0.629518] systemd[1]: Inserted module 'autofs4'
[ 0.656601] NET: Registered PF_INET6 protocol family
[ 0.657495] Segment Routing with IPv6
[ 0.657523] In-situ OAM (IOAM) with IPv6
[ 0.683745] systemd[1]: systemd 252.6-1 running in system mode (+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
[ 0.683756] systemd[1]: Detected architecture x86-64.
[ 0.688485] systemd[1]: Hostname set to <kodi>.
[ 0.959071] systemd[1]: Queued start job for default target graphical.target.
[ 0.982198] systemd[1]: Created slice system-getty.slice - Slice /system/getty.
[ 0.983286] systemd[1]: Created slice system-modprobe.slice - Slice /system/modprobe.
[ 0.984123] systemd[1]: Created slice user.slice - User and Session Slice.
[ 0.984302] systemd[1]: Started systemd-ask-password-console.path - Dispatch Password Requests to Console Directory Watch.
[ 0.984417] systemd[1]: Started systemd-ask-password-wall.path - Forward Password Requests to Wall Directory Watch.
[ 0.984855] systemd[1]: Set up automount proc-sys-fs-binfmt_misc.automount - Arbitrary Executable File Formats File System Automount Point.
[ 0.984894] systemd[1]: Reached target cryptsetup.target - Local Encrypted Volumes.
[ 0.984934] systemd[1]: Reached target integritysetup.target - Local Integrity Protected Volumes.
[ 0.984972] systemd[1]: Reached target paths.target - Path Units.
[ 0.985006] systemd[1]: Reached target remote-fs.target - Remote File Systems.
[ 0.985038] systemd[1]: Reached target slices.target - Slice Units.
[ 0.985078] systemd[1]: Reached target swap.target - Swaps.
[ 0.985116] systemd[1]: Reached target veritysetup.target - Local Verity Protected Volumes.
[ 0.987550] systemd[1]: Listening on systemd-coredump.socket - Process Core Dump Socket.
[ 0.987802] systemd[1]: Listening on systemd-fsckd.socket - fsck to fsckd communication Socket.
[ 0.987975] systemd[1]: Listening on systemd-initctl.socket - initctl Compatibility Named Pipe.
[ 0.988274] systemd[1]: Listening on systemd-journald-audit.socket - Journal Audit Socket.
[ 0.988543] systemd[1]: Listening on systemd-journald-dev-log.socket - Journal Socket (/dev/log).
[ 0.988822] systemd[1]: Listening on systemd-journald.socket - Journal Socket.
[ 0.989072] systemd[1]: Listening on systemd-networkd.socket - Network Service Netlink Socket.
[ 0.989946] systemd[1]: Listening on systemd-udevd-control.socket - udev Control Socket.
[ 0.990210] systemd[1]: Listening on systemd-udevd-kernel.socket - udev Kernel Socket.
[ 0.992904] systemd[1]: Mounting dev-hugepages.mount - Huge Pages File System...
[ 0.995944] systemd[1]: Mounting dev-mqueue.mount - POSIX Message Queue File System...
[ 0.999142] systemd[1]: Mounting sys-kernel-debug.mount - Kernel Debug File System...
[ 1.002682] systemd[1]: Mounting sys-kernel-tracing.mount - Kernel Trace File System...
[ 1.007256] systemd[1]: Starting kmod-static-nodes.service - Create List of Static Device Nodes...
[ 1.014276] systemd[1]: Starting modprobe@configfs.service - Load Kernel Module configfs...
[ 1.017399] systemd[1]: Starting modprobe@dm_mod.service - Load Kernel Module dm_mod...
[ 1.020657] systemd[1]: Starting modprobe@drm.service - Load Kernel Module drm...
[ 1.024006] systemd[1]: Starting modprobe@efi_pstore.service - Load Kernel Module efi_pstore...
[ 1.027278] systemd[1]: Starting modprobe@fuse.service - Load Kernel Module fuse...
[ 1.036066] systemd[1]: Starting modprobe@loop.service - Load Kernel Module loop...
[ 1.036151] systemd[1]: systemd-firstboot.service - First Boot Wizard was skipped because of an unmet condition check (ConditionFirstBoot=yes).
[ 1.036237] systemd[1]: systemd-fsck-root.service - File System Check on Root Device was skipped because of an unmet condition check (ConditionPathIsReadWrite=!/).
[ 1.036279] systemd[1]: Reached target local-fs.target - Local File Systems.
[ 1.036362] systemd[1]: apparmor.service - Load AppArmor profiles was skipped because of an unmet condition check (ConditionSecurity=apparmor).
[ 1.041356] loop: module loaded
[ 1.043749] systemd[1]: Starting systemd-binfmt.service - Set Up Additional Binary Formats...
[ 1.048983] systemd[1]: Starting systemd-journald.service - Journal Service...
[ 1.050740] fuse: init (API version 7.38)
[ 1.052824] systemd[1]: Starting systemd-random-seed.service - Load/Save Random Seed...
[ 1.063924] systemd[1]: Starting systemd-sysctl.service - Apply Kernel Variables...
[ 1.068061] systemd[1]: Starting systemd-sysusers.service - Create System Users...
[ 1.071328] systemd[1]: Starting systemd-udev-trigger.service - Coldplug All udev Devices...
[ 1.093517] systemd[1]: Mounted dev-hugepages.mount - Huge Pages File System.
[ 1.094134] systemd[1]: Mounted dev-mqueue.mount - POSIX Message Queue File System.
[ 1.094447] systemd[1]: Mounted sys-kernel-debug.mount - Kernel Debug File System.
[ 1.094985] systemd[1]: Mounted sys-kernel-tracing.mount - Kernel Trace File System.
[ 1.095974] systemd[1]: Finished kmod-static-nodes.service - Create List of Static Device Nodes.
[ 1.096888] systemd[1]: modprobe@configfs.service: Deactivated successfully.
[ 1.097257] systemd[1]: Finished modprobe@configfs.service - Load Kernel Module configfs.
[ 1.098001] systemd[1]: modprobe@dm_mod.service: Deactivated successfully.
[ 1.098339] systemd[1]: Finished modprobe@dm_mod.service - Load Kernel Module dm_mod.
[ 1.099077] systemd[1]: modprobe@drm.service: Deactivated successfully.
[ 1.099446] systemd[1]: Finished modprobe@drm.service - Load Kernel Module drm.
[ 1.100518] systemd[1]: modprobe@efi_pstore.service: Deactivated successfully.
[ 1.100846] systemd[1]: Finished modprobe@efi_pstore.service - Load Kernel Module efi_pstore.
[ 1.101749] systemd[1]: modprobe@fuse.service: Deactivated successfully.
[ 1.102064] systemd[1]: Finished modprobe@fuse.service - Load Kernel Module fuse.
[ 1.102810] systemd[1]: modprobe@loop.service: Deactivated successfully.
[ 1.103142] systemd[1]: Finished modprobe@loop.service - Load Kernel Module loop.
[ 1.103830] systemd[1]: proc-sys-fs-binfmt_misc.automount: Got automount request for /proc/sys/fs/binfmt_misc, triggered by 138 (systemd-binfmt)
[ 1.123242] systemd[1]: Mounting proc-sys-fs-binfmt_misc.mount - Arbitrary Executable File Formats File System...
[ 1.140028] systemd[1]: Mounting sys-fs-fuse-connections.mount - FUSE Control File System...
[ 1.148223] systemd[1]: Mounting sys-kernel-config.mount - Kernel Configuration File System...
[ 1.148346] systemd[1]: systemd-pstore.service - Platform Persistent Storage Archival was skipped because of an unmet condition check (ConditionDirectoryNotEmpty=/sys/fs/pstore).
[ 1.148515] systemd[1]: systemd-repart.service - Repartition Root Disk was skipped because no trigger condition checks were met.
[ 1.159909] systemd[1]: Finished systemd-sysctl.service - Apply Kernel Variables.
[ 1.184529] systemd[1]: Finished systemd-sysusers.service - Create System Users.
[ 1.186000] systemd[1]: Mounted proc-sys-fs-binfmt_misc.mount - Arbitrary Executable File Formats File System.
[ 1.186357] systemd[1]: Mounted sys-fs-fuse-connections.mount - FUSE Control File System.
[ 1.186600] systemd[1]: Mounted sys-kernel-config.mount - Kernel Configuration File System.
[ 1.209823] systemd[1]: Starting systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev...
[ 1.210512] systemd[1]: Finished systemd-binfmt.service - Set Up Additional Binary Formats.
[ 1.254273] systemd[1]: Finished systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev.
[ 1.273469] systemd[1]: Starting systemd-udevd.service - Rule-based Manager for Device Events and Files...
[ 1.313286] systemd[1]: Started systemd-journald.service - Journal Service.
[ 1.359636] systemd-journald[139]: Received client request to flush runtime journal.
[ 1.369173] tsc: Refined TSC clocksource calibration: 3900.223 MHz
[ 1.369184] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x70705a6472c, max_idle_ns: 881590586812 ns
[ 1.369200] clocksource: Switched to clocksource tsc
[ 1.594559] sd 6:0:0:0: Attached scsi generic sg0 type 0
[ 1.731704] acpi_cpufreq: overriding BIOS provided _PSD data
[ 1.953174] random: crng init done
[ 1.992654] QUIRK: Enable AMD PLL fix
[ 1.992706] ehci-pci 0000:00:12.2: EHCI Host Controller
[ 1.992738] ehci-pci 0000:00:12.2: new USB bus registered, assigned bus number 1
[ 1.992751] ehci-pci 0000:00:12.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
[ 1.992760] ehci-pci 0000:00:12.2: debug port 1
[ 1.992971] ehci-pci 0000:00:12.2: irq 17, io mem 0xf01cd000
[ 2.005197] ehci-pci 0000:00:12.2: USB 2.0 started, EHCI 1.00
[ 2.005468] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 2.005471] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.005474] usb usb1: Product: EHCI Host Controller
[ 2.005476] usb usb1: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ehci_hcd
[ 2.005478] usb usb1: SerialNumber: 0000:00:12.2
[ 2.006279] hub 1-0:1.0: USB hub found
[ 2.006321] hub 1-0:1.0: 5 ports detected
[ 2.007319] ehci-pci 0000:00:13.2: EHCI Host Controller
[ 2.007345] ehci-pci 0000:00:13.2: new USB bus registered, assigned bus number 2
[ 2.007360] ehci-pci 0000:00:13.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
[ 2.007369] ehci-pci 0000:00:13.2: debug port 1
[ 2.007523] ehci-pci 0000:00:13.2: irq 17, io mem 0xf01ce000
[ 2.016141] piix4_smbus 0000:00:14.0: SMBus Host Controller at 0xb00, revision 0
[ 2.016147] piix4_smbus 0000:00:14.0: Using register 0x2e for SMBus port selection
[ 2.016788] piix4_smbus 0000:00:14.0: Auxiliary SMBus Host Controller at 0xb20
[ 2.021184] ehci-pci 0000:00:13.2: USB 2.0 started, EHCI 1.00
[ 2.021523] usb usb2: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 2.021527] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.021529] usb usb2: Product: EHCI Host Controller
[ 2.021531] usb usb2: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ehci_hcd
[ 2.021533] usb usb2: SerialNumber: 0000:00:13.2
[ 2.032086] hub 2-0:1.0: USB hub found
[ 2.032171] hub 2-0:1.0: 5 ports detected
[ 2.033420] ehci-pci 0000:00:16.2: EHCI Host Controller
[ 2.033455] ehci-pci 0000:00:16.2: new USB bus registered, assigned bus number 3
[ 2.033470] ehci-pci 0000:00:16.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
[ 2.033479] ehci-pci 0000:00:16.2: debug port 1
[ 2.033641] ehci-pci 0000:00:16.2: irq 17, io mem 0xf01cf000
[ 2.049177] ehci-pci 0000:00:16.2: USB 2.0 started, EHCI 1.00
[ 2.049454] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 2.049459] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.049461] usb usb3: Product: EHCI Host Controller
[ 2.049463] usb usb3: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ehci_hcd
[ 2.049465] usb usb3: SerialNumber: 0000:00:16.2
[ 2.050025] hub 3-0:1.0: USB hub found
[ 2.050070] hub 3-0:1.0: 4 ports detected
[ 2.050881] ohci-pci 0000:00:16.0: OHCI PCI host controller
[ 2.050910] ohci-pci 0000:00:16.0: new USB bus registered, assigned bus number 4
[ 2.050999] ohci-pci 0000:00:12.0: OHCI PCI host controller
[ 2.051021] ohci-pci 0000:00:12.0: new USB bus registered, assigned bus number 5
[ 2.051110] ohci-pci 0000:00:16.0: irq 18, io mem 0xf01cb000
[ 2.051111] ohci-pci 0000:00:12.0: irq 18, io mem 0xf01c8000
[ 2.051120] ohci-pci 0000:00:13.0: OHCI PCI host controller
[ 2.051141] ohci-pci 0000:00:13.0: new USB bus registered, assigned bus number 6
[ 2.051240] ohci-pci 0000:00:13.0: irq 18, io mem 0xf01c9000
[ 2.051250] ohci-pci 0000:00:14.5: OHCI PCI host controller
[ 2.051268] ohci-pci 0000:00:14.5: new USB bus registered, assigned bus number 7
[ 2.051356] ohci-pci 0000:00:14.5: irq 18, io mem 0xf01ca000
[ 2.113579] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 2.113586] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.113589] usb usb4: Product: OHCI PCI host controller
[ 2.113591] usb usb4: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ohci_hcd
[ 2.113593] usb usb4: SerialNumber: 0000:00:16.0
[ 2.114096] hub 4-0:1.0: USB hub found
[ 2.114127] hub 4-0:1.0: 4 ports detected
[ 2.126194] usb usb5: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 2.126202] usb usb5: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.126204] usb usb5: Product: OHCI PCI host controller
[ 2.126207] usb usb5: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ohci_hcd
[ 2.126209] usb usb5: SerialNumber: 0000:00:12.0
[ 2.127341] hub 5-0:1.0: USB hub found
[ 2.127380] hub 5-0:1.0: 5 ports detected
[ 2.128522] usb usb6: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 2.128528] usb usb6: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.128531] usb usb6: Product: OHCI PCI host controller
[ 2.128533] usb usb6: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ohci_hcd
[ 2.128535] usb usb6: SerialNumber: 0000:00:13.0
[ 2.129049] hub 6-0:1.0: USB hub found
[ 2.129082] hub 6-0:1.0: 5 ports detected
[ 2.130311] usb usb7: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 2.130316] usb usb7: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.130319] usb usb7: Product: OHCI PCI host controller
[ 2.130321] usb usb7: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ohci_hcd
[ 2.130323] usb usb7: SerialNumber: 0000:00:14.5
[ 2.130855] hub 7-0:1.0: USB hub found
[ 2.130895] hub 7-0:1.0: 2 ports detected
[ 2.148137] 1
[ 2.148196] 2
[ 2.148820] snd_hda_intel 0000:00:01.1: Force to non-snoop mode
[ 2.148832] 3
[ 2.148833] 4
[ 2.148834] 5
[ 2.148835] 7
[ 2.148837] 8
[ 2.148837] 9
[ 2.148929] 1
[ 2.148979] 2
[ 2.149637] 3
[ 2.149641] 4
[ 2.149642] 5
[ 2.149643] 7
[ 2.149645] 8
[ 2.149646] 9
[ 2.166674] xhci_hcd 0000:03:00.0: xHCI Host Controller
[ 2.166707] xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 8
[ 2.206853] input: HDA ATI HDMI HDMI/DP,pcm=3 as /devices/pci0000:00/0000:00:01.1/sound/card0/input1
[ 2.208133] input: HDA ATI HDMI HDMI/DP,pcm=7 as /devices/pci0000:00/0000:00:01.1/sound/card0/input2
[ 2.221615] snd_hda_codec_realtek hdaudioC1D0: ALC892: SKU not ready 0x00000100
[ 2.222377] snd_hda_codec_realtek hdaudioC1D0: autoconfig for ALC892: line_outs=4 (0x14/0x16/0x15/0x17/0x0) type:line
[ 2.222384] snd_hda_codec_realtek hdaudioC1D0: speaker_outs=0 (0x0/0x0/0x0/0x0/0x0)
[ 2.222387] snd_hda_codec_realtek hdaudioC1D0: hp_outs=1 (0x1b/0x0/0x0/0x0/0x0)
[ 2.222390] snd_hda_codec_realtek hdaudioC1D0: mono: mono_out=0x0
[ 2.222392] snd_hda_codec_realtek hdaudioC1D0: dig-out=0x1e/0x0
[ 2.222393] snd_hda_codec_realtek hdaudioC1D0: inputs:
[ 2.222395] snd_hda_codec_realtek hdaudioC1D0: Rear Mic=0x18
[ 2.222398] snd_hda_codec_realtek hdaudioC1D0: Front Mic=0x19
[ 2.222399] snd_hda_codec_realtek hdaudioC1D0: Line=0x1a
[ 2.222401] snd_hda_codec_realtek hdaudioC1D0: CD=0x1c
[ 2.231811] xhci_hcd 0000:03:00.0: hcc params 0x0200f180 hci version 0x96 quirks 0x0000000000080010
[ 2.235245] r8169 0000:04:00.0: enabling device (0000 -> 0003)
[ 2.265051] xhci_hcd 0000:03:00.0: xHCI Host Controller
[ 2.265076] xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 9
[ 2.265092] xhci_hcd 0000:03:00.0: Host supports USB 3.0 SuperSpeed
[ 2.266951] usb usb8: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 2.266958] usb usb8: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.266961] usb usb8: Product: xHCI Host Controller
[ 2.266963] usb usb8: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 xhci-hcd
[ 2.266965] usb usb8: SerialNumber: 0000:03:00.0
[ 2.267014] input: HD-Audio Generic Rear Mic as /devices/pci0000:00/0000:00:14.2/sound/card1/input3
[ 2.267354] input: HD-Audio Generic Front Mic as /devices/pci0000:00/0000:00:14.2/sound/card1/input4
[ 2.267750] input: HD-Audio Generic Line as /devices/pci0000:00/0000:00:14.2/sound/card1/input5
[ 2.268089] input: HD-Audio Generic Line Out Front as /devices/pci0000:00/0000:00:14.2/sound/card1/input6
[ 2.268437] input: HD-Audio Generic Line Out Surround as /devices/pci0000:00/0000:00:14.2/sound/card1/input7
[ 2.268783] input: HD-Audio Generic Line Out CLFE as /devices/pci0000:00/0000:00:14.2/sound/card1/input8
[ 2.269132] input: HD-Audio Generic Line Out Side as /devices/pci0000:00/0000:00:14.2/sound/card1/input9
[ 2.269714] hub 8-0:1.0: USB hub found
[ 2.269815] hub 8-0:1.0: 2 ports detected
[ 2.269956] input: HD-Audio Generic Front Headphone as /devices/pci0000:00/0000:00:14.2/sound/card1/input10
[ 2.272826] usb usb9: We don't know the algorithms for LPM for this host, disabling LPM.
[ 2.273006] usb usb9: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 6.03
[ 2.273010] usb usb9: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.273012] usb usb9: Product: xHCI Host Controller
[ 2.273014] usb usb9: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 xhci-hcd
[ 2.273016] usb usb9: SerialNumber: 0000:03:00.0
[ 2.275521] hub 9-0:1.0: USB hub found
[ 2.275649] hub 9-0:1.0: 2 ports detected
[ 2.295318] r8169 0000:04:00.0 eth0: RTL8168f/8111f, 08:60:6e:74:7a:51, XID 480, IRQ 32
[ 2.295327] r8169 0000:04:00.0 eth0: jumbo features [frames: 9194 bytes, tx checksumming: ko]
[ 2.311845] r8169 0000:04:00.0 enp4s0: renamed from eth0
[ 2.421403] r8169 0000:04:00.0: Direct firmware load for rtl_nic/rtl8168f-1.fw failed with error -2
[ 2.421415] r8169 0000:04:00.0: Unable to load firmware rtl_nic/rtl8168f-1.fw (-2)
[ 2.421903] RTL8211E Gigabit Ethernet r8169-0-400:00: attached PHY driver (mii_bus:phy_addr=r8169-0-400:00, irq=MAC)
[ 2.465163] usb 5-1: new low-speed USB device number 2 using ohci-pci
[ 2.472735] [drm] radeon kernel modesetting enabled.
[ 2.474200] [drm] initializing kernel modesetting (ARUBA 0x1002:0x9996 0x1002:0x9996 0x00).
[ 2.474271] ATOM BIOS: 113
[ 2.474372] radeon 0000:00:01.0: VRAM: 512M 0x0000000000000000 - 0x000000001FFFFFFF (512M used)
[ 2.474376] radeon 0000:00:01.0: GTT: 1024M 0x0000000020000000 - 0x000000005FFFFFFF
[ 2.474384] [drm] Detected VRAM RAM=512M, BAR=256M
[ 2.474385] [drm] RAM width 64bits DDR
[ 2.474589] [drm] radeon: 512M of VRAM memory ready
[ 2.474595] [drm] radeon: 1024M of GTT memory ready.
[ 2.474642] [drm] Loading ARUBA Microcode
[ 2.481608] [drm] Internal thermal controller without fan control
[ 2.481974] [drm] radeon: dpm initialized
[ 2.486664] [drm] Found VCE firmware/feedback version 50.0.1 / 17!
[ 2.486711] [drm] GART: num cpu pages 262144, num gpu pages 262144
[ 2.492402] r8169 0000:04:00.0 enp4s0: Link is Down
[ 2.526396] [drm] GART: Restore entries: num cpu pages 262144, num gpu pages 262144
[ 2.529843] [drm] GART: Done restoring entries
[ 2.529847] [drm] PCIE GART of 1024M enabled (table at 0x00000000001D6000).
[ 2.530087] radeon 0000:00:01.0: WB enabled
[ 2.530090] radeon 0000:00:01.0: fence driver on ring 0 use gpu addr 0x0000000020000c00
[ 2.530468] radeon 0000:00:01.0: fence driver on ring 5 use gpu addr 0x0000000000075a18
[ 2.550526] radeon 0000:00:01.0: fence driver on ring 6 use gpu addr 0x0000000020000c18
[ 2.550531] radeon 0000:00:01.0: fence driver on ring 7 use gpu addr 0x0000000020000c1c
[ 2.550533] radeon 0000:00:01.0: fence driver on ring 1 use gpu addr 0x0000000020000c04
[ 2.550535] radeon 0000:00:01.0: fence driver on ring 2 use gpu addr 0x0000000020000c08
[ 2.550536] radeon 0000:00:01.0: fence driver on ring 3 use gpu addr 0x0000000020000c0c
[ 2.550538] radeon 0000:00:01.0: fence driver on ring 4 use gpu addr 0x0000000020000c10
[ 2.550809] radeon 0000:00:01.0: radeon: MSI limited to 32-bit
[ 2.551003] radeon 0000:00:01.0: radeon: using MSI.
[ 2.551071] [drm] radeon: irq initialized.
[ 2.569579] [drm] ring test on 0 succeeded in 3 usecs
[ 2.569589] [drm] ring test on 3 succeeded in 4 usecs
[ 2.569596] [drm] ring test on 4 succeeded in 4 usecs
[ 2.583595] [drm] ring test on 5 succeeded in 2 usecs
[ 2.585593] [drm] UVD initialized successfully.
[ 2.670213] usb 5-1: New USB device found, idVendor=413c, idProduct=2106, bcdDevice= 1.01
[ 2.670218] usb 5-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 2.670220] usb 5-1: Product: Dell QuietKey Keyboard
[ 2.670222] usb 5-1: Manufacturer: DELL
[ 2.678027] input: DELL Dell QuietKey Keyboard as /devices/pci0000:00/0000:00:12.0/usb5/5-1/5-1:1.0/0003:413C:2106.0001/input/input11
[ 2.694994] [drm] ring test on 6 succeeded in 18 usecs
[ 2.695006] [drm] ring test on 7 succeeded in 3 usecs
[ 2.695007] [drm] VCE initialized successfully.
[ 2.695174] snd_hda_intel 0000:00:01.1: bound 0000:00:01.0 (ops radeon_audio_component_bind_ops [radeon])
[ 2.695346] [drm] ib test on ring 0 succeeded in 0 usecs
[ 2.695400] [drm] ib test on ring 3 succeeded in 0 usecs
[ 2.695449] [drm] ib test on ring 4 succeeded in 0 usecs
[ 2.713280] [drm] ib test on ring 5 succeeded
[ 2.729270] [drm] ib test on ring 6 succeeded in 1 usecs
[ 2.737964] hid-generic 0003:413C:2106.0001: input,hidraw0: USB HID v1.10 Keyboard [DELL Dell QuietKey Keyboard] on usb-0000:00:12.0-1/input0
[ 2.745270] [drm] ib test on ring 7 succeeded in 1 usecs
[ 2.748783] [drm] Radeon Display Connectors
[ 2.748789] [drm] Connector 0:
[ 2.748790] [drm] DP-1
[ 2.748791] [drm] HPD1
[ 2.748792] [drm] DDC: 0x6530 0x6530 0x6534 0x6534 0x6538 0x6538 0x653c 0x653c
[ 2.748795] [drm] Encoders:
[ 2.748796] [drm] DFP1: INTERNAL_UNIPHY2
[ 2.748797] [drm] Connector 1:
[ 2.748798] [drm] VGA-1
[ 2.748799] [drm] HPD2
[ 2.748800] [drm] DDC: 0x6540 0x6540 0x6544 0x6544 0x6548 0x6548 0x654c 0x654c
[ 2.748802] [drm] Encoders:
[ 2.748803] [drm] CRT1: INTERNAL_UNIPHY2
[ 2.748804] [drm] CRT1: NUTMEG
[ 2.748805] [drm] Connector 2:
[ 2.748806] [drm] HDMI-A-1
[ 2.748807] [drm] HPD3
[ 2.748808] [drm] DDC: 0x6550 0x6550 0x6554 0x6554 0x6558 0x6558 0x655c 0x655c
[ 2.748811] [drm] Encoders:
[ 2.748811] [drm] DFP2: INTERNAL_UNIPHY
[ 3.019999] [drm] fb mappable at 0xE03E9000
[ 3.020006] [drm] vram apper at 0xE0000000
[ 3.020009] [drm] size 5242880
[ 3.020011] [drm] fb depth is 24
[ 3.020012] [drm] pitch is 5120
[ 3.020508] fbcon: radeondrmfb (fb0) is primary device
[ 3.201239] usb 5-2: new low-speed USB device number 3 using ohci-pci
[ 3.215988] Console: switching to colour frame buffer device 160x64
[ 3.217767] radeon 0000:00:01.0: [drm] fb0: radeondrmfb frame buffer device
[ 3.237462] [drm] Initialized radeon 2.50.0 20080528 for 0000:00:01.0 on minor 0
[ 3.397485] usb 5-2: New USB device found, idVendor=046d, idProduct=c016, bcdDevice= 3.40
[ 3.397493] usb 5-2: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 3.397495] usb 5-2: Product: Optical USB Mouse
[ 3.397497] usb 5-2: Manufacturer: Logitech
[ 3.406006] input: Logitech Optical USB Mouse as /devices/pci0000:00/0000:00:12.0/usb5/5-2/5-2:1.0/0003:046D:C016.0002/input/input12
[ 3.406790] hid-generic 0003:046D:C016.0002: input,hidraw1: USB HID v1.10 Mouse [Logitech Optical USB Mouse] on usb-0000:00:12.0-2/input0
[ 5.100538] r8169 0000:04:00.0 enp4s0: Link is Up - 1Gbps/Full - flow control rx/tx
[ 5.100555] IPv6: ADDRCONF(NETDEV_CHANGE): enp4s0: link becomes ready
[ 5.945432] [drm] amdgpu kernel modesetting enabled.
[ 10.066361] memfd_create() without MFD_EXEC nor MFD_NOEXEC_SEAL, pid=258 'systemd'
[-- Attachment #3: Type: text/plain, Size: 161 bytes --]
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-17 17:40 ` Paul Menzel
0 siblings, 0 replies; 236+ messages in thread
From: Paul Menzel @ 2023-04-17 17:40 UTC (permalink / raw)
To: Thomas Gleixner
Cc: linux-kernel, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
[-- Attachment #1: Type: text/plain, Size: 1062 bytes --]
Dear Thomas,
Am 17.04.23 um 16:48 schrieb Thomas Gleixner:
> On Mon, Apr 17 2023 at 13:19, Paul Menzel wrote:
>> Am 15.04.23 um 01:44 schrieb Thomas Gleixner:
>> [ 0.258193] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD
>> Graphics (family: 0x15, model: 0x13, stepping: 0x1)
>> […]
>> [ 0.259329] smp: Bringing up secondary CPUs ...
>> [ 0.259527] x86: Booting SMP configuration:
>> [ 0.259528] .... node #0, CPUs: #1
>> [ 0.261007] After schedule_preempt_disabled
>> [ 10.260990] CPU1 failed to report alive state
>
> Weird. CPU1 fails to come up and report that it has reached the
> synchronization point.
>
> Does it work when you add cpuhp.parallel=off on the kernel command line?
Yes, the ten seconds delay is gone with `cpuhp.parallel=off`.
There was a patch set in the past, that worked on that device. I think
up to v4 it did *not* work at all and hung [1]. I need some days to
collect the results again.
Kind regards,
Paul
[1]:
https://lore.kernel.org/lkml/ab28d2ce-4a9c-387d-9eda-558045a0c35b@molgen.mpg.de/
[-- Attachment #2: kodi-linux-6.3-rc6-smp-tglx-cpuhp.paralleloff.txt --]
[-- Type: text/plain, Size: 61806 bytes --]
[ 0.000000] Linux version 6.3.0-rc6-00311-gde8224969f66 (root@bf16f3646a84) (gcc (Debian 11.2.0-12) 11.2.0, GNU ld (GNU Binutils for Debian) 2.40) #446 SMP PREEMPT_DYNAMIC Sat Apr 15 14:12:29 UTC 2023
[ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc6-00311-gde8224969f66 root=/dev/sda3 rw quiet noisapnp cryptomgr.notests ipv6.disable_ipv6=1 selinux=0 cpuhp.parallel=off
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256
[ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format.
[ 0.000000] signal: max sigframe size: 1776
[ 0.000000] BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
[ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005fe4cfff] usable
[ 0.000000] BIOS-e820: [mem 0x000000005fe4d000-0x000000007fffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000f8000000-0x00000000fbffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000fec10000-0x00000000fec10fff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000017effffff] usable
[ 0.000000] NX (Execute Disable) protection: active
[ 0.000000] SMBIOS 3.0.0 present.
[ 0.000000] DMI: ASUS F2A85-M_PRO/F2A85-M_PRO, BIOS 4.18-9-gb640ed51b2 04/17/2023
[ 0.000000] tsc: Fast TSC calibration using PIT
[ 0.000000] tsc: Initial usec timer 9249065
[ 0.000000] tsc: Detected 3899.954 MHz processor
[ 0.000755] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
[ 0.000759] e820: remove [mem 0x000a0000-0x000fffff] usable
[ 0.000763] last_pfn = 0x17f000 max_arch_pfn = 0x400000000
[ 0.000768] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT
[ 0.000938] last_pfn = 0x5fe4d max_arch_pfn = 0x400000000
[ 0.004000] Using GB pages for direct mapping
[ 0.004000] ACPI: Early table checksum verification disabled
[ 0.004000] ACPI: RSDP 0x00000000000F6830 000024 (v02 COREv4)
[ 0.004000] ACPI: XSDT 0x000000005FE5A0E0 000074 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: FACP 0x000000005FE5BBC0 000114 (v06 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: DSDT 0x000000005FE5A280 00193A (v02 COREv4 COREBOOT 00010001 INTL 20200925)
[ 0.004000] ACPI: FACS 0x000000005FE5A240 000040
[ 0.004000] ACPI: FACS 0x000000005FE5A240 000040
[ 0.004000] ACPI: SSDT 0x000000005FE5BCE0 00008A (v02 COREv4 COREBOOT 0000002A CORE 20200925)
[ 0.004000] ACPI: MCFG 0x000000005FE5BD70 00003C (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: APIC 0x000000005FE5BDB0 000062 (v03 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: HPET 0x000000005FE5BE20 000038 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: HEST 0x000000005FE5BE60 0001D0 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: IVRS 0x000000005FE5C030 000070 (v02 AMD AMDIOMMU 00000001 AMD 00000000)
[ 0.004000] ACPI: SSDT 0x000000005FE5C0A0 00051F (v02 AMD ALIB 00000001 MSFT 04000000)
[ 0.004000] ACPI: SSDT 0x000000005FE5C5C0 0006B2 (v01 AMD POWERNOW 00000001 AMD 00000001)
[ 0.004000] ACPI: VFCT 0x000000005FE5CC80 00F269 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: Reserving FACP table memory at [mem 0x5fe5bbc0-0x5fe5bcd3]
[ 0.004000] ACPI: Reserving DSDT table memory at [mem 0x5fe5a280-0x5fe5bbb9]
[ 0.004000] ACPI: Reserving FACS table memory at [mem 0x5fe5a240-0x5fe5a27f]
[ 0.004000] ACPI: Reserving FACS table memory at [mem 0x5fe5a240-0x5fe5a27f]
[ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe5bce0-0x5fe5bd69]
[ 0.004000] ACPI: Reserving MCFG table memory at [mem 0x5fe5bd70-0x5fe5bdab]
[ 0.004000] ACPI: Reserving APIC table memory at [mem 0x5fe5bdb0-0x5fe5be11]
[ 0.004000] ACPI: Reserving HPET table memory at [mem 0x5fe5be20-0x5fe5be57]
[ 0.004000] ACPI: Reserving HEST table memory at [mem 0x5fe5be60-0x5fe5c02f]
[ 0.004000] ACPI: Reserving IVRS table memory at [mem 0x5fe5c030-0x5fe5c09f]
[ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe5c0a0-0x5fe5c5be]
[ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe5c5c0-0x5fe5cc71]
[ 0.004000] ACPI: Reserving VFCT table memory at [mem 0x5fe5cc80-0x5fe6bee8]
[ 0.004000] No NUMA configuration found
[ 0.004000] Faking a node at [mem 0x0000000000000000-0x000000017effffff]
[ 0.004000] NODE_DATA(0) allocated [mem 0x17efe9000-0x17effffff]
[ 0.004000] Zone ranges:
[ 0.004000] DMA [mem 0x0000000000001000-0x0000000000ffffff]
[ 0.004000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]
[ 0.004000] Normal [mem 0x0000000100000000-0x000000017effffff]
[ 0.004000] Device empty
[ 0.004000] Movable zone start for each node
[ 0.004000] Early memory node ranges
[ 0.004000] node 0: [mem 0x0000000000001000-0x000000000009efff]
[ 0.004000] node 0: [mem 0x0000000000100000-0x000000005fe4cfff]
[ 0.004000] node 0: [mem 0x0000000100000000-0x000000017effffff]
[ 0.004000] Initmem setup node 0 [mem 0x0000000000001000-0x000000017effffff]
[ 0.004000] On node 0, zone DMA: 1 pages in unavailable ranges
[ 0.004000] On node 0, zone DMA: 97 pages in unavailable ranges
[ 0.004000] On node 0, zone Normal: 435 pages in unavailable ranges
[ 0.004000] On node 0, zone Normal: 4096 pages in unavailable ranges
[ 0.004000] ACPI: PM-Timer IO Port: 0x818
[ 0.004000] ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1])
[ 0.004000] IOAPIC[0]: apic_id 4, version 33, address 0xfec00000, GSI 0-23
[ 0.004000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[ 0.004000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 low level)
[ 0.004000] ACPI: Using ACPI (MADT) for SMP configuration information
[ 0.004000] ACPI: HPET id: 0x10228210 base: 0xfed00000
[ 0.004000] smpboot: Allowing 2 CPUs, 0 hotplug CPUs
[ 0.004000] smpboot: smpboot: XXX end of prefill_possible_map
[ 0.004000] After prefill_possible_map
[ 0.004000] After init_cpu_to_node
[ 0.004000] After init_gi_nodes
[ 0.004000] After io_apic_init_mappings
[ 0.004000] After x86_init.hyper.guest_late_init
[ 0.004000] [mem 0x80000000-0xf7ffffff] available for PCI devices
[ 0.004000] After e820
[ 0.004000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns
[ 0.004000] After unwind_init
[ 0.004000] After setup_arch
[ 0.004000] After setup_command_line
[ 0.004000] After setup_nr_cpu_ids
[ 0.004000] setup_percpu: NR_CPUS:64 nr_cpumask_bits:2 nr_cpu_ids:2 nr_node_ids:1
[ 0.004000] percpu: Embedded 55 pages/cpu s188328 r8192 d28760 u1048576
[ 0.004000] pcpu-alloc: s188328 r8192 d28760 u1048576 alloc=1*2097152
[ 0.004000] pcpu-alloc: [0] 0 1
[ 0.004000] After setup_per_cpu_areas
[ 0.004000] After smp_perpare_boot_cpu
[ 0.004000] After boot_cpu_hotplug_init
[ 0.004000] Fallback order for Node 0: 0
[ 0.004000] Built 1 zonelists, mobility grouping on. Total pages: 898451
[ 0.004000] Policy zone: Normal
[ 0.004000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc6-00311-gde8224969f66 root=/dev/sda3 rw quiet noisapnp cryptomgr.notests ipv6.disable_ipv6=1 selinux=0 cpuhp.parallel=off
[ 0.004000] Unknown kernel command line parameters "noisapnp BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc6-00311-gde8224969f66", will be passed to user space.
[ 0.004000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[ 0.004000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[ 0.004000] mem auto-init: stack:off, heap alloc:off, heap free:off
[ 0.004000] stackdepot: allocating hash table via alloc_large_system_hash
[ 0.004000] stackdepot hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[ 0.004000] software IO TLB: area num 2.
[ 0.004000] Memory: 3477168K/3651500K available (14336K kernel code, 2340K rwdata, 5308K rodata, 2908K init, 11060K bss, 174072K reserved, 0K cma-reserved)
[ 0.004000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
[ 0.004000] After mm_init
[ 0.004000] After poking_init
[ 0.004000] ftrace: allocating 38664 entries in 152 pages
[ 0.004000] ftrace: allocated 152 pages with 3 groups
[ 0.004000] Dynamic Preempt: full
[ 0.004000] After sched_init
[ 0.004000] rcu: Preemptible hierarchical RCU implementation.
[ 0.004000] rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=2.
[ 0.004000] Trampoline variant of Tasks RCU enabled.
[ 0.004000] Rude variant of Tasks RCU enabled.
[ 0.004000] Tracing variant of Tasks RCU enabled.
[ 0.004000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[ 0.004000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2
[ 0.004000] After rcu_init
[ 0.004000] NR_IRQS: 4352, nr_irqs: 440, preallocated irqs: 16
[ 0.004000] rcu: srcu_init: Setting srcu_struct sizes based on contention.
[ 0.004000] After random_init()
[ 0.004000] After boot_init_stack_canary
[ 0.004000] spurious 8259A interrupt: IRQ7.
[ 0.004000] Console: colour VGA+ 80x25
[ 0.004000] printk: console [tty0] enabled
[ 0.004000] ACPI: Core revision 20221020
[ 0.004000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 133484873504 ns
[ 0.004000] APIC: Switch to symmetric I/O mode setup
[ 0.004000] AMD-Vi: Using global IVHD EFR:0x0, EFR2:0x0
[ 0.004000] APIC: Done
[ 0.004000] Before apic_bsb_setup
[ 0.004000] check_timer begin
[ 0.004000] check_timer after local_irq_disable
[ 0.004000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[ 0.004000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x706e603bb55, max_idle_ns: 881590819133 ns
[ 0.145156] Calibrating delay loop (skipped), value calculated using timer frequency.. 7799.90 BogoMIPS (lpj=15599816)
[ 0.145160] pid_max: default: 32768 minimum: 301
[ 0.145254] LSM: initializing lsm=capability
[ 0.145348] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[ 0.145365] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[ 0.145738] Bit 30 in CPUID ECX not set.
[ 0.145762] Last level iTLB entries: 4KB 512, 2MB 1024, 4MB 512
[ 0.145764] Last level dTLB entries: 4KB 1024, 2MB 1024, 4MB 512, 1GB 0
[ 0.145769] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization
[ 0.145772] Spectre V2 : Mitigation: Retpolines
[ 0.145773] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch
[ 0.145774] Spectre V2 : Spectre v2 / SpectreRSB : Filling RSB on VMEXIT
[ 0.145774] Spectre V2 : Enabling Speculation Barrier for firmware calls
[ 0.145775] RETBleed: Mitigation: untrained return thunk
[ 0.145777] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier
[ 0.145779] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl
[ 0.150244] Freeing SMP alternatives memory: 32K
[ 0.150249] After check_bugs
[ 0.150250] After acpi_subsystem_init
[ 0.150251] After arch_post_acpi_subsys_init
[ 0.150252] After rcu_scheduler_starting
[ 0.150324] After find_task_by_pid_ns and PF_NO_SETAFFINITY
[ 0.150329] After numa_default_policy
[ 0.150349] After rcu_read_lock
[ 0.150350] After rcu_read_unlock
[ 0.150351] After kthreadd_done
[ 0.150363] smpboot: Start of smp_prepare_cpus_common
[ 0.150365] smpboot: smpboot: zalloc 0
[ 0.150366] smpboot: smpboot: zalloc 1
[ 0.150367] smpboot: smpboot: After set_sched_topology()
[ 0.150369] smpboot: smpboot: After smp_sanity_check()
[ 0.150369] smpboot: smpboot: Before x86_init.timers.setup_percpu_clockev()
[ 0.258381] smpboot: smpboot: After x86_init.timers.setup_percpu_clockev()
[ 0.258382] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD Graphics (family: 0x15, model: 0x13, stepping: 0x1)
[ 0.258615] cblist_init_generic: Setting adjustable number of callback queues.
[ 0.258618] cblist_init_generic: Setting shift to 1 and lim to 1.
[ 0.258648] cblist_init_generic: Setting shift to 1 and lim to 1.
[ 0.258675] cblist_init_generic: Setting shift to 1 and lim to 1.
[ 0.258703] Performance Events: Fam15h core perfctr, AMD PMU driver.
[ 0.258726] ... version: 0
[ 0.258727] ... bit width: 48
[ 0.258728] ... generic registers: 6
[ 0.258729] ... value mask: 0000ffffffffffff
[ 0.258730] ... max period: 00007fffffffffff
[ 0.258731] ... fixed-purpose events: 0
[ 0.258732] ... event mask: 000000000000003f
[ 0.258852] rcu: Hierarchical SRCU implementation.
[ 0.258853] rcu: Max phase no-delay instances is 1000.
[ 0.259441] NMI watchdog: Enabled. Permanently consumes one hw-PMU counter.
[ 0.259515] smp: Bringing up secondary CPUs ...
[ 0.259714] x86: Booting SMP configuration:
[ 0.259715] .... node #0, CPUs: #1
[ 0.121151] Bit 30 in CPUID ECX not set.
[ 0.259844] After schedule_preempt_disabled
[ 0.259849] smp: Brought up 1 node, 2 CPUs
[ 0.259849] smpboot: Max logical packages: 1
[ 0.259849] smpboot: Total of 2 processors activated (15599.81 BogoMIPS)
[ 0.261311] devtmpfs: initialized
[ 0.261311] x86/mm: Memory block size: 128MB
[ 0.262220] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[ 0.262220] futex hash table entries: 512 (order: 3, 32768 bytes, linear)
[ 0.262220] pinctrl core: initialized pinctrl subsystem
[ 0.262220] PM: RTC time: 17:24:09, date: 2023-04-17
[ 0.262220] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[ 0.262434] audit: initializing netlink subsys (disabled)
[ 0.262453] audit: type=2000 audit(1681752249.140:1): state=initialized audit_enabled=0 res=1
[ 0.262453] thermal_sys: Registered thermal governor 'fair_share'
[ 0.262453] thermal_sys: Registered thermal governor 'bang_bang'
[ 0.262453] thermal_sys: Registered thermal governor 'step_wise'
[ 0.262453] thermal_sys: Registered thermal governor 'user_space'
[ 0.262453] cpuidle: using governor ladder
[ 0.262453] cpuidle: using governor menu
[ 0.262453] PCI: MMCONFIG for domain 0000 [bus 00-3f] at [mem 0xf8000000-0xfbffffff] (base 0xf8000000)
[ 0.262453] PCI: MMCONFIG at [mem 0xf8000000-0xfbffffff] reserved as E820 entry
[ 0.262453] PCI: Using configuration type 1 for base access
[ 0.262453] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.
[ 0.273235] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages
[ 0.273235] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page
[ 0.273235] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages
[ 0.273235] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page
[ 0.275428] cryptd: max_cpu_qlen set to 1000
[ 0.275428] ACPI: Added _OSI(Module Device)
[ 0.275428] ACPI: Added _OSI(Processor Device)
[ 0.275428] ACPI: Added _OSI(3.0 _SCP Extensions)
[ 0.275428] ACPI: Added _OSI(Processor Aggregator Device)
[ 0.280879] ACPI: DSDT successfully acquired and loaded
[ 0.281154] ACPI: 4 ACPI AML tables successfully acquired and loaded
[ 0.281649] ACPI: Interpreter enabled
[ 0.281669] ACPI: PM: (supports S0 S1 S3 S5)
[ 0.281671] ACPI: Using IOAPIC for interrupt routing
[ 0.281724] HEST: Table parsing has been initialized.
[ 0.281743] GHES: Failed to enable APEI firmware first mode.
[ 0.281745] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[ 0.281746] PCI: Ignoring E820 reservations for host bridge windows
[ 0.282003] ACPI: Enabled 8 GPEs in block 00 to 1F
[ 0.286735] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[ 0.286746] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]
[ 0.286838] acpi PNP0A03:00: _OSC: OS now controls [PME AER PCIeCapability LTR]
[ 0.286852] acpi PNP0A03:00: [Firmware Info]: MMCONFIG for domain 0000 [bus 00-3f] only partially covers this bridge
[ 0.286942] acpi PNP0A03:00: host bridge window expanded to [io 0x0000-0x0cf7 window]; [io 0x03b0-0x03df window] ignored
[ 0.287197] PCI host bridge to bus 0000:00
[ 0.287199] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]
[ 0.287201] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]
[ 0.287203] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000dffff]
[ 0.287205] pci_bus 0000:00: root bus resource [mem 0x80000000-0xffffffff]
[ 0.287210] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 0.287234] pci 0000:00:00.0: [1022:1410] type 00 class 0x060000
[ 0.287388] pci 0000:00:00.2: [1022:1419] type 00 class 0x080600
[ 0.287479] pci 0000:00:01.0: [1002:9996] type 00 class 0x030000
[ 0.287488] pci 0000:00:01.0: reg 0x10: [mem 0xe0000000-0xefffffff pref]
[ 0.287493] pci 0000:00:01.0: reg 0x14: [io 0x1000-0x10ff]
[ 0.287498] pci 0000:00:01.0: reg 0x18: [mem 0xf0180000-0xf01bffff]
[ 0.287514] pci 0000:00:01.0: enabling Extended Tags
[ 0.287524] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]
[ 0.287541] pci 0000:00:01.0: supports D1 D2
[ 0.287611] pci 0000:00:01.1: [1002:9902] type 00 class 0x040300
[ 0.287618] pci 0000:00:01.1: reg 0x10: [mem 0xf01c0000-0xf01c3fff]
[ 0.287640] pci 0000:00:01.1: enabling Extended Tags
[ 0.287663] pci 0000:00:01.1: supports D1 D2
[ 0.287750] pci 0000:00:11.0: [1022:7801] type 00 class 0x010601
[ 0.287763] pci 0000:00:11.0: reg 0x10: [io 0x1410-0x1417]
[ 0.287771] pci 0000:00:11.0: reg 0x14: [io 0x1420-0x1423]
[ 0.287778] pci 0000:00:11.0: reg 0x18: [io 0x1418-0x141f]
[ 0.287785] pci 0000:00:11.0: reg 0x1c: [io 0x1424-0x1427]
[ 0.287793] pci 0000:00:11.0: reg 0x20: [io 0x1400-0x140f]
[ 0.287800] pci 0000:00:11.0: reg 0x24: [mem 0xf01cc000-0xf01cc7ff]
[ 0.287954] pci 0000:00:12.0: [1022:7807] type 00 class 0x0c0310
[ 0.287967] pci 0000:00:12.0: reg 0x10: [mem 0xf01c8000-0xf01c8fff]
[ 0.288152] pci 0000:00:12.2: [1022:7808] type 00 class 0x0c0320
[ 0.288165] pci 0000:00:12.2: reg 0x10: [mem 0xf01cd000-0xf01cd0ff]
[ 0.288230] pci 0000:00:12.2: supports D1 D2
[ 0.288231] pci 0000:00:12.2: PME# supported from D0 D1 D2 D3hot
[ 0.288233] pci 0000:00:12.2: pme_poll = true
[ 0.288234] pci 0000:00:12.2: after device_set_wakeup_capable()
[ 0.288237] pci 0000:00:12.2: after pci_pme_active()
[ 0.288373] pci 0000:00:13.0: [1022:7807] type 00 class 0x0c0310
[ 0.288387] pci 0000:00:13.0: reg 0x10: [mem 0xf01c9000-0xf01c9fff]
[ 0.288570] pci 0000:00:13.2: [1022:7808] type 00 class 0x0c0320
[ 0.288583] pci 0000:00:13.2: reg 0x10: [mem 0xf01ce000-0xf01ce0ff]
[ 0.288650] pci 0000:00:13.2: supports D1 D2
[ 0.288652] pci 0000:00:13.2: PME# supported from D0 D1 D2 D3hot
[ 0.288653] pci 0000:00:13.2: pme_poll = true
[ 0.288654] pci 0000:00:13.2: after device_set_wakeup_capable()
[ 0.288657] pci 0000:00:13.2: after pci_pme_active()
[ 0.288798] pci 0000:00:14.0: [1022:780b] type 00 class 0x0c0500
[ 0.288980] pci 0000:00:14.2: [1022:780d] type 00 class 0x040300
[ 0.288997] pci 0000:00:14.2: reg 0x10: [mem 0xf01c4000-0xf01c7fff 64bit]
[ 0.289052] pci 0000:00:14.2: PME# supported from D0 D3hot D3cold
[ 0.289054] pci 0000:00:14.2: pme_poll = true
[ 0.289055] pci 0000:00:14.2: after device_set_wakeup_capable()
[ 0.289058] pci 0000:00:14.2: after pci_pme_active()
[ 0.290131] pci 0000:00:14.3: [1022:780e] type 00 class 0x060100
[ 0.290335] pci 0000:00:14.4: [1022:780f] type 01 class 0x060401
[ 0.290490] pci 0000:00:14.5: [1022:7809] type 00 class 0x0c0310
[ 0.290504] pci 0000:00:14.5: reg 0x10: [mem 0xf01ca000-0xf01cafff]
[ 0.290677] pci 0000:00:15.0: [1022:43a0] type 01 class 0x060400
[ 0.290705] pci 0000:00:15.0: enabling Extended Tags
[ 0.290746] pci 0000:00:15.0: supports D1 D2
[ 0.290907] pci 0000:00:15.1: [1022:43a1] type 01 class 0x060400
[ 0.290939] pci 0000:00:15.1: enabling Extended Tags
[ 0.290978] pci 0000:00:15.1: supports D1 D2
[ 0.291146] pci 0000:00:15.2: [1022:43a2] type 01 class 0x060400
[ 0.291174] pci 0000:00:15.2: enabling Extended Tags
[ 0.291214] pci 0000:00:15.2: supports D1 D2
[ 0.291289] pci 0000:00:16.0: [1022:7807] type 00 class 0x0c0310
[ 0.291303] pci 0000:00:16.0: reg 0x10: [mem 0xf01cb000-0xf01cbfff]
[ 0.291473] pci 0000:00:16.2: [1022:7808] type 00 class 0x0c0320
[ 0.291486] pci 0000:00:16.2: reg 0x10: [mem 0xf01cf000-0xf01cf0ff]
[ 0.291551] pci 0000:00:16.2: supports D1 D2
[ 0.291552] pci 0000:00:16.2: PME# supported from D0 D1 D2 D3hot
[ 0.291554] pci 0000:00:16.2: pme_poll = true
[ 0.291555] pci 0000:00:16.2: after device_set_wakeup_capable()
[ 0.291558] pci 0000:00:16.2: after pci_pme_active()
[ 0.291684] pci 0000:00:18.0: [1022:1400] type 00 class 0x060000
[ 0.291748] pci 0000:00:18.1: [1022:1401] type 00 class 0x060000
[ 0.291806] pci 0000:00:18.2: [1022:1402] type 00 class 0x060000
[ 0.291866] pci 0000:00:18.3: [1022:1403] type 00 class 0x060000
[ 0.291998] pci 0000:00:18.4: [1022:1404] type 00 class 0x060000
[ 0.292064] pci 0000:00:18.5: [1022:1405] type 00 class 0x060000
[ 0.292136] pci_bus 0000:01: extended config space not accessible
[ 0.292200] pci 0000:00:14.4: PCI bridge to [bus 01] (subtractive decode)
[ 0.292208] pci 0000:00:14.4: bridge window [io 0x0000-0x0cf7 window] (subtractive decode)
[ 0.292211] pci 0000:00:14.4: bridge window [io 0x0d00-0xffff window] (subtractive decode)
[ 0.292214] pci 0000:00:14.4: bridge window [mem 0x000a0000-0x000dffff] (subtractive decode)
[ 0.292216] pci 0000:00:14.4: bridge window [mem 0x80000000-0xffffffff] (subtractive decode)
[ 0.292261] pci 0000:00:15.0: PCI bridge to [bus 02]
[ 0.292345] pci 0000:03:00.0: [1b21:1042] type 00 class 0x0c0330
[ 0.292382] pci 0000:03:00.0: reg 0x10: [mem 0xf0000000-0xf0007fff 64bit]
[ 0.292559] pci 0000:03:00.0: PME# supported from D3hot D3cold
[ 0.292560] pci 0000:03:00.0: pme_poll = true
[ 0.292562] pci 0000:03:00.0: after device_set_wakeup_capable()
[ 0.292567] pci 0000:03:00.0: after pci_pme_active()
[ 0.292605] pci 0000:03:00.0: 2.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s PCIe x1 link at 0000:00:15.1 (capable of 4.000 Gb/s with 5.0 GT/s PCIe x1 link)
[ 0.305215] pci 0000:00:15.1: PCI bridge to [bus 03]
[ 0.305228] pci 0000:00:15.1: bridge window [mem 0xf0000000-0xf00fffff]
[ 0.305237] pci 0000:00:15.2: bridge configuration invalid ([bus 00-00]), reconfiguring
[ 0.305359] pci 0000:04:00.0: [10ec:8168] type 00 class 0x020000
[ 0.305378] pci 0000:04:00.0: reg 0x10: [io 0x0000-0x00ff]
[ 0.305399] pci 0000:04:00.0: reg 0x18: [mem 0x00000000-0x00000fff 64bit pref]
[ 0.305413] pci 0000:04:00.0: reg 0x20: [mem 0x00000000-0x00003fff 64bit pref]
[ 0.305520] pci 0000:04:00.0: supports D1 D2
[ 0.305522] pci 0000:04:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[ 0.305524] pci 0000:04:00.0: pme_poll = true
[ 0.305525] pci 0000:04:00.0: after device_set_wakeup_capable()
[ 0.305529] pci 0000:04:00.0: after pci_pme_active()
[ 0.321217] pci 0000:00:15.2: PCI bridge to [bus 04-ff]
[ 0.321228] pci 0000:00:15.2: bridge window [io 0x0000-0x0fff]
[ 0.321232] pci 0000:00:15.2: bridge window [mem 0x00000000-0x000fffff]
[ 0.321237] pci 0000:00:15.2: bridge window [mem 0x00000000-0x000fffff 64bit pref]
[ 0.321240] pci_bus 0000:04: busn_res: [bus 04-ff] end is updated to 04
[ 0.321759] ACPI: PCI: Interrupt link INTA configured for IRQ 0
[ 0.321855] ACPI: PCI: Interrupt link INTB configured for IRQ 0
[ 0.321949] ACPI: PCI: Interrupt link INTC configured for IRQ 0
[ 0.322041] ACPI: PCI: Interrupt link INTD configured for IRQ 0
[ 0.322134] ACPI: PCI: Interrupt link INTE configured for IRQ 0
[ 0.322232] ACPI: PCI: Interrupt link INTF configured for IRQ 0
[ 0.322325] ACPI: PCI: Interrupt link INTG configured for IRQ 0
[ 0.322418] ACPI: PCI: Interrupt link INTH configured for IRQ 0
[ 0.322640] iommu: Default domain type: Translated
[ 0.322642] iommu: DMA domain TLB invalidation policy: lazy mode
[ 0.322839] SCSI subsystem initialized
[ 0.325159] libata version 3.00 loaded.
[ 0.325159] ACPI: bus type USB registered
[ 0.325159] usbcore: registered new interface driver usbfs
[ 0.325159] usbcore: registered new interface driver hub
[ 0.325159] usbcore: registered new device driver usb
[ 0.325159] PCI: Using ACPI for IRQ routing
[ 0.325159] PCI: pci_cache_line_size set to 64 bytes
[ 0.325159] e820: reserve RAM buffer [mem 0x0009fc00-0x0009ffff]
[ 0.325159] e820: reserve RAM buffer [mem 0x5fe4d000-0x5fffffff]
[ 0.325159] e820: reserve RAM buffer [mem 0x17f000000-0x17fffffff]
[ 0.325159] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0
[ 0.325159] hpet0: 3 comparators, 32-bit 14.318180 MHz counter
[ 0.330232] clocksource: Switched to clocksource tsc-early
[ 0.330474] VFS: Disk quotas dquot_6.6.0
[ 0.330500] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[ 0.330620] pnp: PnP ACPI init
[ 0.330969] system 00:00: [mem 0xfec10002-0xfec11001] could not be reserved
[ 0.331362] pnp: PnP ACPI: found 2 devices
[ 0.337986] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
[ 0.338186] NET: Registered PF_INET protocol family
[ 0.338359] IP idents hash table entries: 65536 (order: 7, 524288 bytes, linear)
[ 0.340153] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear)
[ 0.340169] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
[ 0.340177] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear)
[ 0.340251] TCP bind hash table entries: 32768 (order: 8, 1048576 bytes, linear)
[ 0.340614] TCP: Hash tables configured (established 32768 bind 32768)
[ 0.340685] UDP hash table entries: 2048 (order: 4, 65536 bytes, linear)
[ 0.340711] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear)
[ 0.340827] NET: Registered PF_UNIX/PF_LOCAL protocol family
[ 0.340862] pci 0000:00:15.2: BAR 15: assigned [mem 0x80000000-0x800fffff 64bit pref]
[ 0.340869] pci 0000:00:15.2: BAR 13: assigned [io 0x2000-0x2fff]
[ 0.340874] pci 0000:00:14.4: PCI bridge to [bus 01]
[ 0.340886] pci 0000:00:15.0: PCI bridge to [bus 02]
[ 0.340894] pci 0000:00:15.1: PCI bridge to [bus 03]
[ 0.340898] pci 0000:00:15.1: bridge window [mem 0xf0000000-0xf00fffff]
[ 0.340908] pci 0000:04:00.0: BAR 4: assigned [mem 0x80000000-0x80003fff 64bit pref]
[ 0.340922] pci 0000:04:00.0: BAR 2: assigned [mem 0x80004000-0x80004fff 64bit pref]
[ 0.340934] pci 0000:04:00.0: BAR 0: assigned [io 0x2000-0x20ff]
[ 0.340941] pci 0000:00:15.2: PCI bridge to [bus 04]
[ 0.340943] pci 0000:00:15.2: bridge window [io 0x2000-0x2fff]
[ 0.340949] pci 0000:00:15.2: bridge window [mem 0x80000000-0x800fffff 64bit pref]
[ 0.340956] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]
[ 0.340958] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]
[ 0.340960] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000dffff]
[ 0.340964] pci_bus 0000:00: resource 7 [mem 0x80000000-0xffffffff]
[ 0.340966] pci_bus 0000:01: resource 4 [io 0x0000-0x0cf7 window]
[ 0.340968] pci_bus 0000:01: resource 5 [io 0x0d00-0xffff window]
[ 0.340970] pci_bus 0000:01: resource 6 [mem 0x000a0000-0x000dffff]
[ 0.340973] pci_bus 0000:01: resource 7 [mem 0x80000000-0xffffffff]
[ 0.340975] pci_bus 0000:03: resource 1 [mem 0xf0000000-0xf00fffff]
[ 0.340977] pci_bus 0000:04: resource 0 [io 0x2000-0x2fff]
[ 0.340979] pci_bus 0000:04: resource 2 [mem 0x80000000-0x800fffff 64bit pref]
[ 0.341104] pci 0000:00:01.1: D0 power state depends on 0000:00:01.0
[ 0.341412] pci 0000:00:12.0: AMD USB device
[ 0.341436] pci 0000:00:12.0: AMD USB ohci handoff
[ 0.341833] pci 0000:00:12.2: AMD USB device
[ 0.341847] pci 0000:00:12.2: AMD USB ehci handoff
[ 0.342032] pci 0000:00:12.2: PME# does not work under D3, disabling it
[ 0.342235] pci 0000:00:13.0: AMD USB device
[ 0.342250] pci 0000:00:13.0: AMD USB ohci handoff
[ 0.342633] pci 0000:00:13.2: AMD USB device
[ 0.342650] pci 0000:00:13.2: AMD USB ehci handoff
[ 0.342830] pci 0000:00:13.2: PME# does not work under D3, disabling it
[ 0.343041] pci 0000:00:14.5: AMD USB device
[ 0.343057] pci 0000:00:14.5: AMD USB ohci handoff
[ 0.343577] pci 0000:00:16.0: AMD USB device
[ 0.343598] pci 0000:00:16.0: AMD USB ohci handoff
[ 0.343993] pci 0000:00:16.2: AMD USB device
[ 0.344008] pci 0000:00:16.2: AMD USB ehci handoff
[ 0.344202] pci 0000:00:16.2: PME# does not work under D3, disabling it
[ 0.344510] pci 0000:03:00.0: AMD USB xhci handoff
[ 0.344577] PCI: CLS 64 bytes, default 64
[ 0.344678] pci 0000:00:00.2: AMD-Vi: Applying erratum 746 workaround
[ 0.344790] pci 0000:00:01.0: Adding to iommu group 0
[ 0.344813] pci 0000:00:01.1: Adding to iommu group 0
[ 0.344844] pci 0000:00:11.0: Adding to iommu group 1
[ 0.344885] pci 0000:00:12.0: Adding to iommu group 2
[ 0.344905] pci 0000:00:12.2: Adding to iommu group 2
[ 0.344949] pci 0000:00:13.0: Adding to iommu group 3
[ 0.344970] pci 0000:00:13.2: Adding to iommu group 3
[ 0.345020] pci 0000:00:14.0: Adding to iommu group 4
[ 0.345043] pci 0000:00:14.2: Adding to iommu group 4
[ 0.345066] pci 0000:00:14.3: Adding to iommu group 4
[ 0.345102] pci 0000:00:14.4: Adding to iommu group 5
[ 0.345128] pci 0000:00:14.5: Adding to iommu group 6
[ 0.345172] pci 0000:00:15.0: Adding to iommu group 7
[ 0.345198] pci 0000:00:15.1: Adding to iommu group 7
[ 0.345219] pci 0000:00:15.2: Adding to iommu group 7
[ 0.345261] pci 0000:00:16.0: Adding to iommu group 8
[ 0.345285] pci 0000:00:16.2: Adding to iommu group 8
[ 0.345349] pci 0000:00:18.0: Adding to iommu group 9
[ 0.345374] pci 0000:00:18.1: Adding to iommu group 9
[ 0.345400] pci 0000:00:18.2: Adding to iommu group 9
[ 0.345428] pci 0000:00:18.3: Adding to iommu group 9
[ 0.345451] pci 0000:00:18.4: Adding to iommu group 9
[ 0.345479] pci 0000:00:18.5: Adding to iommu group 9
[ 0.345489] pci 0000:03:00.0: Adding to iommu group 7
[ 0.345499] pci 0000:04:00.0: Adding to iommu group 7
[ 0.348265] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[ 0.348272] AMD-Vi: Extended features (0x800000853, 0x0): PreF PPR GT IA
[ 0.348279] AMD-Vi: Interrupt remapping enabled
[ 0.348484] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[ 0.348486] software IO TLB: mapped [mem 0x000000005be4d000-0x000000005fe4d000] (64MB)
[ 0.348547] LVT offset 0 assigned for vector 0x400
[ 0.348595] perf: AMD IBS detected (0x000000ff)
[ 0.348602] amd_uncore: 4 amd_nb counters detected
[ 0.352225] workingset: timestamp_bits=37 max_order=20 bucket_order=0
[ 0.352259] zbud: loaded
[ 0.352752] NET: Registered PF_ALG protocol family
[ 0.352758] Key type asymmetric registered
[ 0.352759] Asymmetric key parser 'x509' registered
[ 0.353113] alg: self-tests disabled
[ 0.353209] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 251)
[ 0.353263] io scheduler mq-deadline registered
[ 0.353265] io scheduler kyber registered
[ 0.354479] pcieport 0000:00:15.0: PME: Signaling with IRQ 25
[ 0.354644] pcieport 0000:00:15.1: PME: Signaling with IRQ 26
[ 0.354712] pcieport 0000:00:15.2: enabling device (0000 -> 0003)
[ 0.354941] pcieport 0000:00:15.2: PME: Signaling with IRQ 27
[ 0.355211] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
[ 0.355298] ACPI: button: Power Button [PWRF]
[ 0.355363] ACPI: \_SB_.P000: Found 2 idle states
[ 0.355495] ACPI: \_SB_.P001: Found 2 idle states
[ 0.356442] thermal LNXTHERM:00: registered as thermal_zone0
[ 0.356445] ACPI: thermal: Thermal Zone [TZ00] (21 C)
[ 0.356797] Non-volatile memory driver v1.3
[ 0.356864] AMD-Vi: AMD IOMMUv2 loaded and initialized
[ 0.357065] ahci 0000:00:11.0: version 3.0
[ 0.357356] ahci 0000:00:11.0: AHCI 0001.0300 32 slots 8 ports 6 Gbps 0x40 impl SATA mode
[ 0.357360] ahci 0000:00:11.0: flags: 64bit ncq sntf ilck led clo pio
[ 0.358785] scsi host0: ahci
[ 0.358999] scsi host1: ahci
[ 0.359215] scsi host2: ahci
[ 0.359424] scsi host3: ahci
[ 0.359624] scsi host4: ahci
[ 0.359836] scsi host5: ahci
[ 0.360048] scsi host6: ahci
[ 0.360243] scsi host7: ahci
[ 0.360336] ata port1: DUMMY
[ 0.360338] ata port2: DUMMY
[ 0.360339] ata port3: DUMMY
[ 0.360341] ata port4: DUMMY
[ 0.360342] ata port5: DUMMY
[ 0.360343] ata port6: DUMMY
[ 0.360345] ata port7: SATA max UDMA/133 abar m2048@0xf01cc000 port 0xf01cc400 irq 19
[ 0.360347] ata port8: DUMMY
[ 0.360431] ACPI: bus type drm_connector registered
[ 0.360734] i8042: PNP: No PS/2 controller found.
[ 0.360736] i8042: Probing ports directly.
[ 0.363078] serio: i8042 KBD port at 0x60,0x64 irq 1
[ 0.363185] serio: i8042 AUX port at 0x60,0x64 irq 12
[ 0.363347] mousedev: PS/2 mouse device common for all mice
[ 0.363420] rtc_cmos 00:01: RTC can wake from S4
[ 0.363924] rtc_cmos 00:01: registered as rtc0
[ 0.363950] rtc_cmos 00:01: setting system clock to 2023-04-17T17:24:09 UTC (1681752249)
[ 0.364011] rtc_cmos 00:01: alarms up to one day, y3k, 114 bytes nvram, hpet irqs
[ 0.364048] device-mapper: uevent: version 1.0.3
[ 0.364136] device-mapper: ioctl: 4.47.0-ioctl (2022-07-28) initialised: dm-devel@redhat.com
[ 0.364301] hid: raw HID events driver (C) Jiri Kosina
[ 0.364335] usbcore: registered new interface driver usbhid
[ 0.364336] usbhid: USB HID core driver
[ 0.364427] Initializing XFRM netlink socket
[ 0.364436] NET: Registered PF_PACKET protocol family
[ 0.364438] x86/pm: family 0x15 cpu detected, MSR saving is needed during suspending.
[ 0.364652] microcode: CPU1: patch_level=0x0600111f
[ 0.364652] microcode: CPU0: patch_level=0x0600111f
[ 0.364664] microcode: Microcode Update Driver: v2.2.
[ 0.364668] IPI shorthand broadcast: enabled
[ 0.364678] AVX version of gcm_enc/dec engaged.
[ 0.364710] AES CTR mode by8 optimization enabled
[ 0.368578] sched_clock: Marking stable (249972593, 117151517)->(369867515, -2743405)
[ 0.368823] registered taskstats version 1
[ 0.369080] zswap: loaded using pool lzo/zbud
[ 0.371663] ata link7: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[ 0.371939] ata dev7.0: ATA-9: SanDisk SDSSDP064G, 2.0.0, max UDMA/133
[ 0.371942] ata dev7.0: 125045424 sectors, multi 1: LBA48 NCQ (depth 32)
[ 0.372148] ata dev7.0: configured for UDMA/133
[ 0.372283] scsi 6:0:0:0: Direct-Access ATA SanDisk SDSSDP06 0 PQ: 0 ANSI: 5
[ 0.373496] kmemleak: Kernel memory leak detector initialized (mem pool available: 15679)
[ 0.373501] debug_vm_pgtable: [debug_vm_pgtable ]: Validating architecture page table helpers
[ 0.373959] sd 6:0:0:0: [sda] 125045424 512-byte logical blocks: (64.0 GB/59.6 GiB)
[ 0.373982] sd 6:0:0:0: [sda] Write Protect is off
[ 0.373987] sd 6:0:0:0: [sda] Mode Sense: 00 3a 00 00
[ 0.374015] sd 6:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 0.374059] sd 6:0:0:0: [sda] Preferred minimum I/O size 512 bytes
[ 0.380407] kmemleak: Automatic memory scanning thread started
[ 0.380633] sda: sda1 sda2 sda3
[ 0.381120] Key type encrypted registered
[ 0.381248] sd 6:0:0:0: [sda] Attached SCSI disk
[ 0.384294] PM: Magic number: 3:443:441
[ 0.396721] EXT4-fs (sda3): mounted filesystem fe29e0dc-6303-4401-987c-8472bc1b9516 with ordered data mode. Quota mode: none.
[ 0.396766] VFS: Mounted root (ext4 filesystem) on device 8:3.
[ 0.398650] devtmpfs: mounted
[ 0.398668] After kernel_init_freeable
[ 0.403170] Freeing unused kernel image (initmem) memory: 2908K
[ 0.415639] Write protecting the kernel read-only data: 20480k
[ 0.415922] Freeing unused kernel image (rodata/data gap) memory: 836K
[ 0.453024] x86/mm: Checked W+X mappings: passed, no W+X pages found.
[ 0.453030] rodata_test: all tests were successful
[ 0.453031] After mark_readonly
[ 0.453031] After pti_finalize
[ 0.453045] rcu_end_inkernel_boot
[ 0.453054] Run /sbin/init as init process
[ 0.453055] with arguments:
[ 0.453057] /sbin/init
[ 0.453058] noisapnp
[ 0.453059] with environment:
[ 0.453060] HOME=/
[ 0.453060] TERM=linux
[ 0.453061] BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc6-00311-gde8224969f66
[ 0.629518] systemd[1]: Inserted module 'autofs4'
[ 0.656601] NET: Registered PF_INET6 protocol family
[ 0.657495] Segment Routing with IPv6
[ 0.657523] In-situ OAM (IOAM) with IPv6
[ 0.683745] systemd[1]: systemd 252.6-1 running in system mode (+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
[ 0.683756] systemd[1]: Detected architecture x86-64.
[ 0.688485] systemd[1]: Hostname set to <kodi>.
[ 0.959071] systemd[1]: Queued start job for default target graphical.target.
[ 0.982198] systemd[1]: Created slice system-getty.slice - Slice /system/getty.
[ 0.983286] systemd[1]: Created slice system-modprobe.slice - Slice /system/modprobe.
[ 0.984123] systemd[1]: Created slice user.slice - User and Session Slice.
[ 0.984302] systemd[1]: Started systemd-ask-password-console.path - Dispatch Password Requests to Console Directory Watch.
[ 0.984417] systemd[1]: Started systemd-ask-password-wall.path - Forward Password Requests to Wall Directory Watch.
[ 0.984855] systemd[1]: Set up automount proc-sys-fs-binfmt_misc.automount - Arbitrary Executable File Formats File System Automount Point.
[ 0.984894] systemd[1]: Reached target cryptsetup.target - Local Encrypted Volumes.
[ 0.984934] systemd[1]: Reached target integritysetup.target - Local Integrity Protected Volumes.
[ 0.984972] systemd[1]: Reached target paths.target - Path Units.
[ 0.985006] systemd[1]: Reached target remote-fs.target - Remote File Systems.
[ 0.985038] systemd[1]: Reached target slices.target - Slice Units.
[ 0.985078] systemd[1]: Reached target swap.target - Swaps.
[ 0.985116] systemd[1]: Reached target veritysetup.target - Local Verity Protected Volumes.
[ 0.987550] systemd[1]: Listening on systemd-coredump.socket - Process Core Dump Socket.
[ 0.987802] systemd[1]: Listening on systemd-fsckd.socket - fsck to fsckd communication Socket.
[ 0.987975] systemd[1]: Listening on systemd-initctl.socket - initctl Compatibility Named Pipe.
[ 0.988274] systemd[1]: Listening on systemd-journald-audit.socket - Journal Audit Socket.
[ 0.988543] systemd[1]: Listening on systemd-journald-dev-log.socket - Journal Socket (/dev/log).
[ 0.988822] systemd[1]: Listening on systemd-journald.socket - Journal Socket.
[ 0.989072] systemd[1]: Listening on systemd-networkd.socket - Network Service Netlink Socket.
[ 0.989946] systemd[1]: Listening on systemd-udevd-control.socket - udev Control Socket.
[ 0.990210] systemd[1]: Listening on systemd-udevd-kernel.socket - udev Kernel Socket.
[ 0.992904] systemd[1]: Mounting dev-hugepages.mount - Huge Pages File System...
[ 0.995944] systemd[1]: Mounting dev-mqueue.mount - POSIX Message Queue File System...
[ 0.999142] systemd[1]: Mounting sys-kernel-debug.mount - Kernel Debug File System...
[ 1.002682] systemd[1]: Mounting sys-kernel-tracing.mount - Kernel Trace File System...
[ 1.007256] systemd[1]: Starting kmod-static-nodes.service - Create List of Static Device Nodes...
[ 1.014276] systemd[1]: Starting modprobe@configfs.service - Load Kernel Module configfs...
[ 1.017399] systemd[1]: Starting modprobe@dm_mod.service - Load Kernel Module dm_mod...
[ 1.020657] systemd[1]: Starting modprobe@drm.service - Load Kernel Module drm...
[ 1.024006] systemd[1]: Starting modprobe@efi_pstore.service - Load Kernel Module efi_pstore...
[ 1.027278] systemd[1]: Starting modprobe@fuse.service - Load Kernel Module fuse...
[ 1.036066] systemd[1]: Starting modprobe@loop.service - Load Kernel Module loop...
[ 1.036151] systemd[1]: systemd-firstboot.service - First Boot Wizard was skipped because of an unmet condition check (ConditionFirstBoot=yes).
[ 1.036237] systemd[1]: systemd-fsck-root.service - File System Check on Root Device was skipped because of an unmet condition check (ConditionPathIsReadWrite=!/).
[ 1.036279] systemd[1]: Reached target local-fs.target - Local File Systems.
[ 1.036362] systemd[1]: apparmor.service - Load AppArmor profiles was skipped because of an unmet condition check (ConditionSecurity=apparmor).
[ 1.041356] loop: module loaded
[ 1.043749] systemd[1]: Starting systemd-binfmt.service - Set Up Additional Binary Formats...
[ 1.048983] systemd[1]: Starting systemd-journald.service - Journal Service...
[ 1.050740] fuse: init (API version 7.38)
[ 1.052824] systemd[1]: Starting systemd-random-seed.service - Load/Save Random Seed...
[ 1.063924] systemd[1]: Starting systemd-sysctl.service - Apply Kernel Variables...
[ 1.068061] systemd[1]: Starting systemd-sysusers.service - Create System Users...
[ 1.071328] systemd[1]: Starting systemd-udev-trigger.service - Coldplug All udev Devices...
[ 1.093517] systemd[1]: Mounted dev-hugepages.mount - Huge Pages File System.
[ 1.094134] systemd[1]: Mounted dev-mqueue.mount - POSIX Message Queue File System.
[ 1.094447] systemd[1]: Mounted sys-kernel-debug.mount - Kernel Debug File System.
[ 1.094985] systemd[1]: Mounted sys-kernel-tracing.mount - Kernel Trace File System.
[ 1.095974] systemd[1]: Finished kmod-static-nodes.service - Create List of Static Device Nodes.
[ 1.096888] systemd[1]: modprobe@configfs.service: Deactivated successfully.
[ 1.097257] systemd[1]: Finished modprobe@configfs.service - Load Kernel Module configfs.
[ 1.098001] systemd[1]: modprobe@dm_mod.service: Deactivated successfully.
[ 1.098339] systemd[1]: Finished modprobe@dm_mod.service - Load Kernel Module dm_mod.
[ 1.099077] systemd[1]: modprobe@drm.service: Deactivated successfully.
[ 1.099446] systemd[1]: Finished modprobe@drm.service - Load Kernel Module drm.
[ 1.100518] systemd[1]: modprobe@efi_pstore.service: Deactivated successfully.
[ 1.100846] systemd[1]: Finished modprobe@efi_pstore.service - Load Kernel Module efi_pstore.
[ 1.101749] systemd[1]: modprobe@fuse.service: Deactivated successfully.
[ 1.102064] systemd[1]: Finished modprobe@fuse.service - Load Kernel Module fuse.
[ 1.102810] systemd[1]: modprobe@loop.service: Deactivated successfully.
[ 1.103142] systemd[1]: Finished modprobe@loop.service - Load Kernel Module loop.
[ 1.103830] systemd[1]: proc-sys-fs-binfmt_misc.automount: Got automount request for /proc/sys/fs/binfmt_misc, triggered by 138 (systemd-binfmt)
[ 1.123242] systemd[1]: Mounting proc-sys-fs-binfmt_misc.mount - Arbitrary Executable File Formats File System...
[ 1.140028] systemd[1]: Mounting sys-fs-fuse-connections.mount - FUSE Control File System...
[ 1.148223] systemd[1]: Mounting sys-kernel-config.mount - Kernel Configuration File System...
[ 1.148346] systemd[1]: systemd-pstore.service - Platform Persistent Storage Archival was skipped because of an unmet condition check (ConditionDirectoryNotEmpty=/sys/fs/pstore).
[ 1.148515] systemd[1]: systemd-repart.service - Repartition Root Disk was skipped because no trigger condition checks were met.
[ 1.159909] systemd[1]: Finished systemd-sysctl.service - Apply Kernel Variables.
[ 1.184529] systemd[1]: Finished systemd-sysusers.service - Create System Users.
[ 1.186000] systemd[1]: Mounted proc-sys-fs-binfmt_misc.mount - Arbitrary Executable File Formats File System.
[ 1.186357] systemd[1]: Mounted sys-fs-fuse-connections.mount - FUSE Control File System.
[ 1.186600] systemd[1]: Mounted sys-kernel-config.mount - Kernel Configuration File System.
[ 1.209823] systemd[1]: Starting systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev...
[ 1.210512] systemd[1]: Finished systemd-binfmt.service - Set Up Additional Binary Formats.
[ 1.254273] systemd[1]: Finished systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev.
[ 1.273469] systemd[1]: Starting systemd-udevd.service - Rule-based Manager for Device Events and Files...
[ 1.313286] systemd[1]: Started systemd-journald.service - Journal Service.
[ 1.359636] systemd-journald[139]: Received client request to flush runtime journal.
[ 1.369173] tsc: Refined TSC clocksource calibration: 3900.223 MHz
[ 1.369184] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x70705a6472c, max_idle_ns: 881590586812 ns
[ 1.369200] clocksource: Switched to clocksource tsc
[ 1.594559] sd 6:0:0:0: Attached scsi generic sg0 type 0
[ 1.731704] acpi_cpufreq: overriding BIOS provided _PSD data
[ 1.953174] random: crng init done
[ 1.992654] QUIRK: Enable AMD PLL fix
[ 1.992706] ehci-pci 0000:00:12.2: EHCI Host Controller
[ 1.992738] ehci-pci 0000:00:12.2: new USB bus registered, assigned bus number 1
[ 1.992751] ehci-pci 0000:00:12.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
[ 1.992760] ehci-pci 0000:00:12.2: debug port 1
[ 1.992971] ehci-pci 0000:00:12.2: irq 17, io mem 0xf01cd000
[ 2.005197] ehci-pci 0000:00:12.2: USB 2.0 started, EHCI 1.00
[ 2.005468] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 2.005471] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.005474] usb usb1: Product: EHCI Host Controller
[ 2.005476] usb usb1: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ehci_hcd
[ 2.005478] usb usb1: SerialNumber: 0000:00:12.2
[ 2.006279] hub 1-0:1.0: USB hub found
[ 2.006321] hub 1-0:1.0: 5 ports detected
[ 2.007319] ehci-pci 0000:00:13.2: EHCI Host Controller
[ 2.007345] ehci-pci 0000:00:13.2: new USB bus registered, assigned bus number 2
[ 2.007360] ehci-pci 0000:00:13.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
[ 2.007369] ehci-pci 0000:00:13.2: debug port 1
[ 2.007523] ehci-pci 0000:00:13.2: irq 17, io mem 0xf01ce000
[ 2.016141] piix4_smbus 0000:00:14.0: SMBus Host Controller at 0xb00, revision 0
[ 2.016147] piix4_smbus 0000:00:14.0: Using register 0x2e for SMBus port selection
[ 2.016788] piix4_smbus 0000:00:14.0: Auxiliary SMBus Host Controller at 0xb20
[ 2.021184] ehci-pci 0000:00:13.2: USB 2.0 started, EHCI 1.00
[ 2.021523] usb usb2: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 2.021527] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.021529] usb usb2: Product: EHCI Host Controller
[ 2.021531] usb usb2: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ehci_hcd
[ 2.021533] usb usb2: SerialNumber: 0000:00:13.2
[ 2.032086] hub 2-0:1.0: USB hub found
[ 2.032171] hub 2-0:1.0: 5 ports detected
[ 2.033420] ehci-pci 0000:00:16.2: EHCI Host Controller
[ 2.033455] ehci-pci 0000:00:16.2: new USB bus registered, assigned bus number 3
[ 2.033470] ehci-pci 0000:00:16.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
[ 2.033479] ehci-pci 0000:00:16.2: debug port 1
[ 2.033641] ehci-pci 0000:00:16.2: irq 17, io mem 0xf01cf000
[ 2.049177] ehci-pci 0000:00:16.2: USB 2.0 started, EHCI 1.00
[ 2.049454] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 2.049459] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.049461] usb usb3: Product: EHCI Host Controller
[ 2.049463] usb usb3: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ehci_hcd
[ 2.049465] usb usb3: SerialNumber: 0000:00:16.2
[ 2.050025] hub 3-0:1.0: USB hub found
[ 2.050070] hub 3-0:1.0: 4 ports detected
[ 2.050881] ohci-pci 0000:00:16.0: OHCI PCI host controller
[ 2.050910] ohci-pci 0000:00:16.0: new USB bus registered, assigned bus number 4
[ 2.050999] ohci-pci 0000:00:12.0: OHCI PCI host controller
[ 2.051021] ohci-pci 0000:00:12.0: new USB bus registered, assigned bus number 5
[ 2.051110] ohci-pci 0000:00:16.0: irq 18, io mem 0xf01cb000
[ 2.051111] ohci-pci 0000:00:12.0: irq 18, io mem 0xf01c8000
[ 2.051120] ohci-pci 0000:00:13.0: OHCI PCI host controller
[ 2.051141] ohci-pci 0000:00:13.0: new USB bus registered, assigned bus number 6
[ 2.051240] ohci-pci 0000:00:13.0: irq 18, io mem 0xf01c9000
[ 2.051250] ohci-pci 0000:00:14.5: OHCI PCI host controller
[ 2.051268] ohci-pci 0000:00:14.5: new USB bus registered, assigned bus number 7
[ 2.051356] ohci-pci 0000:00:14.5: irq 18, io mem 0xf01ca000
[ 2.113579] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 2.113586] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.113589] usb usb4: Product: OHCI PCI host controller
[ 2.113591] usb usb4: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ohci_hcd
[ 2.113593] usb usb4: SerialNumber: 0000:00:16.0
[ 2.114096] hub 4-0:1.0: USB hub found
[ 2.114127] hub 4-0:1.0: 4 ports detected
[ 2.126194] usb usb5: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 2.126202] usb usb5: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.126204] usb usb5: Product: OHCI PCI host controller
[ 2.126207] usb usb5: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ohci_hcd
[ 2.126209] usb usb5: SerialNumber: 0000:00:12.0
[ 2.127341] hub 5-0:1.0: USB hub found
[ 2.127380] hub 5-0:1.0: 5 ports detected
[ 2.128522] usb usb6: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 2.128528] usb usb6: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.128531] usb usb6: Product: OHCI PCI host controller
[ 2.128533] usb usb6: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ohci_hcd
[ 2.128535] usb usb6: SerialNumber: 0000:00:13.0
[ 2.129049] hub 6-0:1.0: USB hub found
[ 2.129082] hub 6-0:1.0: 5 ports detected
[ 2.130311] usb usb7: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 2.130316] usb usb7: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.130319] usb usb7: Product: OHCI PCI host controller
[ 2.130321] usb usb7: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 ohci_hcd
[ 2.130323] usb usb7: SerialNumber: 0000:00:14.5
[ 2.130855] hub 7-0:1.0: USB hub found
[ 2.130895] hub 7-0:1.0: 2 ports detected
[ 2.148137] 1
[ 2.148196] 2
[ 2.148820] snd_hda_intel 0000:00:01.1: Force to non-snoop mode
[ 2.148832] 3
[ 2.148833] 4
[ 2.148834] 5
[ 2.148835] 7
[ 2.148837] 8
[ 2.148837] 9
[ 2.148929] 1
[ 2.148979] 2
[ 2.149637] 3
[ 2.149641] 4
[ 2.149642] 5
[ 2.149643] 7
[ 2.149645] 8
[ 2.149646] 9
[ 2.166674] xhci_hcd 0000:03:00.0: xHCI Host Controller
[ 2.166707] xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 8
[ 2.206853] input: HDA ATI HDMI HDMI/DP,pcm=3 as /devices/pci0000:00/0000:00:01.1/sound/card0/input1
[ 2.208133] input: HDA ATI HDMI HDMI/DP,pcm=7 as /devices/pci0000:00/0000:00:01.1/sound/card0/input2
[ 2.221615] snd_hda_codec_realtek hdaudioC1D0: ALC892: SKU not ready 0x00000100
[ 2.222377] snd_hda_codec_realtek hdaudioC1D0: autoconfig for ALC892: line_outs=4 (0x14/0x16/0x15/0x17/0x0) type:line
[ 2.222384] snd_hda_codec_realtek hdaudioC1D0: speaker_outs=0 (0x0/0x0/0x0/0x0/0x0)
[ 2.222387] snd_hda_codec_realtek hdaudioC1D0: hp_outs=1 (0x1b/0x0/0x0/0x0/0x0)
[ 2.222390] snd_hda_codec_realtek hdaudioC1D0: mono: mono_out=0x0
[ 2.222392] snd_hda_codec_realtek hdaudioC1D0: dig-out=0x1e/0x0
[ 2.222393] snd_hda_codec_realtek hdaudioC1D0: inputs:
[ 2.222395] snd_hda_codec_realtek hdaudioC1D0: Rear Mic=0x18
[ 2.222398] snd_hda_codec_realtek hdaudioC1D0: Front Mic=0x19
[ 2.222399] snd_hda_codec_realtek hdaudioC1D0: Line=0x1a
[ 2.222401] snd_hda_codec_realtek hdaudioC1D0: CD=0x1c
[ 2.231811] xhci_hcd 0000:03:00.0: hcc params 0x0200f180 hci version 0x96 quirks 0x0000000000080010
[ 2.235245] r8169 0000:04:00.0: enabling device (0000 -> 0003)
[ 2.265051] xhci_hcd 0000:03:00.0: xHCI Host Controller
[ 2.265076] xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 9
[ 2.265092] xhci_hcd 0000:03:00.0: Host supports USB 3.0 SuperSpeed
[ 2.266951] usb usb8: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 2.266958] usb usb8: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.266961] usb usb8: Product: xHCI Host Controller
[ 2.266963] usb usb8: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 xhci-hcd
[ 2.266965] usb usb8: SerialNumber: 0000:03:00.0
[ 2.267014] input: HD-Audio Generic Rear Mic as /devices/pci0000:00/0000:00:14.2/sound/card1/input3
[ 2.267354] input: HD-Audio Generic Front Mic as /devices/pci0000:00/0000:00:14.2/sound/card1/input4
[ 2.267750] input: HD-Audio Generic Line as /devices/pci0000:00/0000:00:14.2/sound/card1/input5
[ 2.268089] input: HD-Audio Generic Line Out Front as /devices/pci0000:00/0000:00:14.2/sound/card1/input6
[ 2.268437] input: HD-Audio Generic Line Out Surround as /devices/pci0000:00/0000:00:14.2/sound/card1/input7
[ 2.268783] input: HD-Audio Generic Line Out CLFE as /devices/pci0000:00/0000:00:14.2/sound/card1/input8
[ 2.269132] input: HD-Audio Generic Line Out Side as /devices/pci0000:00/0000:00:14.2/sound/card1/input9
[ 2.269714] hub 8-0:1.0: USB hub found
[ 2.269815] hub 8-0:1.0: 2 ports detected
[ 2.269956] input: HD-Audio Generic Front Headphone as /devices/pci0000:00/0000:00:14.2/sound/card1/input10
[ 2.272826] usb usb9: We don't know the algorithms for LPM for this host, disabling LPM.
[ 2.273006] usb usb9: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 6.03
[ 2.273010] usb usb9: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.273012] usb usb9: Product: xHCI Host Controller
[ 2.273014] usb usb9: Manufacturer: Linux 6.3.0-rc6-00311-gde8224969f66 xhci-hcd
[ 2.273016] usb usb9: SerialNumber: 0000:03:00.0
[ 2.275521] hub 9-0:1.0: USB hub found
[ 2.275649] hub 9-0:1.0: 2 ports detected
[ 2.295318] r8169 0000:04:00.0 eth0: RTL8168f/8111f, 08:60:6e:74:7a:51, XID 480, IRQ 32
[ 2.295327] r8169 0000:04:00.0 eth0: jumbo features [frames: 9194 bytes, tx checksumming: ko]
[ 2.311845] r8169 0000:04:00.0 enp4s0: renamed from eth0
[ 2.421403] r8169 0000:04:00.0: Direct firmware load for rtl_nic/rtl8168f-1.fw failed with error -2
[ 2.421415] r8169 0000:04:00.0: Unable to load firmware rtl_nic/rtl8168f-1.fw (-2)
[ 2.421903] RTL8211E Gigabit Ethernet r8169-0-400:00: attached PHY driver (mii_bus:phy_addr=r8169-0-400:00, irq=MAC)
[ 2.465163] usb 5-1: new low-speed USB device number 2 using ohci-pci
[ 2.472735] [drm] radeon kernel modesetting enabled.
[ 2.474200] [drm] initializing kernel modesetting (ARUBA 0x1002:0x9996 0x1002:0x9996 0x00).
[ 2.474271] ATOM BIOS: 113
[ 2.474372] radeon 0000:00:01.0: VRAM: 512M 0x0000000000000000 - 0x000000001FFFFFFF (512M used)
[ 2.474376] radeon 0000:00:01.0: GTT: 1024M 0x0000000020000000 - 0x000000005FFFFFFF
[ 2.474384] [drm] Detected VRAM RAM=512M, BAR=256M
[ 2.474385] [drm] RAM width 64bits DDR
[ 2.474589] [drm] radeon: 512M of VRAM memory ready
[ 2.474595] [drm] radeon: 1024M of GTT memory ready.
[ 2.474642] [drm] Loading ARUBA Microcode
[ 2.481608] [drm] Internal thermal controller without fan control
[ 2.481974] [drm] radeon: dpm initialized
[ 2.486664] [drm] Found VCE firmware/feedback version 50.0.1 / 17!
[ 2.486711] [drm] GART: num cpu pages 262144, num gpu pages 262144
[ 2.492402] r8169 0000:04:00.0 enp4s0: Link is Down
[ 2.526396] [drm] GART: Restore entries: num cpu pages 262144, num gpu pages 262144
[ 2.529843] [drm] GART: Done restoring entries
[ 2.529847] [drm] PCIE GART of 1024M enabled (table at 0x00000000001D6000).
[ 2.530087] radeon 0000:00:01.0: WB enabled
[ 2.530090] radeon 0000:00:01.0: fence driver on ring 0 use gpu addr 0x0000000020000c00
[ 2.530468] radeon 0000:00:01.0: fence driver on ring 5 use gpu addr 0x0000000000075a18
[ 2.550526] radeon 0000:00:01.0: fence driver on ring 6 use gpu addr 0x0000000020000c18
[ 2.550531] radeon 0000:00:01.0: fence driver on ring 7 use gpu addr 0x0000000020000c1c
[ 2.550533] radeon 0000:00:01.0: fence driver on ring 1 use gpu addr 0x0000000020000c04
[ 2.550535] radeon 0000:00:01.0: fence driver on ring 2 use gpu addr 0x0000000020000c08
[ 2.550536] radeon 0000:00:01.0: fence driver on ring 3 use gpu addr 0x0000000020000c0c
[ 2.550538] radeon 0000:00:01.0: fence driver on ring 4 use gpu addr 0x0000000020000c10
[ 2.550809] radeon 0000:00:01.0: radeon: MSI limited to 32-bit
[ 2.551003] radeon 0000:00:01.0: radeon: using MSI.
[ 2.551071] [drm] radeon: irq initialized.
[ 2.569579] [drm] ring test on 0 succeeded in 3 usecs
[ 2.569589] [drm] ring test on 3 succeeded in 4 usecs
[ 2.569596] [drm] ring test on 4 succeeded in 4 usecs
[ 2.583595] [drm] ring test on 5 succeeded in 2 usecs
[ 2.585593] [drm] UVD initialized successfully.
[ 2.670213] usb 5-1: New USB device found, idVendor=413c, idProduct=2106, bcdDevice= 1.01
[ 2.670218] usb 5-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 2.670220] usb 5-1: Product: Dell QuietKey Keyboard
[ 2.670222] usb 5-1: Manufacturer: DELL
[ 2.678027] input: DELL Dell QuietKey Keyboard as /devices/pci0000:00/0000:00:12.0/usb5/5-1/5-1:1.0/0003:413C:2106.0001/input/input11
[ 2.694994] [drm] ring test on 6 succeeded in 18 usecs
[ 2.695006] [drm] ring test on 7 succeeded in 3 usecs
[ 2.695007] [drm] VCE initialized successfully.
[ 2.695174] snd_hda_intel 0000:00:01.1: bound 0000:00:01.0 (ops radeon_audio_component_bind_ops [radeon])
[ 2.695346] [drm] ib test on ring 0 succeeded in 0 usecs
[ 2.695400] [drm] ib test on ring 3 succeeded in 0 usecs
[ 2.695449] [drm] ib test on ring 4 succeeded in 0 usecs
[ 2.713280] [drm] ib test on ring 5 succeeded
[ 2.729270] [drm] ib test on ring 6 succeeded in 1 usecs
[ 2.737964] hid-generic 0003:413C:2106.0001: input,hidraw0: USB HID v1.10 Keyboard [DELL Dell QuietKey Keyboard] on usb-0000:00:12.0-1/input0
[ 2.745270] [drm] ib test on ring 7 succeeded in 1 usecs
[ 2.748783] [drm] Radeon Display Connectors
[ 2.748789] [drm] Connector 0:
[ 2.748790] [drm] DP-1
[ 2.748791] [drm] HPD1
[ 2.748792] [drm] DDC: 0x6530 0x6530 0x6534 0x6534 0x6538 0x6538 0x653c 0x653c
[ 2.748795] [drm] Encoders:
[ 2.748796] [drm] DFP1: INTERNAL_UNIPHY2
[ 2.748797] [drm] Connector 1:
[ 2.748798] [drm] VGA-1
[ 2.748799] [drm] HPD2
[ 2.748800] [drm] DDC: 0x6540 0x6540 0x6544 0x6544 0x6548 0x6548 0x654c 0x654c
[ 2.748802] [drm] Encoders:
[ 2.748803] [drm] CRT1: INTERNAL_UNIPHY2
[ 2.748804] [drm] CRT1: NUTMEG
[ 2.748805] [drm] Connector 2:
[ 2.748806] [drm] HDMI-A-1
[ 2.748807] [drm] HPD3
[ 2.748808] [drm] DDC: 0x6550 0x6550 0x6554 0x6554 0x6558 0x6558 0x655c 0x655c
[ 2.748811] [drm] Encoders:
[ 2.748811] [drm] DFP2: INTERNAL_UNIPHY
[ 3.019999] [drm] fb mappable at 0xE03E9000
[ 3.020006] [drm] vram apper at 0xE0000000
[ 3.020009] [drm] size 5242880
[ 3.020011] [drm] fb depth is 24
[ 3.020012] [drm] pitch is 5120
[ 3.020508] fbcon: radeondrmfb (fb0) is primary device
[ 3.201239] usb 5-2: new low-speed USB device number 3 using ohci-pci
[ 3.215988] Console: switching to colour frame buffer device 160x64
[ 3.217767] radeon 0000:00:01.0: [drm] fb0: radeondrmfb frame buffer device
[ 3.237462] [drm] Initialized radeon 2.50.0 20080528 for 0000:00:01.0 on minor 0
[ 3.397485] usb 5-2: New USB device found, idVendor=046d, idProduct=c016, bcdDevice= 3.40
[ 3.397493] usb 5-2: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 3.397495] usb 5-2: Product: Optical USB Mouse
[ 3.397497] usb 5-2: Manufacturer: Logitech
[ 3.406006] input: Logitech Optical USB Mouse as /devices/pci0000:00/0000:00:12.0/usb5/5-2/5-2:1.0/0003:046D:C016.0002/input/input12
[ 3.406790] hid-generic 0003:046D:C016.0002: input,hidraw1: USB HID v1.10 Mouse [Logitech Optical USB Mouse] on usb-0000:00:12.0-2/input0
[ 5.100538] r8169 0000:04:00.0 enp4s0: Link is Up - 1Gbps/Full - flow control rx/tx
[ 5.100555] IPv6: ADDRCONF(NETDEV_CHANGE): enp4s0: link becomes ready
[ 5.945432] [drm] amdgpu kernel modesetting enabled.
[ 10.066361] memfd_create() without MFD_EXEC nor MFD_NOEXEC_SEAL, pid=258 'systemd'
[-- Attachment #3: Type: text/plain, Size: 176 bytes --]
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 16/37] x86/xen/smp_pv: Remove wait for CPU online
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-17 20:46 ` Boris Ostrovsky
-1 siblings, 0 replies; 236+ messages in thread
From: Boris Ostrovsky @ 2023-04-17 20:46 UTC (permalink / raw)
To: Thomas Gleixner, LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Juergen Gross, xen-devel,
David Woodhouse, Usama Arif, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On 4/14/23 7:44 PM, Thomas Gleixner wrote:
> Now that the core code drops sparse_irq_lock after the idle thread
> synchronized, it's pointless to wait for the AP to mark itself online.
>
> Whether the control CPU runs in a wait loop or sleeps in the core code
> waiting for the online operation to complete makes no difference.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Juergen Gross <jgross@suse.com>
> Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
> Cc: xen-devel@lists.xenproject.org
> ---
> arch/x86/xen/smp_pv.c | 10 +++++-----
> 1 file changed, 5 insertions(+), 5 deletions(-)
>
> --- a/arch/x86/xen/smp_pv.c
> +++ b/arch/x86/xen/smp_pv.c
> @@ -340,11 +340,11 @@ static int xen_pv_cpu_up(unsigned int cp
>
> xen_pmu_init(cpu);
>
> - rc = HYPERVISOR_vcpu_op(VCPUOP_up, xen_vcpu_nr(cpu), NULL);
> - BUG_ON(rc);
> -
> - while (cpu_report_state(cpu) != CPU_ONLINE)
> - HYPERVISOR_sched_op(SCHEDOP_yield, NULL);
> + /*
> + * Why is this a BUG? If the hypercall fails then everything can be
> + * rolled back, no?
> + */
In many cases this indicates either some sort of hypervisor internal error or broken logic in the guest, so it is, well, a bug. But I suppose it may also be some transient condition in the hypervisor (I don't see it now but it can happen in the future) so perhaps we should indeed try not to die on the spot.
-boris
> + BUG_ON(HYPERVISOR_vcpu_op(VCPUOP_up, xen_vcpu_nr(cpu), NULL));
>
> return 0;
> }
>
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 16/37] x86/xen/smp_pv: Remove wait for CPU online
@ 2023-04-17 20:46 ` Boris Ostrovsky
0 siblings, 0 replies; 236+ messages in thread
From: Boris Ostrovsky @ 2023-04-17 20:46 UTC (permalink / raw)
To: Thomas Gleixner, LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Juergen Gross, xen-devel,
David Woodhouse, Usama Arif, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On 4/14/23 7:44 PM, Thomas Gleixner wrote:
> Now that the core code drops sparse_irq_lock after the idle thread
> synchronized, it's pointless to wait for the AP to mark itself online.
>
> Whether the control CPU runs in a wait loop or sleeps in the core code
> waiting for the online operation to complete makes no difference.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Juergen Gross <jgross@suse.com>
> Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
> Cc: xen-devel@lists.xenproject.org
> ---
> arch/x86/xen/smp_pv.c | 10 +++++-----
> 1 file changed, 5 insertions(+), 5 deletions(-)
>
> --- a/arch/x86/xen/smp_pv.c
> +++ b/arch/x86/xen/smp_pv.c
> @@ -340,11 +340,11 @@ static int xen_pv_cpu_up(unsigned int cp
>
> xen_pmu_init(cpu);
>
> - rc = HYPERVISOR_vcpu_op(VCPUOP_up, xen_vcpu_nr(cpu), NULL);
> - BUG_ON(rc);
> -
> - while (cpu_report_state(cpu) != CPU_ONLINE)
> - HYPERVISOR_sched_op(SCHEDOP_yield, NULL);
> + /*
> + * Why is this a BUG? If the hypercall fails then everything can be
> + * rolled back, no?
> + */
In many cases this indicates either some sort of hypervisor internal error or broken logic in the guest, so it is, well, a bug. But I suppose it may also be some transient condition in the hypervisor (I don't see it now but it can happen in the future) so perhaps we should indeed try not to die on the spot.
-boris
> + BUG_ON(HYPERVISOR_vcpu_op(VCPUOP_up, xen_vcpu_nr(cpu), NULL));
>
> return 0;
> }
>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 16/37] x86/xen/smp_pv: Remove wait for CPU online
@ 2023-04-17 20:46 ` Boris Ostrovsky
0 siblings, 0 replies; 236+ messages in thread
From: Boris Ostrovsky @ 2023-04-17 20:46 UTC (permalink / raw)
To: Thomas Gleixner, LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Juergen Gross, xen-devel,
David Woodhouse, Usama Arif, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On 4/14/23 7:44 PM, Thomas Gleixner wrote:
> Now that the core code drops sparse_irq_lock after the idle thread
> synchronized, it's pointless to wait for the AP to mark itself online.
>
> Whether the control CPU runs in a wait loop or sleeps in the core code
> waiting for the online operation to complete makes no difference.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Juergen Gross <jgross@suse.com>
> Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
> Cc: xen-devel@lists.xenproject.org
> ---
> arch/x86/xen/smp_pv.c | 10 +++++-----
> 1 file changed, 5 insertions(+), 5 deletions(-)
>
> --- a/arch/x86/xen/smp_pv.c
> +++ b/arch/x86/xen/smp_pv.c
> @@ -340,11 +340,11 @@ static int xen_pv_cpu_up(unsigned int cp
>
> xen_pmu_init(cpu);
>
> - rc = HYPERVISOR_vcpu_op(VCPUOP_up, xen_vcpu_nr(cpu), NULL);
> - BUG_ON(rc);
> -
> - while (cpu_report_state(cpu) != CPU_ONLINE)
> - HYPERVISOR_sched_op(SCHEDOP_yield, NULL);
> + /*
> + * Why is this a BUG? If the hypercall fails then everything can be
> + * rolled back, no?
> + */
In many cases this indicates either some sort of hypervisor internal error or broken logic in the guest, so it is, well, a bug. But I suppose it may also be some transient condition in the hypervisor (I don't see it now but it can happen in the future) so perhaps we should indeed try not to die on the spot.
-boris
> + BUG_ON(HYPERVISOR_vcpu_op(VCPUOP_up, xen_vcpu_nr(cpu), NULL));
>
> return 0;
> }
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
2023-04-17 17:40 ` Paul Menzel
(?)
@ 2023-04-18 6:58 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-18 6:58 UTC (permalink / raw)
To: Paul Menzel
Cc: linux-kernel, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Paul!
On Mon, Apr 17 2023 at 19:40, Paul Menzel wrote:
> Am 17.04.23 um 16:48 schrieb Thomas Gleixner:
>
>> On Mon, Apr 17 2023 at 13:19, Paul Menzel wrote:
>>> Am 15.04.23 um 01:44 schrieb Thomas Gleixner:
>>> [ 0.258193] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD
>>> Graphics (family: 0x15, model: 0x13, stepping: 0x1)
>>> […]
>>> [ 0.259329] smp: Bringing up secondary CPUs ...
>>> [ 0.259527] x86: Booting SMP configuration:
>>> [ 0.259528] .... node #0, CPUs: #1
>>> [ 0.261007] After schedule_preempt_disabled
>>> [ 10.260990] CPU1 failed to report alive state
>>
>> Weird. CPU1 fails to come up and report that it has reached the
>> synchronization point.
>>
>> Does it work when you add cpuhp.parallel=off on the kernel command line?
>
> Yes, the ten seconds delay is gone with `cpuhp.parallel=off`.
>
> There was a patch set in the past, that worked on that device. I think
> up to v4 it did *not* work at all and hung [1]. I need some days to
> collect the results again.
Can you please apply the patch below on top of the pile remove the
command line option again?
Thanks,
tglx
---
kernel/cpu.c | 1 +
1 file changed, 1 insertion(+)
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -1777,6 +1777,7 @@ static void __init cpuhp_bringup_mask(co
*/
WARN_ON(cpuhp_invoke_callback_range(false, cpu, st, CPUHP_OFFLINE));
}
+ msleep(20);
}
}
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-18 6:58 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-18 6:58 UTC (permalink / raw)
To: Paul Menzel
Cc: linux-kernel, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Paul!
On Mon, Apr 17 2023 at 19:40, Paul Menzel wrote:
> Am 17.04.23 um 16:48 schrieb Thomas Gleixner:
>
>> On Mon, Apr 17 2023 at 13:19, Paul Menzel wrote:
>>> Am 15.04.23 um 01:44 schrieb Thomas Gleixner:
>>> [ 0.258193] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD
>>> Graphics (family: 0x15, model: 0x13, stepping: 0x1)
>>> […]
>>> [ 0.259329] smp: Bringing up secondary CPUs ...
>>> [ 0.259527] x86: Booting SMP configuration:
>>> [ 0.259528] .... node #0, CPUs: #1
>>> [ 0.261007] After schedule_preempt_disabled
>>> [ 10.260990] CPU1 failed to report alive state
>>
>> Weird. CPU1 fails to come up and report that it has reached the
>> synchronization point.
>>
>> Does it work when you add cpuhp.parallel=off on the kernel command line?
>
> Yes, the ten seconds delay is gone with `cpuhp.parallel=off`.
>
> There was a patch set in the past, that worked on that device. I think
> up to v4 it did *not* work at all and hung [1]. I need some days to
> collect the results again.
Can you please apply the patch below on top of the pile remove the
command line option again?
Thanks,
tglx
---
kernel/cpu.c | 1 +
1 file changed, 1 insertion(+)
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -1777,6 +1777,7 @@ static void __init cpuhp_bringup_mask(co
*/
WARN_ON(cpuhp_invoke_callback_range(false, cpu, st, CPUHP_OFFLINE));
}
+ msleep(20);
}
}
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-18 6:58 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-18 6:58 UTC (permalink / raw)
To: Paul Menzel
Cc: linux-kernel, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Paul!
On Mon, Apr 17 2023 at 19:40, Paul Menzel wrote:
> Am 17.04.23 um 16:48 schrieb Thomas Gleixner:
>
>> On Mon, Apr 17 2023 at 13:19, Paul Menzel wrote:
>>> Am 15.04.23 um 01:44 schrieb Thomas Gleixner:
>>> [ 0.258193] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD
>>> Graphics (family: 0x15, model: 0x13, stepping: 0x1)
>>> […]
>>> [ 0.259329] smp: Bringing up secondary CPUs ...
>>> [ 0.259527] x86: Booting SMP configuration:
>>> [ 0.259528] .... node #0, CPUs: #1
>>> [ 0.261007] After schedule_preempt_disabled
>>> [ 10.260990] CPU1 failed to report alive state
>>
>> Weird. CPU1 fails to come up and report that it has reached the
>> synchronization point.
>>
>> Does it work when you add cpuhp.parallel=off on the kernel command line?
>
> Yes, the ten seconds delay is gone with `cpuhp.parallel=off`.
>
> There was a patch set in the past, that worked on that device. I think
> up to v4 it did *not* work at all and hung [1]. I need some days to
> collect the results again.
Can you please apply the patch below on top of the pile remove the
command line option again?
Thanks,
tglx
---
kernel/cpu.c | 1 +
1 file changed, 1 insertion(+)
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -1777,6 +1777,7 @@ static void __init cpuhp_bringup_mask(co
*/
WARN_ON(cpuhp_invoke_callback_range(false, cpu, st, CPUHP_OFFLINE));
}
+ msleep(20);
}
}
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
2023-04-18 6:58 ` Thomas Gleixner
(?)
@ 2023-04-18 8:40 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-18 8:40 UTC (permalink / raw)
To: Paul Menzel
Cc: linux-kernel, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Tue, Apr 18 2023 at 08:58, Thomas Gleixner wrote:
> On Mon, Apr 17 2023 at 19:40, Paul Menzel wrote:
>> Am 17.04.23 um 16:48 schrieb Thomas Gleixner:
>>
>>> On Mon, Apr 17 2023 at 13:19, Paul Menzel wrote:
>>>> Am 15.04.23 um 01:44 schrieb Thomas Gleixner:
>>>> [ 0.258193] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD
>>>> Graphics (family: 0x15, model: 0x13, stepping: 0x1)
>>>> […]
>>>> [ 0.259329] smp: Bringing up secondary CPUs ...
>>>> [ 0.259527] x86: Booting SMP configuration:
>>>> [ 0.259528] .... node #0, CPUs: #1
>>>> [ 0.261007] After schedule_preempt_disabled
>>>> [ 10.260990] CPU1 failed to report alive state
>>>
>>> Weird. CPU1 fails to come up and report that it has reached the
>>> synchronization point.
>>>
>>> Does it work when you add cpuhp.parallel=off on the kernel command line?
>>
>> Yes, the ten seconds delay is gone with `cpuhp.parallel=off`.
>>
>> There was a patch set in the past, that worked on that device. I think
>> up to v4 it did *not* work at all and hung [1]. I need some days to
>> collect the results again.
>
> Can you please apply the patch below on top of the pile remove the
> command line option again?
Bah. That patch does not make any sense at all. Not enough coffee.
Can you please provide the output of cpuid?
Thanks,
tglx
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-18 8:40 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-18 8:40 UTC (permalink / raw)
To: Paul Menzel
Cc: linux-kernel, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Tue, Apr 18 2023 at 08:58, Thomas Gleixner wrote:
> On Mon, Apr 17 2023 at 19:40, Paul Menzel wrote:
>> Am 17.04.23 um 16:48 schrieb Thomas Gleixner:
>>
>>> On Mon, Apr 17 2023 at 13:19, Paul Menzel wrote:
>>>> Am 15.04.23 um 01:44 schrieb Thomas Gleixner:
>>>> [ 0.258193] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD
>>>> Graphics (family: 0x15, model: 0x13, stepping: 0x1)
>>>> […]
>>>> [ 0.259329] smp: Bringing up secondary CPUs ...
>>>> [ 0.259527] x86: Booting SMP configuration:
>>>> [ 0.259528] .... node #0, CPUs: #1
>>>> [ 0.261007] After schedule_preempt_disabled
>>>> [ 10.260990] CPU1 failed to report alive state
>>>
>>> Weird. CPU1 fails to come up and report that it has reached the
>>> synchronization point.
>>>
>>> Does it work when you add cpuhp.parallel=off on the kernel command line?
>>
>> Yes, the ten seconds delay is gone with `cpuhp.parallel=off`.
>>
>> There was a patch set in the past, that worked on that device. I think
>> up to v4 it did *not* work at all and hung [1]. I need some days to
>> collect the results again.
>
> Can you please apply the patch below on top of the pile remove the
> command line option again?
Bah. That patch does not make any sense at all. Not enough coffee.
Can you please provide the output of cpuid?
Thanks,
tglx
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-18 8:40 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-18 8:40 UTC (permalink / raw)
To: Paul Menzel
Cc: linux-kernel, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Tue, Apr 18 2023 at 08:58, Thomas Gleixner wrote:
> On Mon, Apr 17 2023 at 19:40, Paul Menzel wrote:
>> Am 17.04.23 um 16:48 schrieb Thomas Gleixner:
>>
>>> On Mon, Apr 17 2023 at 13:19, Paul Menzel wrote:
>>>> Am 15.04.23 um 01:44 schrieb Thomas Gleixner:
>>>> [ 0.258193] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD
>>>> Graphics (family: 0x15, model: 0x13, stepping: 0x1)
>>>> […]
>>>> [ 0.259329] smp: Bringing up secondary CPUs ...
>>>> [ 0.259527] x86: Booting SMP configuration:
>>>> [ 0.259528] .... node #0, CPUs: #1
>>>> [ 0.261007] After schedule_preempt_disabled
>>>> [ 10.260990] CPU1 failed to report alive state
>>>
>>> Weird. CPU1 fails to come up and report that it has reached the
>>> synchronization point.
>>>
>>> Does it work when you add cpuhp.parallel=off on the kernel command line?
>>
>> Yes, the ten seconds delay is gone with `cpuhp.parallel=off`.
>>
>> There was a patch set in the past, that worked on that device. I think
>> up to v4 it did *not* work at all and hung [1]. I need some days to
>> collect the results again.
>
> Can you please apply the patch below on top of the pile remove the
> command line option again?
Bah. That patch does not make any sense at all. Not enough coffee.
Can you please provide the output of cpuid?
Thanks,
tglx
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
2023-04-18 8:40 ` Thomas Gleixner
(?)
@ 2023-04-18 20:10 ` Paul Menzel
-1 siblings, 0 replies; 236+ messages in thread
From: Paul Menzel @ 2023-04-18 20:10 UTC (permalink / raw)
To: Thomas Gleixner
Cc: linux-kernel, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
[-- Attachment #1: Type: text/plain, Size: 1924 bytes --]
Dear Thomas,
Am 18.04.23 um 10:40 schrieb Thomas Gleixner:
> On Tue, Apr 18 2023 at 08:58, Thomas Gleixner wrote:
>> On Mon, Apr 17 2023 at 19:40, Paul Menzel wrote:
>>> Am 17.04.23 um 16:48 schrieb Thomas Gleixner:
>>>
>>>> On Mon, Apr 17 2023 at 13:19, Paul Menzel wrote:
>>>>> Am 15.04.23 um 01:44 schrieb Thomas Gleixner:
>>>>> [ 0.258193] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD Graphics (family: 0x15, model: 0x13, stepping: 0x1)
>>>>> […]
>>>>> [ 0.259329] smp: Bringing up secondary CPUs ...
>>>>> [ 0.259527] x86: Booting SMP configuration:
>>>>> [ 0.259528] .... node #0, CPUs: #1
>>>>> [ 0.261007] After schedule_preempt_disabled
>>>>> [ 10.260990] CPU1 failed to report alive state
>>>>
>>>> Weird. CPU1 fails to come up and report that it has reached the
>>>> synchronization point.
>>>>
>>>> Does it work when you add cpuhp.parallel=off on the kernel command line?
>>>
>>> Yes, the ten seconds delay is gone with `cpuhp.parallel=off`.
>>>
>>> There was a patch set in the past, that worked on that device. I think
>>> up to v4 it did *not* work at all and hung [1]. I need some days to
>>> collect the results again.
>>
>> Can you please apply the patch below on top of the pile remove the
>> command line option again?
>
> Bah. That patch does not make any sense at all. Not enough coffee.
>
> Can you please provide the output of cpuid?
Of course. Here the top, and the whole output is attached.
```
CPU 0:
vendor_id = "AuthenticAMD"
version information (1/eax):
processor type = primary processor (0)
family = 0xf (15)
model = 0x3 (3)
stepping id = 0x1 (1)
extended family = 0x6 (6)
extended model = 0x1 (1)
(family synth) = 0x15 (21)
(model synth) = 0x13 (19)
(simple synth) = AMD (unknown type) (Richland RL-A1)
[Piledriver], 32nm
[…]
```
Kind regards,
Paul
[-- Attachment #2: cpuid.txt --]
[-- Type: text/plain, Size: 65008 bytes --]
CPU 0:
vendor_id = "AuthenticAMD"
version information (1/eax):
processor type = primary processor (0)
family = 0xf (15)
model = 0x3 (3)
stepping id = 0x1 (1)
extended family = 0x6 (6)
extended model = 0x1 (1)
(family synth) = 0x15 (21)
(model synth) = 0x13 (19)
(simple synth) = AMD (unknown type) (Richland RL-A1) [Piledriver], 32nm
miscellaneous (1/ebx):
process local APIC physical ID = 0x0 (0)
maximum IDs for CPUs in pkg = 0x2 (2)
CLFLUSH line size = 0x8 (8)
brand index = 0x0 (0)
brand id = 0x00 (0): unknown
feature information (1/edx):
x87 FPU on chip = true
VME: virtual-8086 mode enhancement = true
DE: debugging extensions = true
PSE: page size extensions = true
TSC: time stamp counter = true
RDMSR and WRMSR support = true
PAE: physical address extensions = true
MCE: machine check exception = true
CMPXCHG8B inst. = true
APIC on chip = true
SYSENTER and SYSEXIT = true
MTRR: memory type range registers = true
PTE global bit = true
MCA: machine check architecture = true
CMOV: conditional move/compare instr = true
PAT: page attribute table = true
PSE-36: page size extension = true
PSN: processor serial number = false
CLFLUSH instruction = true
DS: debug store = false
ACPI: thermal monitor and clock ctrl = false
MMX Technology = true
FXSAVE/FXRSTOR = true
SSE extensions = true
SSE2 extensions = true
SS: self snoop = false
hyper-threading / multi-core supported = true
TM: therm. monitor = false
IA64 = false
PBE: pending break event = false
feature information (1/ecx):
PNI/SSE3: Prescott New Instructions = true
PCLMULDQ instruction = true
DTES64: 64-bit debug store = false
MONITOR/MWAIT = true
CPL-qualified debug store = false
VMX: virtual machine extensions = false
SMX: safer mode extensions = false
Enhanced Intel SpeedStep Technology = false
TM2: thermal monitor 2 = false
SSSE3 extensions = true
context ID: adaptive or shared L1 data = false
SDBG: IA32_DEBUG_INTERFACE = false
FMA instruction = true
CMPXCHG16B instruction = true
xTPR disable = false
PDCM: perfmon and debug = false
PCID: process context identifiers = false
DCA: direct cache access = false
SSE4.1 extensions = true
SSE4.2 extensions = true
x2APIC: extended xAPIC support = false
MOVBE instruction = false
POPCNT instruction = true
time stamp counter deadline = false
AES instruction = true
XSAVE/XSTOR states = true
OS-enabled XSAVE/XSTOR = true
AVX: advanced vector extensions = true
F16C half-precision convert instruction = true
RDRAND instruction = false
hypervisor guest status = false
cache and TLB information (2):
processor serial number = 0061-0F31-0000-0000-0000-0000
deterministic cache parameters (4):
--- cache 0 ---
cache type = no more caches (0)
MONITOR/MWAIT (5):
smallest monitor-line size (bytes) = 0x40 (64)
largest monitor-line size (bytes) = 0x40 (64)
enum of Monitor-MWAIT exts supported = true
supports intrs as break-event for MWAIT = true
number of C0 sub C-states using MWAIT = 0x0 (0)
number of C1 sub C-states using MWAIT = 0x0 (0)
number of C2 sub C-states using MWAIT = 0x0 (0)
number of C3 sub C-states using MWAIT = 0x0 (0)
number of C4 sub C-states using MWAIT = 0x0 (0)
number of C5 sub C-states using MWAIT = 0x0 (0)
number of C6 sub C-states using MWAIT = 0x0 (0)
number of C7 sub C-states using MWAIT = 0x0 (0)
Thermal and Power Management Features (6):
digital thermometer = false
Intel Turbo Boost Technology = false
ARAT always running APIC timer = false
PLN power limit notification = false
ECMD extended clock modulation duty = false
PTM package thermal management = false
HWP base registers = false
HWP notification = false
HWP activity window = false
HWP energy performance preference = false
HWP package level request = false
HDC base registers = false
Intel Turbo Boost Max Technology 3.0 = false
HWP capabilities = false
HWP PECI override = false
flexible HWP = false
IA32_HWP_REQUEST MSR fast access mode = false
HW_FEEDBACK MSRs supported = false
ignoring idle logical processor HWP req = false
Thread Director = false
IA32_HW_FEEDBACK_THREAD_CONFIG bit 25 = false
digital thermometer thresholds = 0x0 (0)
hardware coordination feedback = true
ACNT2 available = false
performance-energy bias capability = false
number of enh hardware feedback classes = 0x0 (0)
performance capability reporting = false
energy efficiency capability reporting = false
size of feedback struct (4KB pages) = 0x1 (1)
index of CPU's row in feedback struct = 0x0 (0)
extended feature flags (7):
FSGSBASE instructions = false
IA32_TSC_ADJUST MSR supported = false
SGX: Software Guard Extensions supported = false
BMI1 instructions = true
HLE hardware lock elision = false
AVX2: advanced vector extensions 2 = false
FDP_EXCPTN_ONLY = false
SMEP supervisor mode exec protection = false
BMI2 instructions = false
enhanced REP MOVSB/STOSB = false
INVPCID instruction = false
RTM: restricted transactional memory = false
RDT-CMT/PQoS cache monitoring = false
deprecated FPU CS/DS = false
MPX: intel memory protection extensions = false
RDT-CAT/PQE cache allocation = false
AVX512F: AVX-512 foundation instructions = false
AVX512DQ: double & quadword instructions = false
RDSEED instruction = false
ADX instructions = false
SMAP: supervisor mode access prevention = false
AVX512IFMA: integer fused multiply add = false
PCOMMIT instruction = false
CLFLUSHOPT instruction = false
CLWB instruction = false
Intel processor trace = false
AVX512PF: prefetch instructions = false
AVX512ER: exponent & reciprocal instrs = false
AVX512CD: conflict detection instrs = false
SHA instructions = false
AVX512BW: byte & word instructions = false
AVX512VL: vector length = false
PREFETCHWT1 = false
AVX512VBMI: vector byte manipulation = false
UMIP: user-mode instruction prevention = false
PKU protection keys for user-mode = false
OSPKE CR4.PKE and RDPKRU/WRPKRU = false
WAITPKG instructions = false
AVX512_VBMI2: byte VPCOMPRESS, VPEXPAND = false
CET_SS: CET shadow stack = false
GFNI: Galois Field New Instructions = false
VAES instructions = false
VPCLMULQDQ instruction = false
AVX512_VNNI: neural network instructions = false
AVX512_BITALG: bit count/shiffle = false
TME: Total Memory Encryption = false
AVX512: VPOPCNTDQ instruction = false
LA57: 57-bit addrs & 5-level paging = false
BNDLDX/BNDSTX MAWAU value in 64-bit mode = 0x0 (0)
RDPID: read processor ID supported = false
KL: key locker = false
bus lock detection = false
CLDEMOTE supports cache line demote = false
MOVDIRI instruction = false
MOVDIR64B instruction = false
ENQCMD instruction = false
SGX_LC: SGX launch config supported = false
PKS: supervisor protection keys = false
SGX-KEYS: SGX attestation services = false
AVX512_4VNNIW: neural network instrs = false
AVX512_4FMAPS: multiply acc single prec = false
fast short REP MOV = false
UINTR: user interrupts = false
AVX512_VP2INTERSECT: intersect mask regs = false
IA32_MCU_OPT_CTRL SRBDS mitigation MSR = false
VERW MD_CLEAR microcode support = false
RTM transaction always aborts = false
IA32_TSX_FORCE_ABORT MSR = false
SERIALIZE instruction = false
hybrid part = false
TSXLDTRK: TSX suspend load addr tracking = false
PCONFIG instruction = false
LBR: architectural last branch records = false
CET_IBT: CET indirect branch tracking = false
AMX-BF16: tile bfloat16 support = false
AVX512_FP16: fp16 support = false
AMX-TILE: tile architecture support = false
AMX-INT8: tile 8-bit integer support = false
IBRS/IBPB: indirect branch restrictions = false
STIBP: 1 thr indirect branch predictor = false
L1D_FLUSH: IA32_FLUSH_CMD MSR = false
IA32_ARCH_CAPABILITIES MSR = false
IA32_CORE_CAPABILITIES MSR = false
SSBD: speculative store bypass disable = false
Direct Cache Access Parameters (9):
PLATFORM_DCA_CAP MSR bits = 0
Architecture Performance Monitoring Features (0xa):
version ID = 0x0 (0)
number of counters per logical processor = 0x0 (0)
bit width of counter = 0x0 (0)
length of EBX bit vector = 0x0 (0)
core cycle event = not available
instruction retired event = not available
reference cycles event = not available
last-level cache ref event = not available
last-level cache miss event = not available
branch inst retired event = not available
branch mispred retired event = not available
top-down slots event = not available
fixed counter 0 supported = false
fixed counter 1 supported = false
fixed counter 2 supported = false
fixed counter 3 supported = false
fixed counter 4 supported = false
fixed counter 5 supported = false
fixed counter 6 supported = false
fixed counter 7 supported = false
fixed counter 8 supported = false
fixed counter 9 supported = false
fixed counter 10 supported = false
fixed counter 11 supported = false
fixed counter 12 supported = false
fixed counter 13 supported = false
fixed counter 14 supported = false
fixed counter 15 supported = false
fixed counter 16 supported = false
fixed counter 17 supported = false
fixed counter 18 supported = false
fixed counter 19 supported = false
fixed counter 20 supported = false
fixed counter 21 supported = false
fixed counter 22 supported = false
fixed counter 23 supported = false
fixed counter 24 supported = false
fixed counter 25 supported = false
fixed counter 26 supported = false
fixed counter 27 supported = false
fixed counter 28 supported = false
fixed counter 29 supported = false
fixed counter 30 supported = false
fixed counter 31 supported = false
number of contiguous fixed counters = 0x0 (0)
bit width of fixed counters = 0x0 (0)
anythread deprecation = false
x2APIC features / processor topology (0xb):
extended APIC ID = 0
--- level 0 ---
level number = 0x0 (0)
level type = invalid (0)
bit width of level = 0x0 (0)
number of logical processors at level = 0x0 (0)
XSAVE features (0xd/0):
XCR0 valid bit field mask = 0x4000000000000007
x87 state = true
SSE state = true
AVX state = true
MPX BNDREGS = false
MPX BNDCSR = false
AVX-512 opmask = false
AVX-512 ZMM_Hi256 = false
AVX-512 Hi16_ZMM = false
PKRU state = false
XTILECFG state = false
XTILEDATA state = false
bytes required by fields in XCR0 = 0x00000340 (832)
bytes required by XSAVE/XRSTOR area = 0x000003c0 (960)
XSAVEOPT instruction = false
XSAVEC instruction = false
XGETBV instruction = false
XSAVES/XRSTORS instructions = false
XFD: extended feature disable supported = false
SAVE area size in bytes = 0x00000000 (0)
IA32_XSS valid bit field mask = 0x0000000000000000
PT state = false
PASID state = false
CET_U user state = false
CET_S supervisor state = false
HDC state = false
UINTR state = false
LBR state = false
HWP state = false
AVX/YMM features (0xd/2):
AVX/YMM save state byte size = 0x00000100 (256)
AVX/YMM save state byte offset = 0x00000240 (576)
supported in IA32_XSS or XCR0 = XCR0 (user state)
64-byte alignment in compacted XSAVE = false
XFD faulting supported = false
LWP features (0xd/0x3e):
LWP save state byte size = 0x00000080 (128)
LWP save state byte offset = 0x00000340 (832)
supported in IA32_XSS or XCR0 = XCR0 (user state)
64-byte alignment in compacted XSAVE = false
XFD faulting supported = false
extended processor signature (0x80000001/eax):
family/generation = 0xf (15)
model = 0x3 (3)
stepping id = 0x1 (1)
extended family = 0x6 (6)
extended model = 0x1 (1)
(family synth) = 0x15 (21)
(model synth) = 0x13 (19)
(simple synth) = AMD (unknown type) (Richland RL-A1) [Piledriver], 32nm
extended feature flags (0x80000001/edx):
x87 FPU on chip = true
virtual-8086 mode enhancement = true
debugging extensions = true
page size extensions = true
time stamp counter = true
RDMSR and WRMSR support = true
physical address extensions = true
machine check exception = true
CMPXCHG8B inst. = true
APIC on chip = true
SYSCALL and SYSRET instructions = true
memory type range registers = true
global paging extension = true
machine check architecture = true
conditional move/compare instruction = true
page attribute table = true
page size extension = true
multiprocessing capable = false
no-execute page protection = true
AMD multimedia instruction extensions = true
MMX Technology = true
FXSAVE/FXRSTOR = true
SSE extensions = true
1-GB large page support = true
RDTSCP = true
long mode (AA-64) = true
3DNow! instruction extensions = false
3DNow! instructions = false
extended brand id (0x80000001/ebx):
raw = 0x20000000 (536870912)
BrandId = 0x0 (0)
PkgType = FM2 (PGA) (2)
AMD feature flags (0x80000001/ecx):
LAHF/SAHF supported in 64-bit mode = true
CMP Legacy = true
SVM: secure virtual machine = true
extended APIC space = true
AltMovCr8 = true
LZCNT advanced bit manipulation = true
SSE4A support = true
misaligned SSE mode = true
3DNow! PREFETCH/PREFETCHW instructions = true
OS visible workaround = true
instruction based sampling = true
XOP support = true
SKINIT/STGI support = true
watchdog timer support = true
lightweight profiling support = true
4-operand FMA instruction = true
TCE: translation cache extension = true
NodeId MSR C001100C = true
TBM support = true
topology extensions = true
core performance counter extensions = true
NB/DF performance counter extensions = true
data breakpoint extension = false
performance time-stamp counter support = false
LLC performance counter extensions = false
MWAITX/MONITORX supported = false
Address mask extension support = false
brand = "AMD A6-6400K APU with Radeon(tm) HD Graphics "
L1 TLB/cache information: 2M/4M pages & L1 TLB (0x80000005/eax):
instruction # entries = 0x18 (24)
instruction associativity = 0xff (255)
data # entries = 0x40 (64)
data associativity = 0xff (255)
L1 TLB/cache information: 4K pages & L1 TLB (0x80000005/ebx):
instruction # entries = 0x30 (48)
instruction associativity = 0xff (255)
data # entries = 0x40 (64)
data associativity = 0xff (255)
L1 data cache information (0x80000005/ecx):
line size (bytes) = 0x40 (64)
lines per tag = 0x1 (1)
associativity = 0x4 (4)
size (KB) = 0x10 (16)
L1 instruction cache information (0x80000005/edx):
line size (bytes) = 0x40 (64)
lines per tag = 0x1 (1)
associativity = 0x2 (2)
size (KB) = 0x40 (64)
L2 TLB/cache information: 2M/4M pages & L2 TLB (0x80000006/eax):
instruction # entries = 0x400 (1024)
instruction associativity = 8 to 15-way (6)
data # entries = 0x400 (1024)
data associativity = 8 to 15-way (6)
L2 TLB/cache information: 4K pages & L2 TLB (0x80000006/ebx):
instruction # entries = 0x200 (512)
instruction associativity = 4 to 5-way (4)
data # entries = 0x400 (1024)
data associativity = 8 to 15-way (6)
L2 unified cache information (0x80000006/ecx):
line size (bytes) = 0x40 (64)
lines per tag = 0x1 (1)
associativity = 16 to 31-way (8)
size (KB) = 0x400 (1024)
L3 cache information (0x80000006/edx):
line size (bytes) = 0x0 (0)
lines per tag = 0x0 (0)
associativity = L2 off (0)
size (in 512KB units) = 0x0 (0)
RAS Capability (0x80000007/ebx):
MCA overflow recovery support = false
SUCCOR support = false
HWA: hardware assert support = false
scalable MCA support = false
Advanced Power Management Features (0x80000007/ecx):
CmpUnitPwrSampleTimeRatio = 0x0 (0)
Advanced Power Management Features (0x80000007/edx):
TS: temperature sensing diode = true
FID: frequency ID control = false
VID: voltage ID control = false
TTP: thermal trip = true
TM: thermal monitor = true
STC: software thermal control = false
100 MHz multiplier control = true
hardware P-State control = true
TscInvariant = true
CPB: core performance boost = true
read-only effective frequency interface = true
processor feedback interface = false
APM power reporting = false
connected standby = false
RAPL: running average power limit = false
Physical Address and Linear Address Size (0x80000008/eax):
maximum physical address bits = 0x30 (48)
maximum linear (virtual) address bits = 0x30 (48)
maximum guest physical address bits = 0x0 (0)
Extended Feature Extensions ID (0x80000008/ebx):
CLZERO instruction = false
instructions retired count support = false
always save/restore error pointers = false
INVLPGB instruction = false
RDPRU instruction = false
memory bandwidth enforcement = false
MCOMMIT instruction = false
WBNOINVD instruction = false
IBPB: indirect branch prediction barrier = true
interruptible WBINVD, WBNOINVD = false
IBRS: indirect branch restr speculation = false
STIBP: 1 thr indirect branch predictor = false
CPU prefers: IBRS always on = false
CPU prefers: STIBP always on = false
IBRS preferred over software solution = false
IBRS provides same mode protection = false
EFER[LMSLE] not supported = false
INVLPGB supports TLB flush guest nested = false
ppin processor id number supported = false
SSBD: speculative store bypass disable = false
virtualized SSBD = false
SSBD fixed in hardware = false
CPPC: collaborative processor perf ctrl = false
PSFD: predictive store forward disable = false
not vulnerable to branch type confusion = false
branch sampling feature support = false
(vuln to branch type confusion synth) = true
Size Identifiers (0x80000008/ecx):
number of CPU cores = 0x2 (2)
ApicIdCoreIdSize = 0x4 (4)
performance time-stamp counter size = 40 bits (0)
Feature Extended Size (0x80000008/edx):
max page count for INVLPGB instruction = 0x0 (0)
RDPRU instruction max input support = 0x0 (0)
SVM Secure Virtual Machine (0x8000000a/eax):
SvmRev: SVM revision = 0x1 (1)
SVM Secure Virtual Machine (0x8000000a/edx):
nested paging = true
LBR virtualization = true
SVM lock = true
NRIP save = true
MSR based TSC rate control = true
VMCB clean bits support = true
flush by ASID = true
decode assists = true
SSSE3/SSE5 opcode set disable = false
pause intercept filter = true
pause filter threshold = true
AVIC: AMD virtual interrupt controller = false
virtualized VMLOAD/VMSAVE = false
virtualized global interrupt flag (GIF) = false
GMET: guest mode execute trap = false
X2AVIC: virtualized X2APIC = false
supervisor shadow stack = false
guest Spec_ctl support = false
ROGPT: read-only guest page table = false
host MCE override = false
INVLPGB/TLBSYNC hyperv interc enable = false
VNMI: NMI virtualization = false
IBS virtualization = false
guest SVME addr check = false
NASID: number of address space identifiers = 0x10000 (65536):
L1 TLB information: 1G pages (0x80000019/eax):
instruction # entries = 0x18 (24)
instruction associativity = full (15)
data # entries = 0x40 (64)
data associativity = full (15)
L2 TLB information: 1G pages (0x80000019/ebx):
instruction # entries = 0x400 (1024)
instruction associativity = 8 to 15-way (6)
data # entries = 0x400 (1024)
data associativity = 8 to 15-way (6)
Performance Optimization Identifiers (0x8000001a/eax):
128-bit SSE executed full-width = true
MOVU* better than MOVL*/MOVH* = true
256-bit SSE executed full-width = false
Instruction Based Sampling Identifiers (0x8000001b/eax):
IBS feature flags valid = true
IBS fetch sampling = true
IBS execution sampling = true
read write of op counter = true
op counting mode = true
branch target address reporting = true
IbsOpCurCnt and IbsOpMaxCnt extend 7 = true
invalid RIP indication support = true
fused branch micro-op indication support = false
IBS fetch control extended MSR support = false
IBS op data 4 MSR support = false
IBS L3 miss filtering support = false
Lightweight Profiling Capabilities: Availability (0x8000001c/eax):
lightweight profiling = false
LWPVAL instruction = false
instruction retired event = false
branch retired event = false
DC miss event = false
core clocks not halted event = false
core reference clocks not halted event = false
continuous mode sampling = false
tsc in event record = false
interrupt on threshold overflow = false
Lightweight Profiling Capabilities: Supported (0x8000001c/edx):
lightweight profiling = true
LWPVAL instruction = true
instruction retired event = true
branch retired event = true
DC miss event = false
core clocks not halted event = false
core reference clocks not halted event = false
continuous mode sampling = false
tsc in event record = false
interrupt on threshold overflow = true
Lightweight Profiling Capabilities (0x8000001c/ebx):
LWPCB byte size = 0x13 (19)
event record byte size = 0x20 (32)
maximum EventId = 0x3 (3)
EventInterval1 field offset = 0x80 (128)
Lightweight Profiling Capabilities (0x8000001c/ecx):
latency counter bit size = 0x0 (0)
data cache miss address valid = false
amount cache latency is rounded = 0x0 (0)
LWP implementation version = 0x1 (1)
event ring buffer size in records = 0x1 (1)
branch prediction filtering = false
IP filtering = false
cache level filtering = false
cache latency filteing = false
Cache Properties (0x8000001d):
--- cache 0 ---
type = data (1)
level = 0x1 (1)
self-initializing = true
fully associative = false
extra cores sharing this cache = 0x0 (0)
line size in bytes = 0x40 (64)
physical line partitions = 0x1 (1)
number of ways = 0x4 (4)
number of sets = 64
write-back invalidate = false
cache inclusive of lower levels = false
(synth size) = 16384 (16 KB)
--- cache 1 ---
type = instruction (2)
level = 0x1 (1)
self-initializing = true
fully associative = false
extra cores sharing this cache = 0x1 (1)
line size in bytes = 0x40 (64)
physical line partitions = 0x1 (1)
number of ways = 0x2 (2)
number of sets = 512
write-back invalidate = false
cache inclusive of lower levels = false
(synth size) = 65536 (64 KB)
--- cache 2 ---
type = unified (3)
level = 0x2 (2)
self-initializing = true
fully associative = false
extra cores sharing this cache = 0x1 (1)
line size in bytes = 0x40 (64)
physical line partitions = 0x1 (1)
number of ways = 0x10 (16)
number of sets = 1024
write-back invalidate = true
cache inclusive of lower levels = false
(synth size) = 1048576 (1024 KB)
extended APIC ID = 16
Compute Unit Identifiers (0x8000001e/ebx):
compute unit ID = 0x0 (0)
cores per compute unit = 0x2 (2)
Node Identifiers (0x8000001e/ecx):
node ID = 0x0 (0)
nodes per processor = 0x1 (1)
(instruction supported synth):
CMPXCHG8B = true
conditional move/compare = true
PREFETCH/PREFETCHW = true
(multi-processing synth) = multi-core (c=2)
(multi-processing method) = AMD
(APIC widths synth): CORE_width=1 SMT_width=0
(APIC synth): PKG_ID=0 CORE_ID=0 SMT_ID=0
(uarch synth) = AMD Piledriver, 32nm
(synth) = AMD A-Series (Richland RL-A1) [Piledriver], 32nm
CPU 1:
vendor_id = "AuthenticAMD"
version information (1/eax):
processor type = primary processor (0)
family = 0xf (15)
model = 0x3 (3)
stepping id = 0x1 (1)
extended family = 0x6 (6)
extended model = 0x1 (1)
(family synth) = 0x15 (21)
(model synth) = 0x13 (19)
(simple synth) = AMD (unknown type) (Richland RL-A1) [Piledriver], 32nm
miscellaneous (1/ebx):
process local APIC physical ID = 0x1 (1)
maximum IDs for CPUs in pkg = 0x2 (2)
CLFLUSH line size = 0x8 (8)
brand index = 0x0 (0)
brand id = 0x00 (0): unknown
feature information (1/edx):
x87 FPU on chip = true
VME: virtual-8086 mode enhancement = true
DE: debugging extensions = true
PSE: page size extensions = true
TSC: time stamp counter = true
RDMSR and WRMSR support = true
PAE: physical address extensions = true
MCE: machine check exception = true
CMPXCHG8B inst. = true
APIC on chip = true
SYSENTER and SYSEXIT = true
MTRR: memory type range registers = true
PTE global bit = true
MCA: machine check architecture = true
CMOV: conditional move/compare instr = true
PAT: page attribute table = true
PSE-36: page size extension = true
PSN: processor serial number = false
CLFLUSH instruction = true
DS: debug store = false
ACPI: thermal monitor and clock ctrl = false
MMX Technology = true
FXSAVE/FXRSTOR = true
SSE extensions = true
SSE2 extensions = true
SS: self snoop = false
hyper-threading / multi-core supported = true
TM: therm. monitor = false
IA64 = false
PBE: pending break event = false
feature information (1/ecx):
PNI/SSE3: Prescott New Instructions = true
PCLMULDQ instruction = true
DTES64: 64-bit debug store = false
MONITOR/MWAIT = true
CPL-qualified debug store = false
VMX: virtual machine extensions = false
SMX: safer mode extensions = false
Enhanced Intel SpeedStep Technology = false
TM2: thermal monitor 2 = false
SSSE3 extensions = true
context ID: adaptive or shared L1 data = false
SDBG: IA32_DEBUG_INTERFACE = false
FMA instruction = true
CMPXCHG16B instruction = true
xTPR disable = false
PDCM: perfmon and debug = false
PCID: process context identifiers = false
DCA: direct cache access = false
SSE4.1 extensions = true
SSE4.2 extensions = true
x2APIC: extended xAPIC support = false
MOVBE instruction = false
POPCNT instruction = true
time stamp counter deadline = false
AES instruction = true
XSAVE/XSTOR states = true
OS-enabled XSAVE/XSTOR = true
AVX: advanced vector extensions = true
F16C half-precision convert instruction = true
RDRAND instruction = false
hypervisor guest status = false
cache and TLB information (2):
processor serial number = 0061-0F31-0000-0000-0000-0000
deterministic cache parameters (4):
--- cache 0 ---
cache type = no more caches (0)
MONITOR/MWAIT (5):
smallest monitor-line size (bytes) = 0x40 (64)
largest monitor-line size (bytes) = 0x40 (64)
enum of Monitor-MWAIT exts supported = true
supports intrs as break-event for MWAIT = true
number of C0 sub C-states using MWAIT = 0x0 (0)
number of C1 sub C-states using MWAIT = 0x0 (0)
number of C2 sub C-states using MWAIT = 0x0 (0)
number of C3 sub C-states using MWAIT = 0x0 (0)
number of C4 sub C-states using MWAIT = 0x0 (0)
number of C5 sub C-states using MWAIT = 0x0 (0)
number of C6 sub C-states using MWAIT = 0x0 (0)
number of C7 sub C-states using MWAIT = 0x0 (0)
Thermal and Power Management Features (6):
digital thermometer = false
Intel Turbo Boost Technology = false
ARAT always running APIC timer = false
PLN power limit notification = false
ECMD extended clock modulation duty = false
PTM package thermal management = false
HWP base registers = false
HWP notification = false
HWP activity window = false
HWP energy performance preference = false
HWP package level request = false
HDC base registers = false
Intel Turbo Boost Max Technology 3.0 = false
HWP capabilities = false
HWP PECI override = false
flexible HWP = false
IA32_HWP_REQUEST MSR fast access mode = false
HW_FEEDBACK MSRs supported = false
ignoring idle logical processor HWP req = false
Thread Director = false
IA32_HW_FEEDBACK_THREAD_CONFIG bit 25 = false
digital thermometer thresholds = 0x0 (0)
hardware coordination feedback = true
ACNT2 available = false
performance-energy bias capability = false
number of enh hardware feedback classes = 0x0 (0)
performance capability reporting = false
energy efficiency capability reporting = false
size of feedback struct (4KB pages) = 0x1 (1)
index of CPU's row in feedback struct = 0x0 (0)
extended feature flags (7):
FSGSBASE instructions = false
IA32_TSC_ADJUST MSR supported = false
SGX: Software Guard Extensions supported = false
BMI1 instructions = true
HLE hardware lock elision = false
AVX2: advanced vector extensions 2 = false
FDP_EXCPTN_ONLY = false
SMEP supervisor mode exec protection = false
BMI2 instructions = false
enhanced REP MOVSB/STOSB = false
INVPCID instruction = false
RTM: restricted transactional memory = false
RDT-CMT/PQoS cache monitoring = false
deprecated FPU CS/DS = false
MPX: intel memory protection extensions = false
RDT-CAT/PQE cache allocation = false
AVX512F: AVX-512 foundation instructions = false
AVX512DQ: double & quadword instructions = false
RDSEED instruction = false
ADX instructions = false
SMAP: supervisor mode access prevention = false
AVX512IFMA: integer fused multiply add = false
PCOMMIT instruction = false
CLFLUSHOPT instruction = false
CLWB instruction = false
Intel processor trace = false
AVX512PF: prefetch instructions = false
AVX512ER: exponent & reciprocal instrs = false
AVX512CD: conflict detection instrs = false
SHA instructions = false
AVX512BW: byte & word instructions = false
AVX512VL: vector length = false
PREFETCHWT1 = false
AVX512VBMI: vector byte manipulation = false
UMIP: user-mode instruction prevention = false
PKU protection keys for user-mode = false
OSPKE CR4.PKE and RDPKRU/WRPKRU = false
WAITPKG instructions = false
AVX512_VBMI2: byte VPCOMPRESS, VPEXPAND = false
CET_SS: CET shadow stack = false
GFNI: Galois Field New Instructions = false
VAES instructions = false
VPCLMULQDQ instruction = false
AVX512_VNNI: neural network instructions = false
AVX512_BITALG: bit count/shiffle = false
TME: Total Memory Encryption = false
AVX512: VPOPCNTDQ instruction = false
LA57: 57-bit addrs & 5-level paging = false
BNDLDX/BNDSTX MAWAU value in 64-bit mode = 0x0 (0)
RDPID: read processor ID supported = false
KL: key locker = false
bus lock detection = false
CLDEMOTE supports cache line demote = false
MOVDIRI instruction = false
MOVDIR64B instruction = false
ENQCMD instruction = false
SGX_LC: SGX launch config supported = false
PKS: supervisor protection keys = false
SGX-KEYS: SGX attestation services = false
AVX512_4VNNIW: neural network instrs = false
AVX512_4FMAPS: multiply acc single prec = false
fast short REP MOV = false
UINTR: user interrupts = false
AVX512_VP2INTERSECT: intersect mask regs = false
IA32_MCU_OPT_CTRL SRBDS mitigation MSR = false
VERW MD_CLEAR microcode support = false
RTM transaction always aborts = false
IA32_TSX_FORCE_ABORT MSR = false
SERIALIZE instruction = false
hybrid part = false
TSXLDTRK: TSX suspend load addr tracking = false
PCONFIG instruction = false
LBR: architectural last branch records = false
CET_IBT: CET indirect branch tracking = false
AMX-BF16: tile bfloat16 support = false
AVX512_FP16: fp16 support = false
AMX-TILE: tile architecture support = false
AMX-INT8: tile 8-bit integer support = false
IBRS/IBPB: indirect branch restrictions = false
STIBP: 1 thr indirect branch predictor = false
L1D_FLUSH: IA32_FLUSH_CMD MSR = false
IA32_ARCH_CAPABILITIES MSR = false
IA32_CORE_CAPABILITIES MSR = false
SSBD: speculative store bypass disable = false
Direct Cache Access Parameters (9):
PLATFORM_DCA_CAP MSR bits = 0
Architecture Performance Monitoring Features (0xa):
version ID = 0x0 (0)
number of counters per logical processor = 0x0 (0)
bit width of counter = 0x0 (0)
length of EBX bit vector = 0x0 (0)
core cycle event = not available
instruction retired event = not available
reference cycles event = not available
last-level cache ref event = not available
last-level cache miss event = not available
branch inst retired event = not available
branch mispred retired event = not available
top-down slots event = not available
fixed counter 0 supported = false
fixed counter 1 supported = false
fixed counter 2 supported = false
fixed counter 3 supported = false
fixed counter 4 supported = false
fixed counter 5 supported = false
fixed counter 6 supported = false
fixed counter 7 supported = false
fixed counter 8 supported = false
fixed counter 9 supported = false
fixed counter 10 supported = false
fixed counter 11 supported = false
fixed counter 12 supported = false
fixed counter 13 supported = false
fixed counter 14 supported = false
fixed counter 15 supported = false
fixed counter 16 supported = false
fixed counter 17 supported = false
fixed counter 18 supported = false
fixed counter 19 supported = false
fixed counter 20 supported = false
fixed counter 21 supported = false
fixed counter 22 supported = false
fixed counter 23 supported = false
fixed counter 24 supported = false
fixed counter 25 supported = false
fixed counter 26 supported = false
fixed counter 27 supported = false
fixed counter 28 supported = false
fixed counter 29 supported = false
fixed counter 30 supported = false
fixed counter 31 supported = false
number of contiguous fixed counters = 0x0 (0)
bit width of fixed counters = 0x0 (0)
anythread deprecation = false
x2APIC features / processor topology (0xb):
extended APIC ID = 0
--- level 0 ---
level number = 0x0 (0)
level type = invalid (0)
bit width of level = 0x0 (0)
number of logical processors at level = 0x0 (0)
XSAVE features (0xd/0):
XCR0 valid bit field mask = 0x4000000000000007
x87 state = true
SSE state = true
AVX state = true
MPX BNDREGS = false
MPX BNDCSR = false
AVX-512 opmask = false
AVX-512 ZMM_Hi256 = false
AVX-512 Hi16_ZMM = false
PKRU state = false
XTILECFG state = false
XTILEDATA state = false
bytes required by fields in XCR0 = 0x00000340 (832)
bytes required by XSAVE/XRSTOR area = 0x000003c0 (960)
XSAVEOPT instruction = false
XSAVEC instruction = false
XGETBV instruction = false
XSAVES/XRSTORS instructions = false
XFD: extended feature disable supported = false
SAVE area size in bytes = 0x00000000 (0)
IA32_XSS valid bit field mask = 0x0000000000000000
PT state = false
PASID state = false
CET_U user state = false
CET_S supervisor state = false
HDC state = false
UINTR state = false
LBR state = false
HWP state = false
AVX/YMM features (0xd/2):
AVX/YMM save state byte size = 0x00000100 (256)
AVX/YMM save state byte offset = 0x00000240 (576)
supported in IA32_XSS or XCR0 = XCR0 (user state)
64-byte alignment in compacted XSAVE = false
XFD faulting supported = false
LWP features (0xd/0x3e):
LWP save state byte size = 0x00000080 (128)
LWP save state byte offset = 0x00000340 (832)
supported in IA32_XSS or XCR0 = XCR0 (user state)
64-byte alignment in compacted XSAVE = false
XFD faulting supported = false
extended processor signature (0x80000001/eax):
family/generation = 0xf (15)
model = 0x3 (3)
stepping id = 0x1 (1)
extended family = 0x6 (6)
extended model = 0x1 (1)
(family synth) = 0x15 (21)
(model synth) = 0x13 (19)
(simple synth) = AMD (unknown type) (Richland RL-A1) [Piledriver], 32nm
extended feature flags (0x80000001/edx):
x87 FPU on chip = true
virtual-8086 mode enhancement = true
debugging extensions = true
page size extensions = true
time stamp counter = true
RDMSR and WRMSR support = true
physical address extensions = true
machine check exception = true
CMPXCHG8B inst. = true
APIC on chip = true
SYSCALL and SYSRET instructions = true
memory type range registers = true
global paging extension = true
machine check architecture = true
conditional move/compare instruction = true
page attribute table = true
page size extension = true
multiprocessing capable = false
no-execute page protection = true
AMD multimedia instruction extensions = true
MMX Technology = true
FXSAVE/FXRSTOR = true
SSE extensions = true
1-GB large page support = true
RDTSCP = true
long mode (AA-64) = true
3DNow! instruction extensions = false
3DNow! instructions = false
extended brand id (0x80000001/ebx):
raw = 0x20000000 (536870912)
BrandId = 0x0 (0)
PkgType = FM2 (PGA) (2)
AMD feature flags (0x80000001/ecx):
LAHF/SAHF supported in 64-bit mode = true
CMP Legacy = true
SVM: secure virtual machine = true
extended APIC space = true
AltMovCr8 = true
LZCNT advanced bit manipulation = true
SSE4A support = true
misaligned SSE mode = true
3DNow! PREFETCH/PREFETCHW instructions = true
OS visible workaround = true
instruction based sampling = true
XOP support = true
SKINIT/STGI support = true
watchdog timer support = true
lightweight profiling support = true
4-operand FMA instruction = true
TCE: translation cache extension = true
NodeId MSR C001100C = true
TBM support = true
topology extensions = true
core performance counter extensions = true
NB/DF performance counter extensions = true
data breakpoint extension = false
performance time-stamp counter support = false
LLC performance counter extensions = false
MWAITX/MONITORX supported = false
Address mask extension support = false
brand = "AMD A6-6400K APU with Radeon(tm) HD Graphics "
L1 TLB/cache information: 2M/4M pages & L1 TLB (0x80000005/eax):
instruction # entries = 0x18 (24)
instruction associativity = 0xff (255)
data # entries = 0x40 (64)
data associativity = 0xff (255)
L1 TLB/cache information: 4K pages & L1 TLB (0x80000005/ebx):
instruction # entries = 0x30 (48)
instruction associativity = 0xff (255)
data # entries = 0x40 (64)
data associativity = 0xff (255)
L1 data cache information (0x80000005/ecx):
line size (bytes) = 0x40 (64)
lines per tag = 0x1 (1)
associativity = 0x4 (4)
size (KB) = 0x10 (16)
L1 instruction cache information (0x80000005/edx):
line size (bytes) = 0x40 (64)
lines per tag = 0x1 (1)
associativity = 0x2 (2)
size (KB) = 0x40 (64)
L2 TLB/cache information: 2M/4M pages & L2 TLB (0x80000006/eax):
instruction # entries = 0x400 (1024)
instruction associativity = 8 to 15-way (6)
data # entries = 0x400 (1024)
data associativity = 8 to 15-way (6)
L2 TLB/cache information: 4K pages & L2 TLB (0x80000006/ebx):
instruction # entries = 0x200 (512)
instruction associativity = 4 to 5-way (4)
data # entries = 0x400 (1024)
data associativity = 8 to 15-way (6)
L2 unified cache information (0x80000006/ecx):
line size (bytes) = 0x40 (64)
lines per tag = 0x1 (1)
associativity = 16 to 31-way (8)
size (KB) = 0x400 (1024)
L3 cache information (0x80000006/edx):
line size (bytes) = 0x0 (0)
lines per tag = 0x0 (0)
associativity = L2 off (0)
size (in 512KB units) = 0x0 (0)
RAS Capability (0x80000007/ebx):
MCA overflow recovery support = false
SUCCOR support = false
HWA: hardware assert support = false
scalable MCA support = false
Advanced Power Management Features (0x80000007/ecx):
CmpUnitPwrSampleTimeRatio = 0x0 (0)
Advanced Power Management Features (0x80000007/edx):
TS: temperature sensing diode = true
FID: frequency ID control = false
VID: voltage ID control = false
TTP: thermal trip = true
TM: thermal monitor = true
STC: software thermal control = false
100 MHz multiplier control = true
hardware P-State control = true
TscInvariant = true
CPB: core performance boost = true
read-only effective frequency interface = true
processor feedback interface = false
APM power reporting = false
connected standby = false
RAPL: running average power limit = false
Physical Address and Linear Address Size (0x80000008/eax):
maximum physical address bits = 0x30 (48)
maximum linear (virtual) address bits = 0x30 (48)
maximum guest physical address bits = 0x0 (0)
Extended Feature Extensions ID (0x80000008/ebx):
CLZERO instruction = false
instructions retired count support = false
always save/restore error pointers = false
INVLPGB instruction = false
RDPRU instruction = false
memory bandwidth enforcement = false
MCOMMIT instruction = false
WBNOINVD instruction = false
IBPB: indirect branch prediction barrier = true
interruptible WBINVD, WBNOINVD = false
IBRS: indirect branch restr speculation = false
STIBP: 1 thr indirect branch predictor = false
CPU prefers: IBRS always on = false
CPU prefers: STIBP always on = false
IBRS preferred over software solution = false
IBRS provides same mode protection = false
EFER[LMSLE] not supported = false
INVLPGB supports TLB flush guest nested = false
ppin processor id number supported = false
SSBD: speculative store bypass disable = false
virtualized SSBD = false
SSBD fixed in hardware = false
CPPC: collaborative processor perf ctrl = false
PSFD: predictive store forward disable = false
not vulnerable to branch type confusion = false
branch sampling feature support = false
(vuln to branch type confusion synth) = true
Size Identifiers (0x80000008/ecx):
number of CPU cores = 0x2 (2)
ApicIdCoreIdSize = 0x4 (4)
performance time-stamp counter size = 40 bits (0)
Feature Extended Size (0x80000008/edx):
max page count for INVLPGB instruction = 0x0 (0)
RDPRU instruction max input support = 0x0 (0)
SVM Secure Virtual Machine (0x8000000a/eax):
SvmRev: SVM revision = 0x1 (1)
SVM Secure Virtual Machine (0x8000000a/edx):
nested paging = true
LBR virtualization = true
SVM lock = true
NRIP save = true
MSR based TSC rate control = true
VMCB clean bits support = true
flush by ASID = true
decode assists = true
SSSE3/SSE5 opcode set disable = false
pause intercept filter = true
pause filter threshold = true
AVIC: AMD virtual interrupt controller = false
virtualized VMLOAD/VMSAVE = false
virtualized global interrupt flag (GIF) = false
GMET: guest mode execute trap = false
X2AVIC: virtualized X2APIC = false
supervisor shadow stack = false
guest Spec_ctl support = false
ROGPT: read-only guest page table = false
host MCE override = false
INVLPGB/TLBSYNC hyperv interc enable = false
VNMI: NMI virtualization = false
IBS virtualization = false
guest SVME addr check = false
NASID: number of address space identifiers = 0x10000 (65536):
L1 TLB information: 1G pages (0x80000019/eax):
instruction # entries = 0x18 (24)
instruction associativity = full (15)
data # entries = 0x40 (64)
data associativity = full (15)
L2 TLB information: 1G pages (0x80000019/ebx):
instruction # entries = 0x400 (1024)
instruction associativity = 8 to 15-way (6)
data # entries = 0x400 (1024)
data associativity = 8 to 15-way (6)
Performance Optimization Identifiers (0x8000001a/eax):
128-bit SSE executed full-width = true
MOVU* better than MOVL*/MOVH* = true
256-bit SSE executed full-width = false
Instruction Based Sampling Identifiers (0x8000001b/eax):
IBS feature flags valid = true
IBS fetch sampling = true
IBS execution sampling = true
read write of op counter = true
op counting mode = true
branch target address reporting = true
IbsOpCurCnt and IbsOpMaxCnt extend 7 = true
invalid RIP indication support = true
fused branch micro-op indication support = false
IBS fetch control extended MSR support = false
IBS op data 4 MSR support = false
IBS L3 miss filtering support = false
Lightweight Profiling Capabilities: Availability (0x8000001c/eax):
lightweight profiling = false
LWPVAL instruction = false
instruction retired event = false
branch retired event = false
DC miss event = false
core clocks not halted event = false
core reference clocks not halted event = false
continuous mode sampling = false
tsc in event record = false
interrupt on threshold overflow = false
Lightweight Profiling Capabilities: Supported (0x8000001c/edx):
lightweight profiling = true
LWPVAL instruction = true
instruction retired event = true
branch retired event = true
DC miss event = false
core clocks not halted event = false
core reference clocks not halted event = false
continuous mode sampling = false
tsc in event record = false
interrupt on threshold overflow = true
Lightweight Profiling Capabilities (0x8000001c/ebx):
LWPCB byte size = 0x13 (19)
event record byte size = 0x20 (32)
maximum EventId = 0x3 (3)
EventInterval1 field offset = 0x80 (128)
Lightweight Profiling Capabilities (0x8000001c/ecx):
latency counter bit size = 0x0 (0)
data cache miss address valid = false
amount cache latency is rounded = 0x0 (0)
LWP implementation version = 0x1 (1)
event ring buffer size in records = 0x1 (1)
branch prediction filtering = false
IP filtering = false
cache level filtering = false
cache latency filteing = false
Cache Properties (0x8000001d):
--- cache 0 ---
type = data (1)
level = 0x1 (1)
self-initializing = true
fully associative = false
extra cores sharing this cache = 0x0 (0)
line size in bytes = 0x40 (64)
physical line partitions = 0x1 (1)
number of ways = 0x4 (4)
number of sets = 64
write-back invalidate = false
cache inclusive of lower levels = false
(synth size) = 16384 (16 KB)
--- cache 1 ---
type = instruction (2)
level = 0x1 (1)
self-initializing = true
fully associative = false
extra cores sharing this cache = 0x1 (1)
line size in bytes = 0x40 (64)
physical line partitions = 0x1 (1)
number of ways = 0x2 (2)
number of sets = 512
write-back invalidate = false
cache inclusive of lower levels = false
(synth size) = 65536 (64 KB)
--- cache 2 ---
type = unified (3)
level = 0x2 (2)
self-initializing = true
fully associative = false
extra cores sharing this cache = 0x1 (1)
line size in bytes = 0x40 (64)
physical line partitions = 0x1 (1)
number of ways = 0x10 (16)
number of sets = 1024
write-back invalidate = true
cache inclusive of lower levels = false
(synth size) = 1048576 (1024 KB)
extended APIC ID = 17
Compute Unit Identifiers (0x8000001e/ebx):
compute unit ID = 0x0 (0)
cores per compute unit = 0x2 (2)
Node Identifiers (0x8000001e/ecx):
node ID = 0x0 (0)
nodes per processor = 0x1 (1)
(instruction supported synth):
CMPXCHG8B = true
conditional move/compare = true
PREFETCH/PREFETCHW = true
(multi-processing synth) = multi-core (c=2)
(multi-processing method) = AMD
(APIC widths synth): CORE_width=1 SMT_width=0
(APIC synth): PKG_ID=0 CORE_ID=1 SMT_ID=0
(uarch synth) = AMD Piledriver, 32nm
(synth) = AMD A-Series (Richland RL-A1) [Piledriver], 32nm
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-18 20:10 ` Paul Menzel
0 siblings, 0 replies; 236+ messages in thread
From: Paul Menzel @ 2023-04-18 20:10 UTC (permalink / raw)
To: Thomas Gleixner
Cc: linux-kernel, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
[-- Attachment #1: Type: text/plain, Size: 1924 bytes --]
Dear Thomas,
Am 18.04.23 um 10:40 schrieb Thomas Gleixner:
> On Tue, Apr 18 2023 at 08:58, Thomas Gleixner wrote:
>> On Mon, Apr 17 2023 at 19:40, Paul Menzel wrote:
>>> Am 17.04.23 um 16:48 schrieb Thomas Gleixner:
>>>
>>>> On Mon, Apr 17 2023 at 13:19, Paul Menzel wrote:
>>>>> Am 15.04.23 um 01:44 schrieb Thomas Gleixner:
>>>>> [ 0.258193] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD Graphics (family: 0x15, model: 0x13, stepping: 0x1)
>>>>> […]
>>>>> [ 0.259329] smp: Bringing up secondary CPUs ...
>>>>> [ 0.259527] x86: Booting SMP configuration:
>>>>> [ 0.259528] .... node #0, CPUs: #1
>>>>> [ 0.261007] After schedule_preempt_disabled
>>>>> [ 10.260990] CPU1 failed to report alive state
>>>>
>>>> Weird. CPU1 fails to come up and report that it has reached the
>>>> synchronization point.
>>>>
>>>> Does it work when you add cpuhp.parallel=off on the kernel command line?
>>>
>>> Yes, the ten seconds delay is gone with `cpuhp.parallel=off`.
>>>
>>> There was a patch set in the past, that worked on that device. I think
>>> up to v4 it did *not* work at all and hung [1]. I need some days to
>>> collect the results again.
>>
>> Can you please apply the patch below on top of the pile remove the
>> command line option again?
>
> Bah. That patch does not make any sense at all. Not enough coffee.
>
> Can you please provide the output of cpuid?
Of course. Here the top, and the whole output is attached.
```
CPU 0:
vendor_id = "AuthenticAMD"
version information (1/eax):
processor type = primary processor (0)
family = 0xf (15)
model = 0x3 (3)
stepping id = 0x1 (1)
extended family = 0x6 (6)
extended model = 0x1 (1)
(family synth) = 0x15 (21)
(model synth) = 0x13 (19)
(simple synth) = AMD (unknown type) (Richland RL-A1)
[Piledriver], 32nm
[…]
```
Kind regards,
Paul
[-- Attachment #2: cpuid.txt --]
[-- Type: text/plain, Size: 65008 bytes --]
CPU 0:
vendor_id = "AuthenticAMD"
version information (1/eax):
processor type = primary processor (0)
family = 0xf (15)
model = 0x3 (3)
stepping id = 0x1 (1)
extended family = 0x6 (6)
extended model = 0x1 (1)
(family synth) = 0x15 (21)
(model synth) = 0x13 (19)
(simple synth) = AMD (unknown type) (Richland RL-A1) [Piledriver], 32nm
miscellaneous (1/ebx):
process local APIC physical ID = 0x0 (0)
maximum IDs for CPUs in pkg = 0x2 (2)
CLFLUSH line size = 0x8 (8)
brand index = 0x0 (0)
brand id = 0x00 (0): unknown
feature information (1/edx):
x87 FPU on chip = true
VME: virtual-8086 mode enhancement = true
DE: debugging extensions = true
PSE: page size extensions = true
TSC: time stamp counter = true
RDMSR and WRMSR support = true
PAE: physical address extensions = true
MCE: machine check exception = true
CMPXCHG8B inst. = true
APIC on chip = true
SYSENTER and SYSEXIT = true
MTRR: memory type range registers = true
PTE global bit = true
MCA: machine check architecture = true
CMOV: conditional move/compare instr = true
PAT: page attribute table = true
PSE-36: page size extension = true
PSN: processor serial number = false
CLFLUSH instruction = true
DS: debug store = false
ACPI: thermal monitor and clock ctrl = false
MMX Technology = true
FXSAVE/FXRSTOR = true
SSE extensions = true
SSE2 extensions = true
SS: self snoop = false
hyper-threading / multi-core supported = true
TM: therm. monitor = false
IA64 = false
PBE: pending break event = false
feature information (1/ecx):
PNI/SSE3: Prescott New Instructions = true
PCLMULDQ instruction = true
DTES64: 64-bit debug store = false
MONITOR/MWAIT = true
CPL-qualified debug store = false
VMX: virtual machine extensions = false
SMX: safer mode extensions = false
Enhanced Intel SpeedStep Technology = false
TM2: thermal monitor 2 = false
SSSE3 extensions = true
context ID: adaptive or shared L1 data = false
SDBG: IA32_DEBUG_INTERFACE = false
FMA instruction = true
CMPXCHG16B instruction = true
xTPR disable = false
PDCM: perfmon and debug = false
PCID: process context identifiers = false
DCA: direct cache access = false
SSE4.1 extensions = true
SSE4.2 extensions = true
x2APIC: extended xAPIC support = false
MOVBE instruction = false
POPCNT instruction = true
time stamp counter deadline = false
AES instruction = true
XSAVE/XSTOR states = true
OS-enabled XSAVE/XSTOR = true
AVX: advanced vector extensions = true
F16C half-precision convert instruction = true
RDRAND instruction = false
hypervisor guest status = false
cache and TLB information (2):
processor serial number = 0061-0F31-0000-0000-0000-0000
deterministic cache parameters (4):
--- cache 0 ---
cache type = no more caches (0)
MONITOR/MWAIT (5):
smallest monitor-line size (bytes) = 0x40 (64)
largest monitor-line size (bytes) = 0x40 (64)
enum of Monitor-MWAIT exts supported = true
supports intrs as break-event for MWAIT = true
number of C0 sub C-states using MWAIT = 0x0 (0)
number of C1 sub C-states using MWAIT = 0x0 (0)
number of C2 sub C-states using MWAIT = 0x0 (0)
number of C3 sub C-states using MWAIT = 0x0 (0)
number of C4 sub C-states using MWAIT = 0x0 (0)
number of C5 sub C-states using MWAIT = 0x0 (0)
number of C6 sub C-states using MWAIT = 0x0 (0)
number of C7 sub C-states using MWAIT = 0x0 (0)
Thermal and Power Management Features (6):
digital thermometer = false
Intel Turbo Boost Technology = false
ARAT always running APIC timer = false
PLN power limit notification = false
ECMD extended clock modulation duty = false
PTM package thermal management = false
HWP base registers = false
HWP notification = false
HWP activity window = false
HWP energy performance preference = false
HWP package level request = false
HDC base registers = false
Intel Turbo Boost Max Technology 3.0 = false
HWP capabilities = false
HWP PECI override = false
flexible HWP = false
IA32_HWP_REQUEST MSR fast access mode = false
HW_FEEDBACK MSRs supported = false
ignoring idle logical processor HWP req = false
Thread Director = false
IA32_HW_FEEDBACK_THREAD_CONFIG bit 25 = false
digital thermometer thresholds = 0x0 (0)
hardware coordination feedback = true
ACNT2 available = false
performance-energy bias capability = false
number of enh hardware feedback classes = 0x0 (0)
performance capability reporting = false
energy efficiency capability reporting = false
size of feedback struct (4KB pages) = 0x1 (1)
index of CPU's row in feedback struct = 0x0 (0)
extended feature flags (7):
FSGSBASE instructions = false
IA32_TSC_ADJUST MSR supported = false
SGX: Software Guard Extensions supported = false
BMI1 instructions = true
HLE hardware lock elision = false
AVX2: advanced vector extensions 2 = false
FDP_EXCPTN_ONLY = false
SMEP supervisor mode exec protection = false
BMI2 instructions = false
enhanced REP MOVSB/STOSB = false
INVPCID instruction = false
RTM: restricted transactional memory = false
RDT-CMT/PQoS cache monitoring = false
deprecated FPU CS/DS = false
MPX: intel memory protection extensions = false
RDT-CAT/PQE cache allocation = false
AVX512F: AVX-512 foundation instructions = false
AVX512DQ: double & quadword instructions = false
RDSEED instruction = false
ADX instructions = false
SMAP: supervisor mode access prevention = false
AVX512IFMA: integer fused multiply add = false
PCOMMIT instruction = false
CLFLUSHOPT instruction = false
CLWB instruction = false
Intel processor trace = false
AVX512PF: prefetch instructions = false
AVX512ER: exponent & reciprocal instrs = false
AVX512CD: conflict detection instrs = false
SHA instructions = false
AVX512BW: byte & word instructions = false
AVX512VL: vector length = false
PREFETCHWT1 = false
AVX512VBMI: vector byte manipulation = false
UMIP: user-mode instruction prevention = false
PKU protection keys for user-mode = false
OSPKE CR4.PKE and RDPKRU/WRPKRU = false
WAITPKG instructions = false
AVX512_VBMI2: byte VPCOMPRESS, VPEXPAND = false
CET_SS: CET shadow stack = false
GFNI: Galois Field New Instructions = false
VAES instructions = false
VPCLMULQDQ instruction = false
AVX512_VNNI: neural network instructions = false
AVX512_BITALG: bit count/shiffle = false
TME: Total Memory Encryption = false
AVX512: VPOPCNTDQ instruction = false
LA57: 57-bit addrs & 5-level paging = false
BNDLDX/BNDSTX MAWAU value in 64-bit mode = 0x0 (0)
RDPID: read processor ID supported = false
KL: key locker = false
bus lock detection = false
CLDEMOTE supports cache line demote = false
MOVDIRI instruction = false
MOVDIR64B instruction = false
ENQCMD instruction = false
SGX_LC: SGX launch config supported = false
PKS: supervisor protection keys = false
SGX-KEYS: SGX attestation services = false
AVX512_4VNNIW: neural network instrs = false
AVX512_4FMAPS: multiply acc single prec = false
fast short REP MOV = false
UINTR: user interrupts = false
AVX512_VP2INTERSECT: intersect mask regs = false
IA32_MCU_OPT_CTRL SRBDS mitigation MSR = false
VERW MD_CLEAR microcode support = false
RTM transaction always aborts = false
IA32_TSX_FORCE_ABORT MSR = false
SERIALIZE instruction = false
hybrid part = false
TSXLDTRK: TSX suspend load addr tracking = false
PCONFIG instruction = false
LBR: architectural last branch records = false
CET_IBT: CET indirect branch tracking = false
AMX-BF16: tile bfloat16 support = false
AVX512_FP16: fp16 support = false
AMX-TILE: tile architecture support = false
AMX-INT8: tile 8-bit integer support = false
IBRS/IBPB: indirect branch restrictions = false
STIBP: 1 thr indirect branch predictor = false
L1D_FLUSH: IA32_FLUSH_CMD MSR = false
IA32_ARCH_CAPABILITIES MSR = false
IA32_CORE_CAPABILITIES MSR = false
SSBD: speculative store bypass disable = false
Direct Cache Access Parameters (9):
PLATFORM_DCA_CAP MSR bits = 0
Architecture Performance Monitoring Features (0xa):
version ID = 0x0 (0)
number of counters per logical processor = 0x0 (0)
bit width of counter = 0x0 (0)
length of EBX bit vector = 0x0 (0)
core cycle event = not available
instruction retired event = not available
reference cycles event = not available
last-level cache ref event = not available
last-level cache miss event = not available
branch inst retired event = not available
branch mispred retired event = not available
top-down slots event = not available
fixed counter 0 supported = false
fixed counter 1 supported = false
fixed counter 2 supported = false
fixed counter 3 supported = false
fixed counter 4 supported = false
fixed counter 5 supported = false
fixed counter 6 supported = false
fixed counter 7 supported = false
fixed counter 8 supported = false
fixed counter 9 supported = false
fixed counter 10 supported = false
fixed counter 11 supported = false
fixed counter 12 supported = false
fixed counter 13 supported = false
fixed counter 14 supported = false
fixed counter 15 supported = false
fixed counter 16 supported = false
fixed counter 17 supported = false
fixed counter 18 supported = false
fixed counter 19 supported = false
fixed counter 20 supported = false
fixed counter 21 supported = false
fixed counter 22 supported = false
fixed counter 23 supported = false
fixed counter 24 supported = false
fixed counter 25 supported = false
fixed counter 26 supported = false
fixed counter 27 supported = false
fixed counter 28 supported = false
fixed counter 29 supported = false
fixed counter 30 supported = false
fixed counter 31 supported = false
number of contiguous fixed counters = 0x0 (0)
bit width of fixed counters = 0x0 (0)
anythread deprecation = false
x2APIC features / processor topology (0xb):
extended APIC ID = 0
--- level 0 ---
level number = 0x0 (0)
level type = invalid (0)
bit width of level = 0x0 (0)
number of logical processors at level = 0x0 (0)
XSAVE features (0xd/0):
XCR0 valid bit field mask = 0x4000000000000007
x87 state = true
SSE state = true
AVX state = true
MPX BNDREGS = false
MPX BNDCSR = false
AVX-512 opmask = false
AVX-512 ZMM_Hi256 = false
AVX-512 Hi16_ZMM = false
PKRU state = false
XTILECFG state = false
XTILEDATA state = false
bytes required by fields in XCR0 = 0x00000340 (832)
bytes required by XSAVE/XRSTOR area = 0x000003c0 (960)
XSAVEOPT instruction = false
XSAVEC instruction = false
XGETBV instruction = false
XSAVES/XRSTORS instructions = false
XFD: extended feature disable supported = false
SAVE area size in bytes = 0x00000000 (0)
IA32_XSS valid bit field mask = 0x0000000000000000
PT state = false
PASID state = false
CET_U user state = false
CET_S supervisor state = false
HDC state = false
UINTR state = false
LBR state = false
HWP state = false
AVX/YMM features (0xd/2):
AVX/YMM save state byte size = 0x00000100 (256)
AVX/YMM save state byte offset = 0x00000240 (576)
supported in IA32_XSS or XCR0 = XCR0 (user state)
64-byte alignment in compacted XSAVE = false
XFD faulting supported = false
LWP features (0xd/0x3e):
LWP save state byte size = 0x00000080 (128)
LWP save state byte offset = 0x00000340 (832)
supported in IA32_XSS or XCR0 = XCR0 (user state)
64-byte alignment in compacted XSAVE = false
XFD faulting supported = false
extended processor signature (0x80000001/eax):
family/generation = 0xf (15)
model = 0x3 (3)
stepping id = 0x1 (1)
extended family = 0x6 (6)
extended model = 0x1 (1)
(family synth) = 0x15 (21)
(model synth) = 0x13 (19)
(simple synth) = AMD (unknown type) (Richland RL-A1) [Piledriver], 32nm
extended feature flags (0x80000001/edx):
x87 FPU on chip = true
virtual-8086 mode enhancement = true
debugging extensions = true
page size extensions = true
time stamp counter = true
RDMSR and WRMSR support = true
physical address extensions = true
machine check exception = true
CMPXCHG8B inst. = true
APIC on chip = true
SYSCALL and SYSRET instructions = true
memory type range registers = true
global paging extension = true
machine check architecture = true
conditional move/compare instruction = true
page attribute table = true
page size extension = true
multiprocessing capable = false
no-execute page protection = true
AMD multimedia instruction extensions = true
MMX Technology = true
FXSAVE/FXRSTOR = true
SSE extensions = true
1-GB large page support = true
RDTSCP = true
long mode (AA-64) = true
3DNow! instruction extensions = false
3DNow! instructions = false
extended brand id (0x80000001/ebx):
raw = 0x20000000 (536870912)
BrandId = 0x0 (0)
PkgType = FM2 (PGA) (2)
AMD feature flags (0x80000001/ecx):
LAHF/SAHF supported in 64-bit mode = true
CMP Legacy = true
SVM: secure virtual machine = true
extended APIC space = true
AltMovCr8 = true
LZCNT advanced bit manipulation = true
SSE4A support = true
misaligned SSE mode = true
3DNow! PREFETCH/PREFETCHW instructions = true
OS visible workaround = true
instruction based sampling = true
XOP support = true
SKINIT/STGI support = true
watchdog timer support = true
lightweight profiling support = true
4-operand FMA instruction = true
TCE: translation cache extension = true
NodeId MSR C001100C = true
TBM support = true
topology extensions = true
core performance counter extensions = true
NB/DF performance counter extensions = true
data breakpoint extension = false
performance time-stamp counter support = false
LLC performance counter extensions = false
MWAITX/MONITORX supported = false
Address mask extension support = false
brand = "AMD A6-6400K APU with Radeon(tm) HD Graphics "
L1 TLB/cache information: 2M/4M pages & L1 TLB (0x80000005/eax):
instruction # entries = 0x18 (24)
instruction associativity = 0xff (255)
data # entries = 0x40 (64)
data associativity = 0xff (255)
L1 TLB/cache information: 4K pages & L1 TLB (0x80000005/ebx):
instruction # entries = 0x30 (48)
instruction associativity = 0xff (255)
data # entries = 0x40 (64)
data associativity = 0xff (255)
L1 data cache information (0x80000005/ecx):
line size (bytes) = 0x40 (64)
lines per tag = 0x1 (1)
associativity = 0x4 (4)
size (KB) = 0x10 (16)
L1 instruction cache information (0x80000005/edx):
line size (bytes) = 0x40 (64)
lines per tag = 0x1 (1)
associativity = 0x2 (2)
size (KB) = 0x40 (64)
L2 TLB/cache information: 2M/4M pages & L2 TLB (0x80000006/eax):
instruction # entries = 0x400 (1024)
instruction associativity = 8 to 15-way (6)
data # entries = 0x400 (1024)
data associativity = 8 to 15-way (6)
L2 TLB/cache information: 4K pages & L2 TLB (0x80000006/ebx):
instruction # entries = 0x200 (512)
instruction associativity = 4 to 5-way (4)
data # entries = 0x400 (1024)
data associativity = 8 to 15-way (6)
L2 unified cache information (0x80000006/ecx):
line size (bytes) = 0x40 (64)
lines per tag = 0x1 (1)
associativity = 16 to 31-way (8)
size (KB) = 0x400 (1024)
L3 cache information (0x80000006/edx):
line size (bytes) = 0x0 (0)
lines per tag = 0x0 (0)
associativity = L2 off (0)
size (in 512KB units) = 0x0 (0)
RAS Capability (0x80000007/ebx):
MCA overflow recovery support = false
SUCCOR support = false
HWA: hardware assert support = false
scalable MCA support = false
Advanced Power Management Features (0x80000007/ecx):
CmpUnitPwrSampleTimeRatio = 0x0 (0)
Advanced Power Management Features (0x80000007/edx):
TS: temperature sensing diode = true
FID: frequency ID control = false
VID: voltage ID control = false
TTP: thermal trip = true
TM: thermal monitor = true
STC: software thermal control = false
100 MHz multiplier control = true
hardware P-State control = true
TscInvariant = true
CPB: core performance boost = true
read-only effective frequency interface = true
processor feedback interface = false
APM power reporting = false
connected standby = false
RAPL: running average power limit = false
Physical Address and Linear Address Size (0x80000008/eax):
maximum physical address bits = 0x30 (48)
maximum linear (virtual) address bits = 0x30 (48)
maximum guest physical address bits = 0x0 (0)
Extended Feature Extensions ID (0x80000008/ebx):
CLZERO instruction = false
instructions retired count support = false
always save/restore error pointers = false
INVLPGB instruction = false
RDPRU instruction = false
memory bandwidth enforcement = false
MCOMMIT instruction = false
WBNOINVD instruction = false
IBPB: indirect branch prediction barrier = true
interruptible WBINVD, WBNOINVD = false
IBRS: indirect branch restr speculation = false
STIBP: 1 thr indirect branch predictor = false
CPU prefers: IBRS always on = false
CPU prefers: STIBP always on = false
IBRS preferred over software solution = false
IBRS provides same mode protection = false
EFER[LMSLE] not supported = false
INVLPGB supports TLB flush guest nested = false
ppin processor id number supported = false
SSBD: speculative store bypass disable = false
virtualized SSBD = false
SSBD fixed in hardware = false
CPPC: collaborative processor perf ctrl = false
PSFD: predictive store forward disable = false
not vulnerable to branch type confusion = false
branch sampling feature support = false
(vuln to branch type confusion synth) = true
Size Identifiers (0x80000008/ecx):
number of CPU cores = 0x2 (2)
ApicIdCoreIdSize = 0x4 (4)
performance time-stamp counter size = 40 bits (0)
Feature Extended Size (0x80000008/edx):
max page count for INVLPGB instruction = 0x0 (0)
RDPRU instruction max input support = 0x0 (0)
SVM Secure Virtual Machine (0x8000000a/eax):
SvmRev: SVM revision = 0x1 (1)
SVM Secure Virtual Machine (0x8000000a/edx):
nested paging = true
LBR virtualization = true
SVM lock = true
NRIP save = true
MSR based TSC rate control = true
VMCB clean bits support = true
flush by ASID = true
decode assists = true
SSSE3/SSE5 opcode set disable = false
pause intercept filter = true
pause filter threshold = true
AVIC: AMD virtual interrupt controller = false
virtualized VMLOAD/VMSAVE = false
virtualized global interrupt flag (GIF) = false
GMET: guest mode execute trap = false
X2AVIC: virtualized X2APIC = false
supervisor shadow stack = false
guest Spec_ctl support = false
ROGPT: read-only guest page table = false
host MCE override = false
INVLPGB/TLBSYNC hyperv interc enable = false
VNMI: NMI virtualization = false
IBS virtualization = false
guest SVME addr check = false
NASID: number of address space identifiers = 0x10000 (65536):
L1 TLB information: 1G pages (0x80000019/eax):
instruction # entries = 0x18 (24)
instruction associativity = full (15)
data # entries = 0x40 (64)
data associativity = full (15)
L2 TLB information: 1G pages (0x80000019/ebx):
instruction # entries = 0x400 (1024)
instruction associativity = 8 to 15-way (6)
data # entries = 0x400 (1024)
data associativity = 8 to 15-way (6)
Performance Optimization Identifiers (0x8000001a/eax):
128-bit SSE executed full-width = true
MOVU* better than MOVL*/MOVH* = true
256-bit SSE executed full-width = false
Instruction Based Sampling Identifiers (0x8000001b/eax):
IBS feature flags valid = true
IBS fetch sampling = true
IBS execution sampling = true
read write of op counter = true
op counting mode = true
branch target address reporting = true
IbsOpCurCnt and IbsOpMaxCnt extend 7 = true
invalid RIP indication support = true
fused branch micro-op indication support = false
IBS fetch control extended MSR support = false
IBS op data 4 MSR support = false
IBS L3 miss filtering support = false
Lightweight Profiling Capabilities: Availability (0x8000001c/eax):
lightweight profiling = false
LWPVAL instruction = false
instruction retired event = false
branch retired event = false
DC miss event = false
core clocks not halted event = false
core reference clocks not halted event = false
continuous mode sampling = false
tsc in event record = false
interrupt on threshold overflow = false
Lightweight Profiling Capabilities: Supported (0x8000001c/edx):
lightweight profiling = true
LWPVAL instruction = true
instruction retired event = true
branch retired event = true
DC miss event = false
core clocks not halted event = false
core reference clocks not halted event = false
continuous mode sampling = false
tsc in event record = false
interrupt on threshold overflow = true
Lightweight Profiling Capabilities (0x8000001c/ebx):
LWPCB byte size = 0x13 (19)
event record byte size = 0x20 (32)
maximum EventId = 0x3 (3)
EventInterval1 field offset = 0x80 (128)
Lightweight Profiling Capabilities (0x8000001c/ecx):
latency counter bit size = 0x0 (0)
data cache miss address valid = false
amount cache latency is rounded = 0x0 (0)
LWP implementation version = 0x1 (1)
event ring buffer size in records = 0x1 (1)
branch prediction filtering = false
IP filtering = false
cache level filtering = false
cache latency filteing = false
Cache Properties (0x8000001d):
--- cache 0 ---
type = data (1)
level = 0x1 (1)
self-initializing = true
fully associative = false
extra cores sharing this cache = 0x0 (0)
line size in bytes = 0x40 (64)
physical line partitions = 0x1 (1)
number of ways = 0x4 (4)
number of sets = 64
write-back invalidate = false
cache inclusive of lower levels = false
(synth size) = 16384 (16 KB)
--- cache 1 ---
type = instruction (2)
level = 0x1 (1)
self-initializing = true
fully associative = false
extra cores sharing this cache = 0x1 (1)
line size in bytes = 0x40 (64)
physical line partitions = 0x1 (1)
number of ways = 0x2 (2)
number of sets = 512
write-back invalidate = false
cache inclusive of lower levels = false
(synth size) = 65536 (64 KB)
--- cache 2 ---
type = unified (3)
level = 0x2 (2)
self-initializing = true
fully associative = false
extra cores sharing this cache = 0x1 (1)
line size in bytes = 0x40 (64)
physical line partitions = 0x1 (1)
number of ways = 0x10 (16)
number of sets = 1024
write-back invalidate = true
cache inclusive of lower levels = false
(synth size) = 1048576 (1024 KB)
extended APIC ID = 16
Compute Unit Identifiers (0x8000001e/ebx):
compute unit ID = 0x0 (0)
cores per compute unit = 0x2 (2)
Node Identifiers (0x8000001e/ecx):
node ID = 0x0 (0)
nodes per processor = 0x1 (1)
(instruction supported synth):
CMPXCHG8B = true
conditional move/compare = true
PREFETCH/PREFETCHW = true
(multi-processing synth) = multi-core (c=2)
(multi-processing method) = AMD
(APIC widths synth): CORE_width=1 SMT_width=0
(APIC synth): PKG_ID=0 CORE_ID=0 SMT_ID=0
(uarch synth) = AMD Piledriver, 32nm
(synth) = AMD A-Series (Richland RL-A1) [Piledriver], 32nm
CPU 1:
vendor_id = "AuthenticAMD"
version information (1/eax):
processor type = primary processor (0)
family = 0xf (15)
model = 0x3 (3)
stepping id = 0x1 (1)
extended family = 0x6 (6)
extended model = 0x1 (1)
(family synth) = 0x15 (21)
(model synth) = 0x13 (19)
(simple synth) = AMD (unknown type) (Richland RL-A1) [Piledriver], 32nm
miscellaneous (1/ebx):
process local APIC physical ID = 0x1 (1)
maximum IDs for CPUs in pkg = 0x2 (2)
CLFLUSH line size = 0x8 (8)
brand index = 0x0 (0)
brand id = 0x00 (0): unknown
feature information (1/edx):
x87 FPU on chip = true
VME: virtual-8086 mode enhancement = true
DE: debugging extensions = true
PSE: page size extensions = true
TSC: time stamp counter = true
RDMSR and WRMSR support = true
PAE: physical address extensions = true
MCE: machine check exception = true
CMPXCHG8B inst. = true
APIC on chip = true
SYSENTER and SYSEXIT = true
MTRR: memory type range registers = true
PTE global bit = true
MCA: machine check architecture = true
CMOV: conditional move/compare instr = true
PAT: page attribute table = true
PSE-36: page size extension = true
PSN: processor serial number = false
CLFLUSH instruction = true
DS: debug store = false
ACPI: thermal monitor and clock ctrl = false
MMX Technology = true
FXSAVE/FXRSTOR = true
SSE extensions = true
SSE2 extensions = true
SS: self snoop = false
hyper-threading / multi-core supported = true
TM: therm. monitor = false
IA64 = false
PBE: pending break event = false
feature information (1/ecx):
PNI/SSE3: Prescott New Instructions = true
PCLMULDQ instruction = true
DTES64: 64-bit debug store = false
MONITOR/MWAIT = true
CPL-qualified debug store = false
VMX: virtual machine extensions = false
SMX: safer mode extensions = false
Enhanced Intel SpeedStep Technology = false
TM2: thermal monitor 2 = false
SSSE3 extensions = true
context ID: adaptive or shared L1 data = false
SDBG: IA32_DEBUG_INTERFACE = false
FMA instruction = true
CMPXCHG16B instruction = true
xTPR disable = false
PDCM: perfmon and debug = false
PCID: process context identifiers = false
DCA: direct cache access = false
SSE4.1 extensions = true
SSE4.2 extensions = true
x2APIC: extended xAPIC support = false
MOVBE instruction = false
POPCNT instruction = true
time stamp counter deadline = false
AES instruction = true
XSAVE/XSTOR states = true
OS-enabled XSAVE/XSTOR = true
AVX: advanced vector extensions = true
F16C half-precision convert instruction = true
RDRAND instruction = false
hypervisor guest status = false
cache and TLB information (2):
processor serial number = 0061-0F31-0000-0000-0000-0000
deterministic cache parameters (4):
--- cache 0 ---
cache type = no more caches (0)
MONITOR/MWAIT (5):
smallest monitor-line size (bytes) = 0x40 (64)
largest monitor-line size (bytes) = 0x40 (64)
enum of Monitor-MWAIT exts supported = true
supports intrs as break-event for MWAIT = true
number of C0 sub C-states using MWAIT = 0x0 (0)
number of C1 sub C-states using MWAIT = 0x0 (0)
number of C2 sub C-states using MWAIT = 0x0 (0)
number of C3 sub C-states using MWAIT = 0x0 (0)
number of C4 sub C-states using MWAIT = 0x0 (0)
number of C5 sub C-states using MWAIT = 0x0 (0)
number of C6 sub C-states using MWAIT = 0x0 (0)
number of C7 sub C-states using MWAIT = 0x0 (0)
Thermal and Power Management Features (6):
digital thermometer = false
Intel Turbo Boost Technology = false
ARAT always running APIC timer = false
PLN power limit notification = false
ECMD extended clock modulation duty = false
PTM package thermal management = false
HWP base registers = false
HWP notification = false
HWP activity window = false
HWP energy performance preference = false
HWP package level request = false
HDC base registers = false
Intel Turbo Boost Max Technology 3.0 = false
HWP capabilities = false
HWP PECI override = false
flexible HWP = false
IA32_HWP_REQUEST MSR fast access mode = false
HW_FEEDBACK MSRs supported = false
ignoring idle logical processor HWP req = false
Thread Director = false
IA32_HW_FEEDBACK_THREAD_CONFIG bit 25 = false
digital thermometer thresholds = 0x0 (0)
hardware coordination feedback = true
ACNT2 available = false
performance-energy bias capability = false
number of enh hardware feedback classes = 0x0 (0)
performance capability reporting = false
energy efficiency capability reporting = false
size of feedback struct (4KB pages) = 0x1 (1)
index of CPU's row in feedback struct = 0x0 (0)
extended feature flags (7):
FSGSBASE instructions = false
IA32_TSC_ADJUST MSR supported = false
SGX: Software Guard Extensions supported = false
BMI1 instructions = true
HLE hardware lock elision = false
AVX2: advanced vector extensions 2 = false
FDP_EXCPTN_ONLY = false
SMEP supervisor mode exec protection = false
BMI2 instructions = false
enhanced REP MOVSB/STOSB = false
INVPCID instruction = false
RTM: restricted transactional memory = false
RDT-CMT/PQoS cache monitoring = false
deprecated FPU CS/DS = false
MPX: intel memory protection extensions = false
RDT-CAT/PQE cache allocation = false
AVX512F: AVX-512 foundation instructions = false
AVX512DQ: double & quadword instructions = false
RDSEED instruction = false
ADX instructions = false
SMAP: supervisor mode access prevention = false
AVX512IFMA: integer fused multiply add = false
PCOMMIT instruction = false
CLFLUSHOPT instruction = false
CLWB instruction = false
Intel processor trace = false
AVX512PF: prefetch instructions = false
AVX512ER: exponent & reciprocal instrs = false
AVX512CD: conflict detection instrs = false
SHA instructions = false
AVX512BW: byte & word instructions = false
AVX512VL: vector length = false
PREFETCHWT1 = false
AVX512VBMI: vector byte manipulation = false
UMIP: user-mode instruction prevention = false
PKU protection keys for user-mode = false
OSPKE CR4.PKE and RDPKRU/WRPKRU = false
WAITPKG instructions = false
AVX512_VBMI2: byte VPCOMPRESS, VPEXPAND = false
CET_SS: CET shadow stack = false
GFNI: Galois Field New Instructions = false
VAES instructions = false
VPCLMULQDQ instruction = false
AVX512_VNNI: neural network instructions = false
AVX512_BITALG: bit count/shiffle = false
TME: Total Memory Encryption = false
AVX512: VPOPCNTDQ instruction = false
LA57: 57-bit addrs & 5-level paging = false
BNDLDX/BNDSTX MAWAU value in 64-bit mode = 0x0 (0)
RDPID: read processor ID supported = false
KL: key locker = false
bus lock detection = false
CLDEMOTE supports cache line demote = false
MOVDIRI instruction = false
MOVDIR64B instruction = false
ENQCMD instruction = false
SGX_LC: SGX launch config supported = false
PKS: supervisor protection keys = false
SGX-KEYS: SGX attestation services = false
AVX512_4VNNIW: neural network instrs = false
AVX512_4FMAPS: multiply acc single prec = false
fast short REP MOV = false
UINTR: user interrupts = false
AVX512_VP2INTERSECT: intersect mask regs = false
IA32_MCU_OPT_CTRL SRBDS mitigation MSR = false
VERW MD_CLEAR microcode support = false
RTM transaction always aborts = false
IA32_TSX_FORCE_ABORT MSR = false
SERIALIZE instruction = false
hybrid part = false
TSXLDTRK: TSX suspend load addr tracking = false
PCONFIG instruction = false
LBR: architectural last branch records = false
CET_IBT: CET indirect branch tracking = false
AMX-BF16: tile bfloat16 support = false
AVX512_FP16: fp16 support = false
AMX-TILE: tile architecture support = false
AMX-INT8: tile 8-bit integer support = false
IBRS/IBPB: indirect branch restrictions = false
STIBP: 1 thr indirect branch predictor = false
L1D_FLUSH: IA32_FLUSH_CMD MSR = false
IA32_ARCH_CAPABILITIES MSR = false
IA32_CORE_CAPABILITIES MSR = false
SSBD: speculative store bypass disable = false
Direct Cache Access Parameters (9):
PLATFORM_DCA_CAP MSR bits = 0
Architecture Performance Monitoring Features (0xa):
version ID = 0x0 (0)
number of counters per logical processor = 0x0 (0)
bit width of counter = 0x0 (0)
length of EBX bit vector = 0x0 (0)
core cycle event = not available
instruction retired event = not available
reference cycles event = not available
last-level cache ref event = not available
last-level cache miss event = not available
branch inst retired event = not available
branch mispred retired event = not available
top-down slots event = not available
fixed counter 0 supported = false
fixed counter 1 supported = false
fixed counter 2 supported = false
fixed counter 3 supported = false
fixed counter 4 supported = false
fixed counter 5 supported = false
fixed counter 6 supported = false
fixed counter 7 supported = false
fixed counter 8 supported = false
fixed counter 9 supported = false
fixed counter 10 supported = false
fixed counter 11 supported = false
fixed counter 12 supported = false
fixed counter 13 supported = false
fixed counter 14 supported = false
fixed counter 15 supported = false
fixed counter 16 supported = false
fixed counter 17 supported = false
fixed counter 18 supported = false
fixed counter 19 supported = false
fixed counter 20 supported = false
fixed counter 21 supported = false
fixed counter 22 supported = false
fixed counter 23 supported = false
fixed counter 24 supported = false
fixed counter 25 supported = false
fixed counter 26 supported = false
fixed counter 27 supported = false
fixed counter 28 supported = false
fixed counter 29 supported = false
fixed counter 30 supported = false
fixed counter 31 supported = false
number of contiguous fixed counters = 0x0 (0)
bit width of fixed counters = 0x0 (0)
anythread deprecation = false
x2APIC features / processor topology (0xb):
extended APIC ID = 0
--- level 0 ---
level number = 0x0 (0)
level type = invalid (0)
bit width of level = 0x0 (0)
number of logical processors at level = 0x0 (0)
XSAVE features (0xd/0):
XCR0 valid bit field mask = 0x4000000000000007
x87 state = true
SSE state = true
AVX state = true
MPX BNDREGS = false
MPX BNDCSR = false
AVX-512 opmask = false
AVX-512 ZMM_Hi256 = false
AVX-512 Hi16_ZMM = false
PKRU state = false
XTILECFG state = false
XTILEDATA state = false
bytes required by fields in XCR0 = 0x00000340 (832)
bytes required by XSAVE/XRSTOR area = 0x000003c0 (960)
XSAVEOPT instruction = false
XSAVEC instruction = false
XGETBV instruction = false
XSAVES/XRSTORS instructions = false
XFD: extended feature disable supported = false
SAVE area size in bytes = 0x00000000 (0)
IA32_XSS valid bit field mask = 0x0000000000000000
PT state = false
PASID state = false
CET_U user state = false
CET_S supervisor state = false
HDC state = false
UINTR state = false
LBR state = false
HWP state = false
AVX/YMM features (0xd/2):
AVX/YMM save state byte size = 0x00000100 (256)
AVX/YMM save state byte offset = 0x00000240 (576)
supported in IA32_XSS or XCR0 = XCR0 (user state)
64-byte alignment in compacted XSAVE = false
XFD faulting supported = false
LWP features (0xd/0x3e):
LWP save state byte size = 0x00000080 (128)
LWP save state byte offset = 0x00000340 (832)
supported in IA32_XSS or XCR0 = XCR0 (user state)
64-byte alignment in compacted XSAVE = false
XFD faulting supported = false
extended processor signature (0x80000001/eax):
family/generation = 0xf (15)
model = 0x3 (3)
stepping id = 0x1 (1)
extended family = 0x6 (6)
extended model = 0x1 (1)
(family synth) = 0x15 (21)
(model synth) = 0x13 (19)
(simple synth) = AMD (unknown type) (Richland RL-A1) [Piledriver], 32nm
extended feature flags (0x80000001/edx):
x87 FPU on chip = true
virtual-8086 mode enhancement = true
debugging extensions = true
page size extensions = true
time stamp counter = true
RDMSR and WRMSR support = true
physical address extensions = true
machine check exception = true
CMPXCHG8B inst. = true
APIC on chip = true
SYSCALL and SYSRET instructions = true
memory type range registers = true
global paging extension = true
machine check architecture = true
conditional move/compare instruction = true
page attribute table = true
page size extension = true
multiprocessing capable = false
no-execute page protection = true
AMD multimedia instruction extensions = true
MMX Technology = true
FXSAVE/FXRSTOR = true
SSE extensions = true
1-GB large page support = true
RDTSCP = true
long mode (AA-64) = true
3DNow! instruction extensions = false
3DNow! instructions = false
extended brand id (0x80000001/ebx):
raw = 0x20000000 (536870912)
BrandId = 0x0 (0)
PkgType = FM2 (PGA) (2)
AMD feature flags (0x80000001/ecx):
LAHF/SAHF supported in 64-bit mode = true
CMP Legacy = true
SVM: secure virtual machine = true
extended APIC space = true
AltMovCr8 = true
LZCNT advanced bit manipulation = true
SSE4A support = true
misaligned SSE mode = true
3DNow! PREFETCH/PREFETCHW instructions = true
OS visible workaround = true
instruction based sampling = true
XOP support = true
SKINIT/STGI support = true
watchdog timer support = true
lightweight profiling support = true
4-operand FMA instruction = true
TCE: translation cache extension = true
NodeId MSR C001100C = true
TBM support = true
topology extensions = true
core performance counter extensions = true
NB/DF performance counter extensions = true
data breakpoint extension = false
performance time-stamp counter support = false
LLC performance counter extensions = false
MWAITX/MONITORX supported = false
Address mask extension support = false
brand = "AMD A6-6400K APU with Radeon(tm) HD Graphics "
L1 TLB/cache information: 2M/4M pages & L1 TLB (0x80000005/eax):
instruction # entries = 0x18 (24)
instruction associativity = 0xff (255)
data # entries = 0x40 (64)
data associativity = 0xff (255)
L1 TLB/cache information: 4K pages & L1 TLB (0x80000005/ebx):
instruction # entries = 0x30 (48)
instruction associativity = 0xff (255)
data # entries = 0x40 (64)
data associativity = 0xff (255)
L1 data cache information (0x80000005/ecx):
line size (bytes) = 0x40 (64)
lines per tag = 0x1 (1)
associativity = 0x4 (4)
size (KB) = 0x10 (16)
L1 instruction cache information (0x80000005/edx):
line size (bytes) = 0x40 (64)
lines per tag = 0x1 (1)
associativity = 0x2 (2)
size (KB) = 0x40 (64)
L2 TLB/cache information: 2M/4M pages & L2 TLB (0x80000006/eax):
instruction # entries = 0x400 (1024)
instruction associativity = 8 to 15-way (6)
data # entries = 0x400 (1024)
data associativity = 8 to 15-way (6)
L2 TLB/cache information: 4K pages & L2 TLB (0x80000006/ebx):
instruction # entries = 0x200 (512)
instruction associativity = 4 to 5-way (4)
data # entries = 0x400 (1024)
data associativity = 8 to 15-way (6)
L2 unified cache information (0x80000006/ecx):
line size (bytes) = 0x40 (64)
lines per tag = 0x1 (1)
associativity = 16 to 31-way (8)
size (KB) = 0x400 (1024)
L3 cache information (0x80000006/edx):
line size (bytes) = 0x0 (0)
lines per tag = 0x0 (0)
associativity = L2 off (0)
size (in 512KB units) = 0x0 (0)
RAS Capability (0x80000007/ebx):
MCA overflow recovery support = false
SUCCOR support = false
HWA: hardware assert support = false
scalable MCA support = false
Advanced Power Management Features (0x80000007/ecx):
CmpUnitPwrSampleTimeRatio = 0x0 (0)
Advanced Power Management Features (0x80000007/edx):
TS: temperature sensing diode = true
FID: frequency ID control = false
VID: voltage ID control = false
TTP: thermal trip = true
TM: thermal monitor = true
STC: software thermal control = false
100 MHz multiplier control = true
hardware P-State control = true
TscInvariant = true
CPB: core performance boost = true
read-only effective frequency interface = true
processor feedback interface = false
APM power reporting = false
connected standby = false
RAPL: running average power limit = false
Physical Address and Linear Address Size (0x80000008/eax):
maximum physical address bits = 0x30 (48)
maximum linear (virtual) address bits = 0x30 (48)
maximum guest physical address bits = 0x0 (0)
Extended Feature Extensions ID (0x80000008/ebx):
CLZERO instruction = false
instructions retired count support = false
always save/restore error pointers = false
INVLPGB instruction = false
RDPRU instruction = false
memory bandwidth enforcement = false
MCOMMIT instruction = false
WBNOINVD instruction = false
IBPB: indirect branch prediction barrier = true
interruptible WBINVD, WBNOINVD = false
IBRS: indirect branch restr speculation = false
STIBP: 1 thr indirect branch predictor = false
CPU prefers: IBRS always on = false
CPU prefers: STIBP always on = false
IBRS preferred over software solution = false
IBRS provides same mode protection = false
EFER[LMSLE] not supported = false
INVLPGB supports TLB flush guest nested = false
ppin processor id number supported = false
SSBD: speculative store bypass disable = false
virtualized SSBD = false
SSBD fixed in hardware = false
CPPC: collaborative processor perf ctrl = false
PSFD: predictive store forward disable = false
not vulnerable to branch type confusion = false
branch sampling feature support = false
(vuln to branch type confusion synth) = true
Size Identifiers (0x80000008/ecx):
number of CPU cores = 0x2 (2)
ApicIdCoreIdSize = 0x4 (4)
performance time-stamp counter size = 40 bits (0)
Feature Extended Size (0x80000008/edx):
max page count for INVLPGB instruction = 0x0 (0)
RDPRU instruction max input support = 0x0 (0)
SVM Secure Virtual Machine (0x8000000a/eax):
SvmRev: SVM revision = 0x1 (1)
SVM Secure Virtual Machine (0x8000000a/edx):
nested paging = true
LBR virtualization = true
SVM lock = true
NRIP save = true
MSR based TSC rate control = true
VMCB clean bits support = true
flush by ASID = true
decode assists = true
SSSE3/SSE5 opcode set disable = false
pause intercept filter = true
pause filter threshold = true
AVIC: AMD virtual interrupt controller = false
virtualized VMLOAD/VMSAVE = false
virtualized global interrupt flag (GIF) = false
GMET: guest mode execute trap = false
X2AVIC: virtualized X2APIC = false
supervisor shadow stack = false
guest Spec_ctl support = false
ROGPT: read-only guest page table = false
host MCE override = false
INVLPGB/TLBSYNC hyperv interc enable = false
VNMI: NMI virtualization = false
IBS virtualization = false
guest SVME addr check = false
NASID: number of address space identifiers = 0x10000 (65536):
L1 TLB information: 1G pages (0x80000019/eax):
instruction # entries = 0x18 (24)
instruction associativity = full (15)
data # entries = 0x40 (64)
data associativity = full (15)
L2 TLB information: 1G pages (0x80000019/ebx):
instruction # entries = 0x400 (1024)
instruction associativity = 8 to 15-way (6)
data # entries = 0x400 (1024)
data associativity = 8 to 15-way (6)
Performance Optimization Identifiers (0x8000001a/eax):
128-bit SSE executed full-width = true
MOVU* better than MOVL*/MOVH* = true
256-bit SSE executed full-width = false
Instruction Based Sampling Identifiers (0x8000001b/eax):
IBS feature flags valid = true
IBS fetch sampling = true
IBS execution sampling = true
read write of op counter = true
op counting mode = true
branch target address reporting = true
IbsOpCurCnt and IbsOpMaxCnt extend 7 = true
invalid RIP indication support = true
fused branch micro-op indication support = false
IBS fetch control extended MSR support = false
IBS op data 4 MSR support = false
IBS L3 miss filtering support = false
Lightweight Profiling Capabilities: Availability (0x8000001c/eax):
lightweight profiling = false
LWPVAL instruction = false
instruction retired event = false
branch retired event = false
DC miss event = false
core clocks not halted event = false
core reference clocks not halted event = false
continuous mode sampling = false
tsc in event record = false
interrupt on threshold overflow = false
Lightweight Profiling Capabilities: Supported (0x8000001c/edx):
lightweight profiling = true
LWPVAL instruction = true
instruction retired event = true
branch retired event = true
DC miss event = false
core clocks not halted event = false
core reference clocks not halted event = false
continuous mode sampling = false
tsc in event record = false
interrupt on threshold overflow = true
Lightweight Profiling Capabilities (0x8000001c/ebx):
LWPCB byte size = 0x13 (19)
event record byte size = 0x20 (32)
maximum EventId = 0x3 (3)
EventInterval1 field offset = 0x80 (128)
Lightweight Profiling Capabilities (0x8000001c/ecx):
latency counter bit size = 0x0 (0)
data cache miss address valid = false
amount cache latency is rounded = 0x0 (0)
LWP implementation version = 0x1 (1)
event ring buffer size in records = 0x1 (1)
branch prediction filtering = false
IP filtering = false
cache level filtering = false
cache latency filteing = false
Cache Properties (0x8000001d):
--- cache 0 ---
type = data (1)
level = 0x1 (1)
self-initializing = true
fully associative = false
extra cores sharing this cache = 0x0 (0)
line size in bytes = 0x40 (64)
physical line partitions = 0x1 (1)
number of ways = 0x4 (4)
number of sets = 64
write-back invalidate = false
cache inclusive of lower levels = false
(synth size) = 16384 (16 KB)
--- cache 1 ---
type = instruction (2)
level = 0x1 (1)
self-initializing = true
fully associative = false
extra cores sharing this cache = 0x1 (1)
line size in bytes = 0x40 (64)
physical line partitions = 0x1 (1)
number of ways = 0x2 (2)
number of sets = 512
write-back invalidate = false
cache inclusive of lower levels = false
(synth size) = 65536 (64 KB)
--- cache 2 ---
type = unified (3)
level = 0x2 (2)
self-initializing = true
fully associative = false
extra cores sharing this cache = 0x1 (1)
line size in bytes = 0x40 (64)
physical line partitions = 0x1 (1)
number of ways = 0x10 (16)
number of sets = 1024
write-back invalidate = true
cache inclusive of lower levels = false
(synth size) = 1048576 (1024 KB)
extended APIC ID = 17
Compute Unit Identifiers (0x8000001e/ebx):
compute unit ID = 0x0 (0)
cores per compute unit = 0x2 (2)
Node Identifiers (0x8000001e/ecx):
node ID = 0x0 (0)
nodes per processor = 0x1 (1)
(instruction supported synth):
CMPXCHG8B = true
conditional move/compare = true
PREFETCH/PREFETCHW = true
(multi-processing synth) = multi-core (c=2)
(multi-processing method) = AMD
(APIC widths synth): CORE_width=1 SMT_width=0
(APIC synth): PKG_ID=0 CORE_ID=1 SMT_ID=0
(uarch synth) = AMD Piledriver, 32nm
(synth) = AMD A-Series (Richland RL-A1) [Piledriver], 32nm
[-- Attachment #3: Type: text/plain, Size: 161 bytes --]
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-18 20:10 ` Paul Menzel
0 siblings, 0 replies; 236+ messages in thread
From: Paul Menzel @ 2023-04-18 20:10 UTC (permalink / raw)
To: Thomas Gleixner
Cc: linux-kernel, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
[-- Attachment #1: Type: text/plain, Size: 1924 bytes --]
Dear Thomas,
Am 18.04.23 um 10:40 schrieb Thomas Gleixner:
> On Tue, Apr 18 2023 at 08:58, Thomas Gleixner wrote:
>> On Mon, Apr 17 2023 at 19:40, Paul Menzel wrote:
>>> Am 17.04.23 um 16:48 schrieb Thomas Gleixner:
>>>
>>>> On Mon, Apr 17 2023 at 13:19, Paul Menzel wrote:
>>>>> Am 15.04.23 um 01:44 schrieb Thomas Gleixner:
>>>>> [ 0.258193] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD Graphics (family: 0x15, model: 0x13, stepping: 0x1)
>>>>> […]
>>>>> [ 0.259329] smp: Bringing up secondary CPUs ...
>>>>> [ 0.259527] x86: Booting SMP configuration:
>>>>> [ 0.259528] .... node #0, CPUs: #1
>>>>> [ 0.261007] After schedule_preempt_disabled
>>>>> [ 10.260990] CPU1 failed to report alive state
>>>>
>>>> Weird. CPU1 fails to come up and report that it has reached the
>>>> synchronization point.
>>>>
>>>> Does it work when you add cpuhp.parallel=off on the kernel command line?
>>>
>>> Yes, the ten seconds delay is gone with `cpuhp.parallel=off`.
>>>
>>> There was a patch set in the past, that worked on that device. I think
>>> up to v4 it did *not* work at all and hung [1]. I need some days to
>>> collect the results again.
>>
>> Can you please apply the patch below on top of the pile remove the
>> command line option again?
>
> Bah. That patch does not make any sense at all. Not enough coffee.
>
> Can you please provide the output of cpuid?
Of course. Here the top, and the whole output is attached.
```
CPU 0:
vendor_id = "AuthenticAMD"
version information (1/eax):
processor type = primary processor (0)
family = 0xf (15)
model = 0x3 (3)
stepping id = 0x1 (1)
extended family = 0x6 (6)
extended model = 0x1 (1)
(family synth) = 0x15 (21)
(model synth) = 0x13 (19)
(simple synth) = AMD (unknown type) (Richland RL-A1)
[Piledriver], 32nm
[…]
```
Kind regards,
Paul
[-- Attachment #2: cpuid.txt --]
[-- Type: text/plain, Size: 65008 bytes --]
CPU 0:
vendor_id = "AuthenticAMD"
version information (1/eax):
processor type = primary processor (0)
family = 0xf (15)
model = 0x3 (3)
stepping id = 0x1 (1)
extended family = 0x6 (6)
extended model = 0x1 (1)
(family synth) = 0x15 (21)
(model synth) = 0x13 (19)
(simple synth) = AMD (unknown type) (Richland RL-A1) [Piledriver], 32nm
miscellaneous (1/ebx):
process local APIC physical ID = 0x0 (0)
maximum IDs for CPUs in pkg = 0x2 (2)
CLFLUSH line size = 0x8 (8)
brand index = 0x0 (0)
brand id = 0x00 (0): unknown
feature information (1/edx):
x87 FPU on chip = true
VME: virtual-8086 mode enhancement = true
DE: debugging extensions = true
PSE: page size extensions = true
TSC: time stamp counter = true
RDMSR and WRMSR support = true
PAE: physical address extensions = true
MCE: machine check exception = true
CMPXCHG8B inst. = true
APIC on chip = true
SYSENTER and SYSEXIT = true
MTRR: memory type range registers = true
PTE global bit = true
MCA: machine check architecture = true
CMOV: conditional move/compare instr = true
PAT: page attribute table = true
PSE-36: page size extension = true
PSN: processor serial number = false
CLFLUSH instruction = true
DS: debug store = false
ACPI: thermal monitor and clock ctrl = false
MMX Technology = true
FXSAVE/FXRSTOR = true
SSE extensions = true
SSE2 extensions = true
SS: self snoop = false
hyper-threading / multi-core supported = true
TM: therm. monitor = false
IA64 = false
PBE: pending break event = false
feature information (1/ecx):
PNI/SSE3: Prescott New Instructions = true
PCLMULDQ instruction = true
DTES64: 64-bit debug store = false
MONITOR/MWAIT = true
CPL-qualified debug store = false
VMX: virtual machine extensions = false
SMX: safer mode extensions = false
Enhanced Intel SpeedStep Technology = false
TM2: thermal monitor 2 = false
SSSE3 extensions = true
context ID: adaptive or shared L1 data = false
SDBG: IA32_DEBUG_INTERFACE = false
FMA instruction = true
CMPXCHG16B instruction = true
xTPR disable = false
PDCM: perfmon and debug = false
PCID: process context identifiers = false
DCA: direct cache access = false
SSE4.1 extensions = true
SSE4.2 extensions = true
x2APIC: extended xAPIC support = false
MOVBE instruction = false
POPCNT instruction = true
time stamp counter deadline = false
AES instruction = true
XSAVE/XSTOR states = true
OS-enabled XSAVE/XSTOR = true
AVX: advanced vector extensions = true
F16C half-precision convert instruction = true
RDRAND instruction = false
hypervisor guest status = false
cache and TLB information (2):
processor serial number = 0061-0F31-0000-0000-0000-0000
deterministic cache parameters (4):
--- cache 0 ---
cache type = no more caches (0)
MONITOR/MWAIT (5):
smallest monitor-line size (bytes) = 0x40 (64)
largest monitor-line size (bytes) = 0x40 (64)
enum of Monitor-MWAIT exts supported = true
supports intrs as break-event for MWAIT = true
number of C0 sub C-states using MWAIT = 0x0 (0)
number of C1 sub C-states using MWAIT = 0x0 (0)
number of C2 sub C-states using MWAIT = 0x0 (0)
number of C3 sub C-states using MWAIT = 0x0 (0)
number of C4 sub C-states using MWAIT = 0x0 (0)
number of C5 sub C-states using MWAIT = 0x0 (0)
number of C6 sub C-states using MWAIT = 0x0 (0)
number of C7 sub C-states using MWAIT = 0x0 (0)
Thermal and Power Management Features (6):
digital thermometer = false
Intel Turbo Boost Technology = false
ARAT always running APIC timer = false
PLN power limit notification = false
ECMD extended clock modulation duty = false
PTM package thermal management = false
HWP base registers = false
HWP notification = false
HWP activity window = false
HWP energy performance preference = false
HWP package level request = false
HDC base registers = false
Intel Turbo Boost Max Technology 3.0 = false
HWP capabilities = false
HWP PECI override = false
flexible HWP = false
IA32_HWP_REQUEST MSR fast access mode = false
HW_FEEDBACK MSRs supported = false
ignoring idle logical processor HWP req = false
Thread Director = false
IA32_HW_FEEDBACK_THREAD_CONFIG bit 25 = false
digital thermometer thresholds = 0x0 (0)
hardware coordination feedback = true
ACNT2 available = false
performance-energy bias capability = false
number of enh hardware feedback classes = 0x0 (0)
performance capability reporting = false
energy efficiency capability reporting = false
size of feedback struct (4KB pages) = 0x1 (1)
index of CPU's row in feedback struct = 0x0 (0)
extended feature flags (7):
FSGSBASE instructions = false
IA32_TSC_ADJUST MSR supported = false
SGX: Software Guard Extensions supported = false
BMI1 instructions = true
HLE hardware lock elision = false
AVX2: advanced vector extensions 2 = false
FDP_EXCPTN_ONLY = false
SMEP supervisor mode exec protection = false
BMI2 instructions = false
enhanced REP MOVSB/STOSB = false
INVPCID instruction = false
RTM: restricted transactional memory = false
RDT-CMT/PQoS cache monitoring = false
deprecated FPU CS/DS = false
MPX: intel memory protection extensions = false
RDT-CAT/PQE cache allocation = false
AVX512F: AVX-512 foundation instructions = false
AVX512DQ: double & quadword instructions = false
RDSEED instruction = false
ADX instructions = false
SMAP: supervisor mode access prevention = false
AVX512IFMA: integer fused multiply add = false
PCOMMIT instruction = false
CLFLUSHOPT instruction = false
CLWB instruction = false
Intel processor trace = false
AVX512PF: prefetch instructions = false
AVX512ER: exponent & reciprocal instrs = false
AVX512CD: conflict detection instrs = false
SHA instructions = false
AVX512BW: byte & word instructions = false
AVX512VL: vector length = false
PREFETCHWT1 = false
AVX512VBMI: vector byte manipulation = false
UMIP: user-mode instruction prevention = false
PKU protection keys for user-mode = false
OSPKE CR4.PKE and RDPKRU/WRPKRU = false
WAITPKG instructions = false
AVX512_VBMI2: byte VPCOMPRESS, VPEXPAND = false
CET_SS: CET shadow stack = false
GFNI: Galois Field New Instructions = false
VAES instructions = false
VPCLMULQDQ instruction = false
AVX512_VNNI: neural network instructions = false
AVX512_BITALG: bit count/shiffle = false
TME: Total Memory Encryption = false
AVX512: VPOPCNTDQ instruction = false
LA57: 57-bit addrs & 5-level paging = false
BNDLDX/BNDSTX MAWAU value in 64-bit mode = 0x0 (0)
RDPID: read processor ID supported = false
KL: key locker = false
bus lock detection = false
CLDEMOTE supports cache line demote = false
MOVDIRI instruction = false
MOVDIR64B instruction = false
ENQCMD instruction = false
SGX_LC: SGX launch config supported = false
PKS: supervisor protection keys = false
SGX-KEYS: SGX attestation services = false
AVX512_4VNNIW: neural network instrs = false
AVX512_4FMAPS: multiply acc single prec = false
fast short REP MOV = false
UINTR: user interrupts = false
AVX512_VP2INTERSECT: intersect mask regs = false
IA32_MCU_OPT_CTRL SRBDS mitigation MSR = false
VERW MD_CLEAR microcode support = false
RTM transaction always aborts = false
IA32_TSX_FORCE_ABORT MSR = false
SERIALIZE instruction = false
hybrid part = false
TSXLDTRK: TSX suspend load addr tracking = false
PCONFIG instruction = false
LBR: architectural last branch records = false
CET_IBT: CET indirect branch tracking = false
AMX-BF16: tile bfloat16 support = false
AVX512_FP16: fp16 support = false
AMX-TILE: tile architecture support = false
AMX-INT8: tile 8-bit integer support = false
IBRS/IBPB: indirect branch restrictions = false
STIBP: 1 thr indirect branch predictor = false
L1D_FLUSH: IA32_FLUSH_CMD MSR = false
IA32_ARCH_CAPABILITIES MSR = false
IA32_CORE_CAPABILITIES MSR = false
SSBD: speculative store bypass disable = false
Direct Cache Access Parameters (9):
PLATFORM_DCA_CAP MSR bits = 0
Architecture Performance Monitoring Features (0xa):
version ID = 0x0 (0)
number of counters per logical processor = 0x0 (0)
bit width of counter = 0x0 (0)
length of EBX bit vector = 0x0 (0)
core cycle event = not available
instruction retired event = not available
reference cycles event = not available
last-level cache ref event = not available
last-level cache miss event = not available
branch inst retired event = not available
branch mispred retired event = not available
top-down slots event = not available
fixed counter 0 supported = false
fixed counter 1 supported = false
fixed counter 2 supported = false
fixed counter 3 supported = false
fixed counter 4 supported = false
fixed counter 5 supported = false
fixed counter 6 supported = false
fixed counter 7 supported = false
fixed counter 8 supported = false
fixed counter 9 supported = false
fixed counter 10 supported = false
fixed counter 11 supported = false
fixed counter 12 supported = false
fixed counter 13 supported = false
fixed counter 14 supported = false
fixed counter 15 supported = false
fixed counter 16 supported = false
fixed counter 17 supported = false
fixed counter 18 supported = false
fixed counter 19 supported = false
fixed counter 20 supported = false
fixed counter 21 supported = false
fixed counter 22 supported = false
fixed counter 23 supported = false
fixed counter 24 supported = false
fixed counter 25 supported = false
fixed counter 26 supported = false
fixed counter 27 supported = false
fixed counter 28 supported = false
fixed counter 29 supported = false
fixed counter 30 supported = false
fixed counter 31 supported = false
number of contiguous fixed counters = 0x0 (0)
bit width of fixed counters = 0x0 (0)
anythread deprecation = false
x2APIC features / processor topology (0xb):
extended APIC ID = 0
--- level 0 ---
level number = 0x0 (0)
level type = invalid (0)
bit width of level = 0x0 (0)
number of logical processors at level = 0x0 (0)
XSAVE features (0xd/0):
XCR0 valid bit field mask = 0x4000000000000007
x87 state = true
SSE state = true
AVX state = true
MPX BNDREGS = false
MPX BNDCSR = false
AVX-512 opmask = false
AVX-512 ZMM_Hi256 = false
AVX-512 Hi16_ZMM = false
PKRU state = false
XTILECFG state = false
XTILEDATA state = false
bytes required by fields in XCR0 = 0x00000340 (832)
bytes required by XSAVE/XRSTOR area = 0x000003c0 (960)
XSAVEOPT instruction = false
XSAVEC instruction = false
XGETBV instruction = false
XSAVES/XRSTORS instructions = false
XFD: extended feature disable supported = false
SAVE area size in bytes = 0x00000000 (0)
IA32_XSS valid bit field mask = 0x0000000000000000
PT state = false
PASID state = false
CET_U user state = false
CET_S supervisor state = false
HDC state = false
UINTR state = false
LBR state = false
HWP state = false
AVX/YMM features (0xd/2):
AVX/YMM save state byte size = 0x00000100 (256)
AVX/YMM save state byte offset = 0x00000240 (576)
supported in IA32_XSS or XCR0 = XCR0 (user state)
64-byte alignment in compacted XSAVE = false
XFD faulting supported = false
LWP features (0xd/0x3e):
LWP save state byte size = 0x00000080 (128)
LWP save state byte offset = 0x00000340 (832)
supported in IA32_XSS or XCR0 = XCR0 (user state)
64-byte alignment in compacted XSAVE = false
XFD faulting supported = false
extended processor signature (0x80000001/eax):
family/generation = 0xf (15)
model = 0x3 (3)
stepping id = 0x1 (1)
extended family = 0x6 (6)
extended model = 0x1 (1)
(family synth) = 0x15 (21)
(model synth) = 0x13 (19)
(simple synth) = AMD (unknown type) (Richland RL-A1) [Piledriver], 32nm
extended feature flags (0x80000001/edx):
x87 FPU on chip = true
virtual-8086 mode enhancement = true
debugging extensions = true
page size extensions = true
time stamp counter = true
RDMSR and WRMSR support = true
physical address extensions = true
machine check exception = true
CMPXCHG8B inst. = true
APIC on chip = true
SYSCALL and SYSRET instructions = true
memory type range registers = true
global paging extension = true
machine check architecture = true
conditional move/compare instruction = true
page attribute table = true
page size extension = true
multiprocessing capable = false
no-execute page protection = true
AMD multimedia instruction extensions = true
MMX Technology = true
FXSAVE/FXRSTOR = true
SSE extensions = true
1-GB large page support = true
RDTSCP = true
long mode (AA-64) = true
3DNow! instruction extensions = false
3DNow! instructions = false
extended brand id (0x80000001/ebx):
raw = 0x20000000 (536870912)
BrandId = 0x0 (0)
PkgType = FM2 (PGA) (2)
AMD feature flags (0x80000001/ecx):
LAHF/SAHF supported in 64-bit mode = true
CMP Legacy = true
SVM: secure virtual machine = true
extended APIC space = true
AltMovCr8 = true
LZCNT advanced bit manipulation = true
SSE4A support = true
misaligned SSE mode = true
3DNow! PREFETCH/PREFETCHW instructions = true
OS visible workaround = true
instruction based sampling = true
XOP support = true
SKINIT/STGI support = true
watchdog timer support = true
lightweight profiling support = true
4-operand FMA instruction = true
TCE: translation cache extension = true
NodeId MSR C001100C = true
TBM support = true
topology extensions = true
core performance counter extensions = true
NB/DF performance counter extensions = true
data breakpoint extension = false
performance time-stamp counter support = false
LLC performance counter extensions = false
MWAITX/MONITORX supported = false
Address mask extension support = false
brand = "AMD A6-6400K APU with Radeon(tm) HD Graphics "
L1 TLB/cache information: 2M/4M pages & L1 TLB (0x80000005/eax):
instruction # entries = 0x18 (24)
instruction associativity = 0xff (255)
data # entries = 0x40 (64)
data associativity = 0xff (255)
L1 TLB/cache information: 4K pages & L1 TLB (0x80000005/ebx):
instruction # entries = 0x30 (48)
instruction associativity = 0xff (255)
data # entries = 0x40 (64)
data associativity = 0xff (255)
L1 data cache information (0x80000005/ecx):
line size (bytes) = 0x40 (64)
lines per tag = 0x1 (1)
associativity = 0x4 (4)
size (KB) = 0x10 (16)
L1 instruction cache information (0x80000005/edx):
line size (bytes) = 0x40 (64)
lines per tag = 0x1 (1)
associativity = 0x2 (2)
size (KB) = 0x40 (64)
L2 TLB/cache information: 2M/4M pages & L2 TLB (0x80000006/eax):
instruction # entries = 0x400 (1024)
instruction associativity = 8 to 15-way (6)
data # entries = 0x400 (1024)
data associativity = 8 to 15-way (6)
L2 TLB/cache information: 4K pages & L2 TLB (0x80000006/ebx):
instruction # entries = 0x200 (512)
instruction associativity = 4 to 5-way (4)
data # entries = 0x400 (1024)
data associativity = 8 to 15-way (6)
L2 unified cache information (0x80000006/ecx):
line size (bytes) = 0x40 (64)
lines per tag = 0x1 (1)
associativity = 16 to 31-way (8)
size (KB) = 0x400 (1024)
L3 cache information (0x80000006/edx):
line size (bytes) = 0x0 (0)
lines per tag = 0x0 (0)
associativity = L2 off (0)
size (in 512KB units) = 0x0 (0)
RAS Capability (0x80000007/ebx):
MCA overflow recovery support = false
SUCCOR support = false
HWA: hardware assert support = false
scalable MCA support = false
Advanced Power Management Features (0x80000007/ecx):
CmpUnitPwrSampleTimeRatio = 0x0 (0)
Advanced Power Management Features (0x80000007/edx):
TS: temperature sensing diode = true
FID: frequency ID control = false
VID: voltage ID control = false
TTP: thermal trip = true
TM: thermal monitor = true
STC: software thermal control = false
100 MHz multiplier control = true
hardware P-State control = true
TscInvariant = true
CPB: core performance boost = true
read-only effective frequency interface = true
processor feedback interface = false
APM power reporting = false
connected standby = false
RAPL: running average power limit = false
Physical Address and Linear Address Size (0x80000008/eax):
maximum physical address bits = 0x30 (48)
maximum linear (virtual) address bits = 0x30 (48)
maximum guest physical address bits = 0x0 (0)
Extended Feature Extensions ID (0x80000008/ebx):
CLZERO instruction = false
instructions retired count support = false
always save/restore error pointers = false
INVLPGB instruction = false
RDPRU instruction = false
memory bandwidth enforcement = false
MCOMMIT instruction = false
WBNOINVD instruction = false
IBPB: indirect branch prediction barrier = true
interruptible WBINVD, WBNOINVD = false
IBRS: indirect branch restr speculation = false
STIBP: 1 thr indirect branch predictor = false
CPU prefers: IBRS always on = false
CPU prefers: STIBP always on = false
IBRS preferred over software solution = false
IBRS provides same mode protection = false
EFER[LMSLE] not supported = false
INVLPGB supports TLB flush guest nested = false
ppin processor id number supported = false
SSBD: speculative store bypass disable = false
virtualized SSBD = false
SSBD fixed in hardware = false
CPPC: collaborative processor perf ctrl = false
PSFD: predictive store forward disable = false
not vulnerable to branch type confusion = false
branch sampling feature support = false
(vuln to branch type confusion synth) = true
Size Identifiers (0x80000008/ecx):
number of CPU cores = 0x2 (2)
ApicIdCoreIdSize = 0x4 (4)
performance time-stamp counter size = 40 bits (0)
Feature Extended Size (0x80000008/edx):
max page count for INVLPGB instruction = 0x0 (0)
RDPRU instruction max input support = 0x0 (0)
SVM Secure Virtual Machine (0x8000000a/eax):
SvmRev: SVM revision = 0x1 (1)
SVM Secure Virtual Machine (0x8000000a/edx):
nested paging = true
LBR virtualization = true
SVM lock = true
NRIP save = true
MSR based TSC rate control = true
VMCB clean bits support = true
flush by ASID = true
decode assists = true
SSSE3/SSE5 opcode set disable = false
pause intercept filter = true
pause filter threshold = true
AVIC: AMD virtual interrupt controller = false
virtualized VMLOAD/VMSAVE = false
virtualized global interrupt flag (GIF) = false
GMET: guest mode execute trap = false
X2AVIC: virtualized X2APIC = false
supervisor shadow stack = false
guest Spec_ctl support = false
ROGPT: read-only guest page table = false
host MCE override = false
INVLPGB/TLBSYNC hyperv interc enable = false
VNMI: NMI virtualization = false
IBS virtualization = false
guest SVME addr check = false
NASID: number of address space identifiers = 0x10000 (65536):
L1 TLB information: 1G pages (0x80000019/eax):
instruction # entries = 0x18 (24)
instruction associativity = full (15)
data # entries = 0x40 (64)
data associativity = full (15)
L2 TLB information: 1G pages (0x80000019/ebx):
instruction # entries = 0x400 (1024)
instruction associativity = 8 to 15-way (6)
data # entries = 0x400 (1024)
data associativity = 8 to 15-way (6)
Performance Optimization Identifiers (0x8000001a/eax):
128-bit SSE executed full-width = true
MOVU* better than MOVL*/MOVH* = true
256-bit SSE executed full-width = false
Instruction Based Sampling Identifiers (0x8000001b/eax):
IBS feature flags valid = true
IBS fetch sampling = true
IBS execution sampling = true
read write of op counter = true
op counting mode = true
branch target address reporting = true
IbsOpCurCnt and IbsOpMaxCnt extend 7 = true
invalid RIP indication support = true
fused branch micro-op indication support = false
IBS fetch control extended MSR support = false
IBS op data 4 MSR support = false
IBS L3 miss filtering support = false
Lightweight Profiling Capabilities: Availability (0x8000001c/eax):
lightweight profiling = false
LWPVAL instruction = false
instruction retired event = false
branch retired event = false
DC miss event = false
core clocks not halted event = false
core reference clocks not halted event = false
continuous mode sampling = false
tsc in event record = false
interrupt on threshold overflow = false
Lightweight Profiling Capabilities: Supported (0x8000001c/edx):
lightweight profiling = true
LWPVAL instruction = true
instruction retired event = true
branch retired event = true
DC miss event = false
core clocks not halted event = false
core reference clocks not halted event = false
continuous mode sampling = false
tsc in event record = false
interrupt on threshold overflow = true
Lightweight Profiling Capabilities (0x8000001c/ebx):
LWPCB byte size = 0x13 (19)
event record byte size = 0x20 (32)
maximum EventId = 0x3 (3)
EventInterval1 field offset = 0x80 (128)
Lightweight Profiling Capabilities (0x8000001c/ecx):
latency counter bit size = 0x0 (0)
data cache miss address valid = false
amount cache latency is rounded = 0x0 (0)
LWP implementation version = 0x1 (1)
event ring buffer size in records = 0x1 (1)
branch prediction filtering = false
IP filtering = false
cache level filtering = false
cache latency filteing = false
Cache Properties (0x8000001d):
--- cache 0 ---
type = data (1)
level = 0x1 (1)
self-initializing = true
fully associative = false
extra cores sharing this cache = 0x0 (0)
line size in bytes = 0x40 (64)
physical line partitions = 0x1 (1)
number of ways = 0x4 (4)
number of sets = 64
write-back invalidate = false
cache inclusive of lower levels = false
(synth size) = 16384 (16 KB)
--- cache 1 ---
type = instruction (2)
level = 0x1 (1)
self-initializing = true
fully associative = false
extra cores sharing this cache = 0x1 (1)
line size in bytes = 0x40 (64)
physical line partitions = 0x1 (1)
number of ways = 0x2 (2)
number of sets = 512
write-back invalidate = false
cache inclusive of lower levels = false
(synth size) = 65536 (64 KB)
--- cache 2 ---
type = unified (3)
level = 0x2 (2)
self-initializing = true
fully associative = false
extra cores sharing this cache = 0x1 (1)
line size in bytes = 0x40 (64)
physical line partitions = 0x1 (1)
number of ways = 0x10 (16)
number of sets = 1024
write-back invalidate = true
cache inclusive of lower levels = false
(synth size) = 1048576 (1024 KB)
extended APIC ID = 16
Compute Unit Identifiers (0x8000001e/ebx):
compute unit ID = 0x0 (0)
cores per compute unit = 0x2 (2)
Node Identifiers (0x8000001e/ecx):
node ID = 0x0 (0)
nodes per processor = 0x1 (1)
(instruction supported synth):
CMPXCHG8B = true
conditional move/compare = true
PREFETCH/PREFETCHW = true
(multi-processing synth) = multi-core (c=2)
(multi-processing method) = AMD
(APIC widths synth): CORE_width=1 SMT_width=0
(APIC synth): PKG_ID=0 CORE_ID=0 SMT_ID=0
(uarch synth) = AMD Piledriver, 32nm
(synth) = AMD A-Series (Richland RL-A1) [Piledriver], 32nm
CPU 1:
vendor_id = "AuthenticAMD"
version information (1/eax):
processor type = primary processor (0)
family = 0xf (15)
model = 0x3 (3)
stepping id = 0x1 (1)
extended family = 0x6 (6)
extended model = 0x1 (1)
(family synth) = 0x15 (21)
(model synth) = 0x13 (19)
(simple synth) = AMD (unknown type) (Richland RL-A1) [Piledriver], 32nm
miscellaneous (1/ebx):
process local APIC physical ID = 0x1 (1)
maximum IDs for CPUs in pkg = 0x2 (2)
CLFLUSH line size = 0x8 (8)
brand index = 0x0 (0)
brand id = 0x00 (0): unknown
feature information (1/edx):
x87 FPU on chip = true
VME: virtual-8086 mode enhancement = true
DE: debugging extensions = true
PSE: page size extensions = true
TSC: time stamp counter = true
RDMSR and WRMSR support = true
PAE: physical address extensions = true
MCE: machine check exception = true
CMPXCHG8B inst. = true
APIC on chip = true
SYSENTER and SYSEXIT = true
MTRR: memory type range registers = true
PTE global bit = true
MCA: machine check architecture = true
CMOV: conditional move/compare instr = true
PAT: page attribute table = true
PSE-36: page size extension = true
PSN: processor serial number = false
CLFLUSH instruction = true
DS: debug store = false
ACPI: thermal monitor and clock ctrl = false
MMX Technology = true
FXSAVE/FXRSTOR = true
SSE extensions = true
SSE2 extensions = true
SS: self snoop = false
hyper-threading / multi-core supported = true
TM: therm. monitor = false
IA64 = false
PBE: pending break event = false
feature information (1/ecx):
PNI/SSE3: Prescott New Instructions = true
PCLMULDQ instruction = true
DTES64: 64-bit debug store = false
MONITOR/MWAIT = true
CPL-qualified debug store = false
VMX: virtual machine extensions = false
SMX: safer mode extensions = false
Enhanced Intel SpeedStep Technology = false
TM2: thermal monitor 2 = false
SSSE3 extensions = true
context ID: adaptive or shared L1 data = false
SDBG: IA32_DEBUG_INTERFACE = false
FMA instruction = true
CMPXCHG16B instruction = true
xTPR disable = false
PDCM: perfmon and debug = false
PCID: process context identifiers = false
DCA: direct cache access = false
SSE4.1 extensions = true
SSE4.2 extensions = true
x2APIC: extended xAPIC support = false
MOVBE instruction = false
POPCNT instruction = true
time stamp counter deadline = false
AES instruction = true
XSAVE/XSTOR states = true
OS-enabled XSAVE/XSTOR = true
AVX: advanced vector extensions = true
F16C half-precision convert instruction = true
RDRAND instruction = false
hypervisor guest status = false
cache and TLB information (2):
processor serial number = 0061-0F31-0000-0000-0000-0000
deterministic cache parameters (4):
--- cache 0 ---
cache type = no more caches (0)
MONITOR/MWAIT (5):
smallest monitor-line size (bytes) = 0x40 (64)
largest monitor-line size (bytes) = 0x40 (64)
enum of Monitor-MWAIT exts supported = true
supports intrs as break-event for MWAIT = true
number of C0 sub C-states using MWAIT = 0x0 (0)
number of C1 sub C-states using MWAIT = 0x0 (0)
number of C2 sub C-states using MWAIT = 0x0 (0)
number of C3 sub C-states using MWAIT = 0x0 (0)
number of C4 sub C-states using MWAIT = 0x0 (0)
number of C5 sub C-states using MWAIT = 0x0 (0)
number of C6 sub C-states using MWAIT = 0x0 (0)
number of C7 sub C-states using MWAIT = 0x0 (0)
Thermal and Power Management Features (6):
digital thermometer = false
Intel Turbo Boost Technology = false
ARAT always running APIC timer = false
PLN power limit notification = false
ECMD extended clock modulation duty = false
PTM package thermal management = false
HWP base registers = false
HWP notification = false
HWP activity window = false
HWP energy performance preference = false
HWP package level request = false
HDC base registers = false
Intel Turbo Boost Max Technology 3.0 = false
HWP capabilities = false
HWP PECI override = false
flexible HWP = false
IA32_HWP_REQUEST MSR fast access mode = false
HW_FEEDBACK MSRs supported = false
ignoring idle logical processor HWP req = false
Thread Director = false
IA32_HW_FEEDBACK_THREAD_CONFIG bit 25 = false
digital thermometer thresholds = 0x0 (0)
hardware coordination feedback = true
ACNT2 available = false
performance-energy bias capability = false
number of enh hardware feedback classes = 0x0 (0)
performance capability reporting = false
energy efficiency capability reporting = false
size of feedback struct (4KB pages) = 0x1 (1)
index of CPU's row in feedback struct = 0x0 (0)
extended feature flags (7):
FSGSBASE instructions = false
IA32_TSC_ADJUST MSR supported = false
SGX: Software Guard Extensions supported = false
BMI1 instructions = true
HLE hardware lock elision = false
AVX2: advanced vector extensions 2 = false
FDP_EXCPTN_ONLY = false
SMEP supervisor mode exec protection = false
BMI2 instructions = false
enhanced REP MOVSB/STOSB = false
INVPCID instruction = false
RTM: restricted transactional memory = false
RDT-CMT/PQoS cache monitoring = false
deprecated FPU CS/DS = false
MPX: intel memory protection extensions = false
RDT-CAT/PQE cache allocation = false
AVX512F: AVX-512 foundation instructions = false
AVX512DQ: double & quadword instructions = false
RDSEED instruction = false
ADX instructions = false
SMAP: supervisor mode access prevention = false
AVX512IFMA: integer fused multiply add = false
PCOMMIT instruction = false
CLFLUSHOPT instruction = false
CLWB instruction = false
Intel processor trace = false
AVX512PF: prefetch instructions = false
AVX512ER: exponent & reciprocal instrs = false
AVX512CD: conflict detection instrs = false
SHA instructions = false
AVX512BW: byte & word instructions = false
AVX512VL: vector length = false
PREFETCHWT1 = false
AVX512VBMI: vector byte manipulation = false
UMIP: user-mode instruction prevention = false
PKU protection keys for user-mode = false
OSPKE CR4.PKE and RDPKRU/WRPKRU = false
WAITPKG instructions = false
AVX512_VBMI2: byte VPCOMPRESS, VPEXPAND = false
CET_SS: CET shadow stack = false
GFNI: Galois Field New Instructions = false
VAES instructions = false
VPCLMULQDQ instruction = false
AVX512_VNNI: neural network instructions = false
AVX512_BITALG: bit count/shiffle = false
TME: Total Memory Encryption = false
AVX512: VPOPCNTDQ instruction = false
LA57: 57-bit addrs & 5-level paging = false
BNDLDX/BNDSTX MAWAU value in 64-bit mode = 0x0 (0)
RDPID: read processor ID supported = false
KL: key locker = false
bus lock detection = false
CLDEMOTE supports cache line demote = false
MOVDIRI instruction = false
MOVDIR64B instruction = false
ENQCMD instruction = false
SGX_LC: SGX launch config supported = false
PKS: supervisor protection keys = false
SGX-KEYS: SGX attestation services = false
AVX512_4VNNIW: neural network instrs = false
AVX512_4FMAPS: multiply acc single prec = false
fast short REP MOV = false
UINTR: user interrupts = false
AVX512_VP2INTERSECT: intersect mask regs = false
IA32_MCU_OPT_CTRL SRBDS mitigation MSR = false
VERW MD_CLEAR microcode support = false
RTM transaction always aborts = false
IA32_TSX_FORCE_ABORT MSR = false
SERIALIZE instruction = false
hybrid part = false
TSXLDTRK: TSX suspend load addr tracking = false
PCONFIG instruction = false
LBR: architectural last branch records = false
CET_IBT: CET indirect branch tracking = false
AMX-BF16: tile bfloat16 support = false
AVX512_FP16: fp16 support = false
AMX-TILE: tile architecture support = false
AMX-INT8: tile 8-bit integer support = false
IBRS/IBPB: indirect branch restrictions = false
STIBP: 1 thr indirect branch predictor = false
L1D_FLUSH: IA32_FLUSH_CMD MSR = false
IA32_ARCH_CAPABILITIES MSR = false
IA32_CORE_CAPABILITIES MSR = false
SSBD: speculative store bypass disable = false
Direct Cache Access Parameters (9):
PLATFORM_DCA_CAP MSR bits = 0
Architecture Performance Monitoring Features (0xa):
version ID = 0x0 (0)
number of counters per logical processor = 0x0 (0)
bit width of counter = 0x0 (0)
length of EBX bit vector = 0x0 (0)
core cycle event = not available
instruction retired event = not available
reference cycles event = not available
last-level cache ref event = not available
last-level cache miss event = not available
branch inst retired event = not available
branch mispred retired event = not available
top-down slots event = not available
fixed counter 0 supported = false
fixed counter 1 supported = false
fixed counter 2 supported = false
fixed counter 3 supported = false
fixed counter 4 supported = false
fixed counter 5 supported = false
fixed counter 6 supported = false
fixed counter 7 supported = false
fixed counter 8 supported = false
fixed counter 9 supported = false
fixed counter 10 supported = false
fixed counter 11 supported = false
fixed counter 12 supported = false
fixed counter 13 supported = false
fixed counter 14 supported = false
fixed counter 15 supported = false
fixed counter 16 supported = false
fixed counter 17 supported = false
fixed counter 18 supported = false
fixed counter 19 supported = false
fixed counter 20 supported = false
fixed counter 21 supported = false
fixed counter 22 supported = false
fixed counter 23 supported = false
fixed counter 24 supported = false
fixed counter 25 supported = false
fixed counter 26 supported = false
fixed counter 27 supported = false
fixed counter 28 supported = false
fixed counter 29 supported = false
fixed counter 30 supported = false
fixed counter 31 supported = false
number of contiguous fixed counters = 0x0 (0)
bit width of fixed counters = 0x0 (0)
anythread deprecation = false
x2APIC features / processor topology (0xb):
extended APIC ID = 0
--- level 0 ---
level number = 0x0 (0)
level type = invalid (0)
bit width of level = 0x0 (0)
number of logical processors at level = 0x0 (0)
XSAVE features (0xd/0):
XCR0 valid bit field mask = 0x4000000000000007
x87 state = true
SSE state = true
AVX state = true
MPX BNDREGS = false
MPX BNDCSR = false
AVX-512 opmask = false
AVX-512 ZMM_Hi256 = false
AVX-512 Hi16_ZMM = false
PKRU state = false
XTILECFG state = false
XTILEDATA state = false
bytes required by fields in XCR0 = 0x00000340 (832)
bytes required by XSAVE/XRSTOR area = 0x000003c0 (960)
XSAVEOPT instruction = false
XSAVEC instruction = false
XGETBV instruction = false
XSAVES/XRSTORS instructions = false
XFD: extended feature disable supported = false
SAVE area size in bytes = 0x00000000 (0)
IA32_XSS valid bit field mask = 0x0000000000000000
PT state = false
PASID state = false
CET_U user state = false
CET_S supervisor state = false
HDC state = false
UINTR state = false
LBR state = false
HWP state = false
AVX/YMM features (0xd/2):
AVX/YMM save state byte size = 0x00000100 (256)
AVX/YMM save state byte offset = 0x00000240 (576)
supported in IA32_XSS or XCR0 = XCR0 (user state)
64-byte alignment in compacted XSAVE = false
XFD faulting supported = false
LWP features (0xd/0x3e):
LWP save state byte size = 0x00000080 (128)
LWP save state byte offset = 0x00000340 (832)
supported in IA32_XSS or XCR0 = XCR0 (user state)
64-byte alignment in compacted XSAVE = false
XFD faulting supported = false
extended processor signature (0x80000001/eax):
family/generation = 0xf (15)
model = 0x3 (3)
stepping id = 0x1 (1)
extended family = 0x6 (6)
extended model = 0x1 (1)
(family synth) = 0x15 (21)
(model synth) = 0x13 (19)
(simple synth) = AMD (unknown type) (Richland RL-A1) [Piledriver], 32nm
extended feature flags (0x80000001/edx):
x87 FPU on chip = true
virtual-8086 mode enhancement = true
debugging extensions = true
page size extensions = true
time stamp counter = true
RDMSR and WRMSR support = true
physical address extensions = true
machine check exception = true
CMPXCHG8B inst. = true
APIC on chip = true
SYSCALL and SYSRET instructions = true
memory type range registers = true
global paging extension = true
machine check architecture = true
conditional move/compare instruction = true
page attribute table = true
page size extension = true
multiprocessing capable = false
no-execute page protection = true
AMD multimedia instruction extensions = true
MMX Technology = true
FXSAVE/FXRSTOR = true
SSE extensions = true
1-GB large page support = true
RDTSCP = true
long mode (AA-64) = true
3DNow! instruction extensions = false
3DNow! instructions = false
extended brand id (0x80000001/ebx):
raw = 0x20000000 (536870912)
BrandId = 0x0 (0)
PkgType = FM2 (PGA) (2)
AMD feature flags (0x80000001/ecx):
LAHF/SAHF supported in 64-bit mode = true
CMP Legacy = true
SVM: secure virtual machine = true
extended APIC space = true
AltMovCr8 = true
LZCNT advanced bit manipulation = true
SSE4A support = true
misaligned SSE mode = true
3DNow! PREFETCH/PREFETCHW instructions = true
OS visible workaround = true
instruction based sampling = true
XOP support = true
SKINIT/STGI support = true
watchdog timer support = true
lightweight profiling support = true
4-operand FMA instruction = true
TCE: translation cache extension = true
NodeId MSR C001100C = true
TBM support = true
topology extensions = true
core performance counter extensions = true
NB/DF performance counter extensions = true
data breakpoint extension = false
performance time-stamp counter support = false
LLC performance counter extensions = false
MWAITX/MONITORX supported = false
Address mask extension support = false
brand = "AMD A6-6400K APU with Radeon(tm) HD Graphics "
L1 TLB/cache information: 2M/4M pages & L1 TLB (0x80000005/eax):
instruction # entries = 0x18 (24)
instruction associativity = 0xff (255)
data # entries = 0x40 (64)
data associativity = 0xff (255)
L1 TLB/cache information: 4K pages & L1 TLB (0x80000005/ebx):
instruction # entries = 0x30 (48)
instruction associativity = 0xff (255)
data # entries = 0x40 (64)
data associativity = 0xff (255)
L1 data cache information (0x80000005/ecx):
line size (bytes) = 0x40 (64)
lines per tag = 0x1 (1)
associativity = 0x4 (4)
size (KB) = 0x10 (16)
L1 instruction cache information (0x80000005/edx):
line size (bytes) = 0x40 (64)
lines per tag = 0x1 (1)
associativity = 0x2 (2)
size (KB) = 0x40 (64)
L2 TLB/cache information: 2M/4M pages & L2 TLB (0x80000006/eax):
instruction # entries = 0x400 (1024)
instruction associativity = 8 to 15-way (6)
data # entries = 0x400 (1024)
data associativity = 8 to 15-way (6)
L2 TLB/cache information: 4K pages & L2 TLB (0x80000006/ebx):
instruction # entries = 0x200 (512)
instruction associativity = 4 to 5-way (4)
data # entries = 0x400 (1024)
data associativity = 8 to 15-way (6)
L2 unified cache information (0x80000006/ecx):
line size (bytes) = 0x40 (64)
lines per tag = 0x1 (1)
associativity = 16 to 31-way (8)
size (KB) = 0x400 (1024)
L3 cache information (0x80000006/edx):
line size (bytes) = 0x0 (0)
lines per tag = 0x0 (0)
associativity = L2 off (0)
size (in 512KB units) = 0x0 (0)
RAS Capability (0x80000007/ebx):
MCA overflow recovery support = false
SUCCOR support = false
HWA: hardware assert support = false
scalable MCA support = false
Advanced Power Management Features (0x80000007/ecx):
CmpUnitPwrSampleTimeRatio = 0x0 (0)
Advanced Power Management Features (0x80000007/edx):
TS: temperature sensing diode = true
FID: frequency ID control = false
VID: voltage ID control = false
TTP: thermal trip = true
TM: thermal monitor = true
STC: software thermal control = false
100 MHz multiplier control = true
hardware P-State control = true
TscInvariant = true
CPB: core performance boost = true
read-only effective frequency interface = true
processor feedback interface = false
APM power reporting = false
connected standby = false
RAPL: running average power limit = false
Physical Address and Linear Address Size (0x80000008/eax):
maximum physical address bits = 0x30 (48)
maximum linear (virtual) address bits = 0x30 (48)
maximum guest physical address bits = 0x0 (0)
Extended Feature Extensions ID (0x80000008/ebx):
CLZERO instruction = false
instructions retired count support = false
always save/restore error pointers = false
INVLPGB instruction = false
RDPRU instruction = false
memory bandwidth enforcement = false
MCOMMIT instruction = false
WBNOINVD instruction = false
IBPB: indirect branch prediction barrier = true
interruptible WBINVD, WBNOINVD = false
IBRS: indirect branch restr speculation = false
STIBP: 1 thr indirect branch predictor = false
CPU prefers: IBRS always on = false
CPU prefers: STIBP always on = false
IBRS preferred over software solution = false
IBRS provides same mode protection = false
EFER[LMSLE] not supported = false
INVLPGB supports TLB flush guest nested = false
ppin processor id number supported = false
SSBD: speculative store bypass disable = false
virtualized SSBD = false
SSBD fixed in hardware = false
CPPC: collaborative processor perf ctrl = false
PSFD: predictive store forward disable = false
not vulnerable to branch type confusion = false
branch sampling feature support = false
(vuln to branch type confusion synth) = true
Size Identifiers (0x80000008/ecx):
number of CPU cores = 0x2 (2)
ApicIdCoreIdSize = 0x4 (4)
performance time-stamp counter size = 40 bits (0)
Feature Extended Size (0x80000008/edx):
max page count for INVLPGB instruction = 0x0 (0)
RDPRU instruction max input support = 0x0 (0)
SVM Secure Virtual Machine (0x8000000a/eax):
SvmRev: SVM revision = 0x1 (1)
SVM Secure Virtual Machine (0x8000000a/edx):
nested paging = true
LBR virtualization = true
SVM lock = true
NRIP save = true
MSR based TSC rate control = true
VMCB clean bits support = true
flush by ASID = true
decode assists = true
SSSE3/SSE5 opcode set disable = false
pause intercept filter = true
pause filter threshold = true
AVIC: AMD virtual interrupt controller = false
virtualized VMLOAD/VMSAVE = false
virtualized global interrupt flag (GIF) = false
GMET: guest mode execute trap = false
X2AVIC: virtualized X2APIC = false
supervisor shadow stack = false
guest Spec_ctl support = false
ROGPT: read-only guest page table = false
host MCE override = false
INVLPGB/TLBSYNC hyperv interc enable = false
VNMI: NMI virtualization = false
IBS virtualization = false
guest SVME addr check = false
NASID: number of address space identifiers = 0x10000 (65536):
L1 TLB information: 1G pages (0x80000019/eax):
instruction # entries = 0x18 (24)
instruction associativity = full (15)
data # entries = 0x40 (64)
data associativity = full (15)
L2 TLB information: 1G pages (0x80000019/ebx):
instruction # entries = 0x400 (1024)
instruction associativity = 8 to 15-way (6)
data # entries = 0x400 (1024)
data associativity = 8 to 15-way (6)
Performance Optimization Identifiers (0x8000001a/eax):
128-bit SSE executed full-width = true
MOVU* better than MOVL*/MOVH* = true
256-bit SSE executed full-width = false
Instruction Based Sampling Identifiers (0x8000001b/eax):
IBS feature flags valid = true
IBS fetch sampling = true
IBS execution sampling = true
read write of op counter = true
op counting mode = true
branch target address reporting = true
IbsOpCurCnt and IbsOpMaxCnt extend 7 = true
invalid RIP indication support = true
fused branch micro-op indication support = false
IBS fetch control extended MSR support = false
IBS op data 4 MSR support = false
IBS L3 miss filtering support = false
Lightweight Profiling Capabilities: Availability (0x8000001c/eax):
lightweight profiling = false
LWPVAL instruction = false
instruction retired event = false
branch retired event = false
DC miss event = false
core clocks not halted event = false
core reference clocks not halted event = false
continuous mode sampling = false
tsc in event record = false
interrupt on threshold overflow = false
Lightweight Profiling Capabilities: Supported (0x8000001c/edx):
lightweight profiling = true
LWPVAL instruction = true
instruction retired event = true
branch retired event = true
DC miss event = false
core clocks not halted event = false
core reference clocks not halted event = false
continuous mode sampling = false
tsc in event record = false
interrupt on threshold overflow = true
Lightweight Profiling Capabilities (0x8000001c/ebx):
LWPCB byte size = 0x13 (19)
event record byte size = 0x20 (32)
maximum EventId = 0x3 (3)
EventInterval1 field offset = 0x80 (128)
Lightweight Profiling Capabilities (0x8000001c/ecx):
latency counter bit size = 0x0 (0)
data cache miss address valid = false
amount cache latency is rounded = 0x0 (0)
LWP implementation version = 0x1 (1)
event ring buffer size in records = 0x1 (1)
branch prediction filtering = false
IP filtering = false
cache level filtering = false
cache latency filteing = false
Cache Properties (0x8000001d):
--- cache 0 ---
type = data (1)
level = 0x1 (1)
self-initializing = true
fully associative = false
extra cores sharing this cache = 0x0 (0)
line size in bytes = 0x40 (64)
physical line partitions = 0x1 (1)
number of ways = 0x4 (4)
number of sets = 64
write-back invalidate = false
cache inclusive of lower levels = false
(synth size) = 16384 (16 KB)
--- cache 1 ---
type = instruction (2)
level = 0x1 (1)
self-initializing = true
fully associative = false
extra cores sharing this cache = 0x1 (1)
line size in bytes = 0x40 (64)
physical line partitions = 0x1 (1)
number of ways = 0x2 (2)
number of sets = 512
write-back invalidate = false
cache inclusive of lower levels = false
(synth size) = 65536 (64 KB)
--- cache 2 ---
type = unified (3)
level = 0x2 (2)
self-initializing = true
fully associative = false
extra cores sharing this cache = 0x1 (1)
line size in bytes = 0x40 (64)
physical line partitions = 0x1 (1)
number of ways = 0x10 (16)
number of sets = 1024
write-back invalidate = true
cache inclusive of lower levels = false
(synth size) = 1048576 (1024 KB)
extended APIC ID = 17
Compute Unit Identifiers (0x8000001e/ebx):
compute unit ID = 0x0 (0)
cores per compute unit = 0x2 (2)
Node Identifiers (0x8000001e/ecx):
node ID = 0x0 (0)
nodes per processor = 0x1 (1)
(instruction supported synth):
CMPXCHG8B = true
conditional move/compare = true
PREFETCH/PREFETCHW = true
(multi-processing synth) = multi-core (c=2)
(multi-processing method) = AMD
(APIC widths synth): CORE_width=1 SMT_width=0
(APIC synth): PKG_ID=0 CORE_ID=1 SMT_ID=0
(uarch synth) = AMD Piledriver, 32nm
(synth) = AMD A-Series (Richland RL-A1) [Piledriver], 32nm
[-- Attachment #3: Type: text/plain, Size: 176 bytes --]
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
2023-04-18 20:10 ` Paul Menzel
(?)
@ 2023-04-19 9:38 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-19 9:38 UTC (permalink / raw)
To: Paul Menzel
Cc: linux-kernel, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Paul!
On Tue, Apr 18 2023 at 22:10, Paul Menzel wrote:
> Am 18.04.23 um 10:40 schrieb Thomas Gleixner:
>> Can you please provide the output of cpuid?
>
> Of course. Here the top, and the whole output is attached.
Thanks for the data. Can you please apply the debug patch below and
provide the dmesg output? Just the line which is added by the patch is
enough. You can boot with cpuhp.parallel=off so you don't have wait for
10 seconds.
Thanks,
tglx
---
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -814,6 +814,7 @@ static int wakeup_secondary_cpu_via_init
unsigned long send_status = 0, accept_status = 0;
int maxlvt, num_starts, j;
+ pr_info("Kicking AP alive: %d\n", phys_apicid);
preempt_disable();
maxlvt = lapic_get_maxlvt();
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-19 9:38 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-19 9:38 UTC (permalink / raw)
To: Paul Menzel
Cc: linux-kernel, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Paul!
On Tue, Apr 18 2023 at 22:10, Paul Menzel wrote:
> Am 18.04.23 um 10:40 schrieb Thomas Gleixner:
>> Can you please provide the output of cpuid?
>
> Of course. Here the top, and the whole output is attached.
Thanks for the data. Can you please apply the debug patch below and
provide the dmesg output? Just the line which is added by the patch is
enough. You can boot with cpuhp.parallel=off so you don't have wait for
10 seconds.
Thanks,
tglx
---
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -814,6 +814,7 @@ static int wakeup_secondary_cpu_via_init
unsigned long send_status = 0, accept_status = 0;
int maxlvt, num_starts, j;
+ pr_info("Kicking AP alive: %d\n", phys_apicid);
preempt_disable();
maxlvt = lapic_get_maxlvt();
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-19 9:38 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-19 9:38 UTC (permalink / raw)
To: Paul Menzel
Cc: linux-kernel, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Paul!
On Tue, Apr 18 2023 at 22:10, Paul Menzel wrote:
> Am 18.04.23 um 10:40 schrieb Thomas Gleixner:
>> Can you please provide the output of cpuid?
>
> Of course. Here the top, and the whole output is attached.
Thanks for the data. Can you please apply the debug patch below and
provide the dmesg output? Just the line which is added by the patch is
enough. You can boot with cpuhp.parallel=off so you don't have wait for
10 seconds.
Thanks,
tglx
---
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -814,6 +814,7 @@ static int wakeup_secondary_cpu_via_init
unsigned long send_status = 0, accept_status = 0;
int maxlvt, num_starts, j;
+ pr_info("Kicking AP alive: %d\n", phys_apicid);
preempt_disable();
maxlvt = lapic_get_maxlvt();
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
2023-04-19 9:38 ` Thomas Gleixner
(?)
@ 2023-04-19 12:38 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-19 12:38 UTC (permalink / raw)
To: Paul Menzel
Cc: linux-kernel, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Wed, Apr 19 2023 at 11:38, Thomas Gleixner wrote:
> On Tue, Apr 18 2023 at 22:10, Paul Menzel wrote:
>> Am 18.04.23 um 10:40 schrieb Thomas Gleixner:
>>> Can you please provide the output of cpuid?
>>
>> Of course. Here the top, and the whole output is attached.
>
> Thanks for the data. Can you please apply the debug patch below and
> provide the dmesg output? Just the line which is added by the patch is
> enough. You can boot with cpuhp.parallel=off so you don't have wait for
> 10 seconds.
Borislav found some a machine which also refuses to boot. It turns of
the debug patch was spot on:
[ 0.462724] .... node #0, CPUs: #1
[ 0.462731] smpboot: Kicking AP alive: 17
[ 0.465723] #2
[ 0.465732] smpboot: Kicking AP alive: 18
[ 0.467641] #3
[ 0.467641] smpboot: Kicking AP alive: 19
So the kernel gets APICID 17, 18, 19 from ACPI but CPUID leaf 0x1
ebx[31:24], which is the initial APICID has:
CPU1 0x01
CPU2 0x02
CPU3 0x03
Which means the APICID to Linux CPU number lookup based on CPUID 0x01
fails for all of them and stops them dead in the low level startup code.
IOW, the BIOS assignes random numbers to the AP APICs for whatever
raisins, which leaves the parallel startup low level code up a creek
without a paddle, except for actually reading the APICID back from the
APIC. *SHUDDER*
I'm leaning towards disabling the CPUID lead 0x01 based discovery and be
done with it.
Thanks,
tglx
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-19 12:38 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-19 12:38 UTC (permalink / raw)
To: Paul Menzel
Cc: linux-kernel, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Wed, Apr 19 2023 at 11:38, Thomas Gleixner wrote:
> On Tue, Apr 18 2023 at 22:10, Paul Menzel wrote:
>> Am 18.04.23 um 10:40 schrieb Thomas Gleixner:
>>> Can you please provide the output of cpuid?
>>
>> Of course. Here the top, and the whole output is attached.
>
> Thanks for the data. Can you please apply the debug patch below and
> provide the dmesg output? Just the line which is added by the patch is
> enough. You can boot with cpuhp.parallel=off so you don't have wait for
> 10 seconds.
Borislav found some a machine which also refuses to boot. It turns of
the debug patch was spot on:
[ 0.462724] .... node #0, CPUs: #1
[ 0.462731] smpboot: Kicking AP alive: 17
[ 0.465723] #2
[ 0.465732] smpboot: Kicking AP alive: 18
[ 0.467641] #3
[ 0.467641] smpboot: Kicking AP alive: 19
So the kernel gets APICID 17, 18, 19 from ACPI but CPUID leaf 0x1
ebx[31:24], which is the initial APICID has:
CPU1 0x01
CPU2 0x02
CPU3 0x03
Which means the APICID to Linux CPU number lookup based on CPUID 0x01
fails for all of them and stops them dead in the low level startup code.
IOW, the BIOS assignes random numbers to the AP APICs for whatever
raisins, which leaves the parallel startup low level code up a creek
without a paddle, except for actually reading the APICID back from the
APIC. *SHUDDER*
I'm leaning towards disabling the CPUID lead 0x01 based discovery and be
done with it.
Thanks,
tglx
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-19 12:38 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-19 12:38 UTC (permalink / raw)
To: Paul Menzel
Cc: linux-kernel, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Wed, Apr 19 2023 at 11:38, Thomas Gleixner wrote:
> On Tue, Apr 18 2023 at 22:10, Paul Menzel wrote:
>> Am 18.04.23 um 10:40 schrieb Thomas Gleixner:
>>> Can you please provide the output of cpuid?
>>
>> Of course. Here the top, and the whole output is attached.
>
> Thanks for the data. Can you please apply the debug patch below and
> provide the dmesg output? Just the line which is added by the patch is
> enough. You can boot with cpuhp.parallel=off so you don't have wait for
> 10 seconds.
Borislav found some a machine which also refuses to boot. It turns of
the debug patch was spot on:
[ 0.462724] .... node #0, CPUs: #1
[ 0.462731] smpboot: Kicking AP alive: 17
[ 0.465723] #2
[ 0.465732] smpboot: Kicking AP alive: 18
[ 0.467641] #3
[ 0.467641] smpboot: Kicking AP alive: 19
So the kernel gets APICID 17, 18, 19 from ACPI but CPUID leaf 0x1
ebx[31:24], which is the initial APICID has:
CPU1 0x01
CPU2 0x02
CPU3 0x03
Which means the APICID to Linux CPU number lookup based on CPUID 0x01
fails for all of them and stops them dead in the low level startup code.
IOW, the BIOS assignes random numbers to the AP APICs for whatever
raisins, which leaves the parallel startup low level code up a creek
without a paddle, except for actually reading the APICID back from the
APIC. *SHUDDER*
I'm leaning towards disabling the CPUID lead 0x01 based discovery and be
done with it.
Thanks,
tglx
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
2023-04-19 12:38 ` Thomas Gleixner
(?)
@ 2023-04-19 13:32 ` David Woodhouse
-1 siblings, 0 replies; 236+ messages in thread
From: David Woodhouse @ 2023-04-19 13:32 UTC (permalink / raw)
To: Thomas Gleixner, Paul Menzel
Cc: linux-kernel, x86, Andrew Cooper, Brian Gerst, Arjan van de Veen,
Paolo Bonzini, Paul McKenney, Tom Lendacky, Sean Christopherson,
Oleksandr Natalenko, Guilherme G. Piccoli, Piotr Gorski,
Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
[-- Attachment #1: Type: text/plain, Size: 286 bytes --]
On Wed, 2023-04-19 at 14:38 +0200, Thomas Gleixner wrote:
>
> I'm leaning towards disabling the CPUID lead 0x01 based discovery and be
> done with it.
Makes sense. The large machines where users really want the parallel
startup all ought to have X2APIC and hence CPUID 0x0b.
[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5965 bytes --]
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-19 13:32 ` David Woodhouse
0 siblings, 0 replies; 236+ messages in thread
From: David Woodhouse @ 2023-04-19 13:32 UTC (permalink / raw)
To: Thomas Gleixner, Paul Menzel
Cc: linux-kernel, x86, Andrew Cooper, Brian Gerst, Arjan van de Veen,
Paolo Bonzini, Paul McKenney, Tom Lendacky, Sean Christopherson,
Oleksandr Natalenko, Guilherme G. Piccoli, Piotr Gorski,
Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
[-- Attachment #1.1: Type: text/plain, Size: 286 bytes --]
On Wed, 2023-04-19 at 14:38 +0200, Thomas Gleixner wrote:
>
> I'm leaning towards disabling the CPUID lead 0x01 based discovery and be
> done with it.
Makes sense. The large machines where users really want the parallel
startup all ought to have X2APIC and hence CPUID 0x0b.
[-- Attachment #1.2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5965 bytes --]
[-- Attachment #2: Type: text/plain, Size: 161 bytes --]
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-19 13:32 ` David Woodhouse
0 siblings, 0 replies; 236+ messages in thread
From: David Woodhouse @ 2023-04-19 13:32 UTC (permalink / raw)
To: Thomas Gleixner, Paul Menzel
Cc: linux-kernel, x86, Andrew Cooper, Brian Gerst, Arjan van de Veen,
Paolo Bonzini, Paul McKenney, Tom Lendacky, Sean Christopherson,
Oleksandr Natalenko, Guilherme G. Piccoli, Piotr Gorski,
Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
[-- Attachment #1.1: Type: text/plain, Size: 286 bytes --]
On Wed, 2023-04-19 at 14:38 +0200, Thomas Gleixner wrote:
>
> I'm leaning towards disabling the CPUID lead 0x01 based discovery and be
> done with it.
Makes sense. The large machines where users really want the parallel
startup all ought to have X2APIC and hence CPUID 0x0b.
[-- Attachment #1.2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5965 bytes --]
[-- Attachment #2: Type: text/plain, Size: 176 bytes --]
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
2023-04-19 12:38 ` Thomas Gleixner
(?)
@ 2023-04-19 13:43 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-19 13:43 UTC (permalink / raw)
To: Paul Menzel
Cc: linux-kernel, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Wed, Apr 19 2023 at 14:38, Thomas Gleixner wrote:
> On Wed, Apr 19 2023 at 11:38, Thomas Gleixner wrote:
> IOW, the BIOS assignes random numbers to the AP APICs for whatever
> raisins, which leaves the parallel startup low level code up a creek
> without a paddle, except for actually reading the APICID back from the
> APIC. *SHUDDER*
So Andrew just pointed out on IRC that this might be related to the
ancient issue of the 3-wire APIC bus where IO/APIC and APIC shared the
ID space, but that system is definitely post 3-wire APIC :)
Thanks,
tglx
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-19 13:43 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-19 13:43 UTC (permalink / raw)
To: Paul Menzel
Cc: linux-kernel, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Wed, Apr 19 2023 at 14:38, Thomas Gleixner wrote:
> On Wed, Apr 19 2023 at 11:38, Thomas Gleixner wrote:
> IOW, the BIOS assignes random numbers to the AP APICs for whatever
> raisins, which leaves the parallel startup low level code up a creek
> without a paddle, except for actually reading the APICID back from the
> APIC. *SHUDDER*
So Andrew just pointed out on IRC that this might be related to the
ancient issue of the 3-wire APIC bus where IO/APIC and APIC shared the
ID space, but that system is definitely post 3-wire APIC :)
Thanks,
tglx
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-19 13:43 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-19 13:43 UTC (permalink / raw)
To: Paul Menzel
Cc: linux-kernel, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Wed, Apr 19 2023 at 14:38, Thomas Gleixner wrote:
> On Wed, Apr 19 2023 at 11:38, Thomas Gleixner wrote:
> IOW, the BIOS assignes random numbers to the AP APICs for whatever
> raisins, which leaves the parallel startup low level code up a creek
> without a paddle, except for actually reading the APICID back from the
> APIC. *SHUDDER*
So Andrew just pointed out on IRC that this might be related to the
ancient issue of the 3-wire APIC bus where IO/APIC and APIC shared the
ID space, but that system is definitely post 3-wire APIC :)
Thanks,
tglx
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
2023-04-19 13:43 ` Thomas Gleixner
(?)
@ 2023-04-19 13:50 ` Andrew Cooper
-1 siblings, 0 replies; 236+ messages in thread
From: Andrew Cooper @ 2023-04-19 13:50 UTC (permalink / raw)
To: Thomas Gleixner, Paul Menzel
Cc: linux-kernel, x86, David Woodhouse, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On 19/04/2023 2:43 pm, Thomas Gleixner wrote:
> On Wed, Apr 19 2023 at 14:38, Thomas Gleixner wrote:
>> On Wed, Apr 19 2023 at 11:38, Thomas Gleixner wrote:
>> IOW, the BIOS assignes random numbers to the AP APICs for whatever
>> raisins, which leaves the parallel startup low level code up a creek
>> without a paddle, except for actually reading the APICID back from the
>> APIC. *SHUDDER*
> So Andrew just pointed out on IRC that this might be related to the
> ancient issue of the 3-wire APIC bus where IO/APIC and APIC shared the
> ID space, but that system is definitely post 3-wire APIC :)
Doesn't mean the BIOS code was updated adequately following that.
What I'm confused by is why this system boots in the first place. I can
only think that's is a system which only has 4-bit APIC IDs, and happens
to function when bit 4 gets truncated off the top of the SIPI destination...
~Andrew
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-19 13:50 ` Andrew Cooper
0 siblings, 0 replies; 236+ messages in thread
From: Andrew Cooper @ 2023-04-19 13:50 UTC (permalink / raw)
To: Thomas Gleixner, Paul Menzel
Cc: linux-kernel, x86, David Woodhouse, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On 19/04/2023 2:43 pm, Thomas Gleixner wrote:
> On Wed, Apr 19 2023 at 14:38, Thomas Gleixner wrote:
>> On Wed, Apr 19 2023 at 11:38, Thomas Gleixner wrote:
>> IOW, the BIOS assignes random numbers to the AP APICs for whatever
>> raisins, which leaves the parallel startup low level code up a creek
>> without a paddle, except for actually reading the APICID back from the
>> APIC. *SHUDDER*
> So Andrew just pointed out on IRC that this might be related to the
> ancient issue of the 3-wire APIC bus where IO/APIC and APIC shared the
> ID space, but that system is definitely post 3-wire APIC :)
Doesn't mean the BIOS code was updated adequately following that.
What I'm confused by is why this system boots in the first place. I can
only think that's is a system which only has 4-bit APIC IDs, and happens
to function when bit 4 gets truncated off the top of the SIPI destination...
~Andrew
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-19 13:50 ` Andrew Cooper
0 siblings, 0 replies; 236+ messages in thread
From: Andrew Cooper @ 2023-04-19 13:50 UTC (permalink / raw)
To: Thomas Gleixner, Paul Menzel
Cc: linux-kernel, x86, David Woodhouse, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On 19/04/2023 2:43 pm, Thomas Gleixner wrote:
> On Wed, Apr 19 2023 at 14:38, Thomas Gleixner wrote:
>> On Wed, Apr 19 2023 at 11:38, Thomas Gleixner wrote:
>> IOW, the BIOS assignes random numbers to the AP APICs for whatever
>> raisins, which leaves the parallel startup low level code up a creek
>> without a paddle, except for actually reading the APICID back from the
>> APIC. *SHUDDER*
> So Andrew just pointed out on IRC that this might be related to the
> ancient issue of the 3-wire APIC bus where IO/APIC and APIC shared the
> ID space, but that system is definitely post 3-wire APIC :)
Doesn't mean the BIOS code was updated adequately following that.
What I'm confused by is why this system boots in the first place. I can
only think that's is a system which only has 4-bit APIC IDs, and happens
to function when bit 4 gets truncated off the top of the SIPI destination...
~Andrew
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
2023-04-19 13:50 ` Andrew Cooper
(?)
@ 2023-04-19 16:21 ` Andrew Cooper
-1 siblings, 0 replies; 236+ messages in thread
From: Andrew Cooper @ 2023-04-19 16:21 UTC (permalink / raw)
To: Thomas Gleixner, Paul Menzel
Cc: linux-kernel, x86, David Woodhouse, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On 19/04/2023 2:50 pm, Andrew Cooper wrote:
> On 19/04/2023 2:43 pm, Thomas Gleixner wrote:
>> On Wed, Apr 19 2023 at 14:38, Thomas Gleixner wrote:
>>> On Wed, Apr 19 2023 at 11:38, Thomas Gleixner wrote:
>>> IOW, the BIOS assignes random numbers to the AP APICs for whatever
>>> raisins, which leaves the parallel startup low level code up a creek
>>> without a paddle, except for actually reading the APICID back from the
>>> APIC. *SHUDDER*
>> So Andrew just pointed out on IRC that this might be related to the
>> ancient issue of the 3-wire APIC bus where IO/APIC and APIC shared the
>> ID space, but that system is definitely post 3-wire APIC :)
> Doesn't mean the BIOS code was updated adequately following that.
>
> What I'm confused by is why this system boots in the first place. I can
> only think that's is a system which only has 4-bit APIC IDs, and happens
> to function when bit 4 gets truncated off the top of the SIPI destination...
https://www.amd.com/system/files/TechDocs/42300_15h_Mod_10h-1Fh_BKDG.pdf
This system does still require the IO-APICs to be at 0, and the LAPICs
to start at some offset, which is clearly 16 in this case. Also, this
system has configurable 4-bit or 8-bit wide APIC IDs, and I can't tell
which mode is active just from the manual.
But, it does mean that the BIOS has genuinely modified the APIC IDs of
the logic processors. This does highlight an error in reasoning with
the parallel bringup code.
For xAPIC, the APIC_ID register is writeable (at least, model
specifically), and CPUID is only the value it would have had at reset.
So the AP bringup logic can't actually use CPUID reliably.
This was changed in x2APIC, which made the x2APIC_ID immutable.
I don't see an option other than the AP bringup code query for xAPIC vs
x2APIC mode, and either looking at the real APIC_ID register, or falling
back to CPUID.
~Andrew
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-19 16:21 ` Andrew Cooper
0 siblings, 0 replies; 236+ messages in thread
From: Andrew Cooper @ 2023-04-19 16:21 UTC (permalink / raw)
To: Thomas Gleixner, Paul Menzel
Cc: linux-kernel, x86, David Woodhouse, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On 19/04/2023 2:50 pm, Andrew Cooper wrote:
> On 19/04/2023 2:43 pm, Thomas Gleixner wrote:
>> On Wed, Apr 19 2023 at 14:38, Thomas Gleixner wrote:
>>> On Wed, Apr 19 2023 at 11:38, Thomas Gleixner wrote:
>>> IOW, the BIOS assignes random numbers to the AP APICs for whatever
>>> raisins, which leaves the parallel startup low level code up a creek
>>> without a paddle, except for actually reading the APICID back from the
>>> APIC. *SHUDDER*
>> So Andrew just pointed out on IRC that this might be related to the
>> ancient issue of the 3-wire APIC bus where IO/APIC and APIC shared the
>> ID space, but that system is definitely post 3-wire APIC :)
> Doesn't mean the BIOS code was updated adequately following that.
>
> What I'm confused by is why this system boots in the first place. I can
> only think that's is a system which only has 4-bit APIC IDs, and happens
> to function when bit 4 gets truncated off the top of the SIPI destination...
https://www.amd.com/system/files/TechDocs/42300_15h_Mod_10h-1Fh_BKDG.pdf
This system does still require the IO-APICs to be at 0, and the LAPICs
to start at some offset, which is clearly 16 in this case. Also, this
system has configurable 4-bit or 8-bit wide APIC IDs, and I can't tell
which mode is active just from the manual.
But, it does mean that the BIOS has genuinely modified the APIC IDs of
the logic processors. This does highlight an error in reasoning with
the parallel bringup code.
For xAPIC, the APIC_ID register is writeable (at least, model
specifically), and CPUID is only the value it would have had at reset.
So the AP bringup logic can't actually use CPUID reliably.
This was changed in x2APIC, which made the x2APIC_ID immutable.
I don't see an option other than the AP bringup code query for xAPIC vs
x2APIC mode, and either looking at the real APIC_ID register, or falling
back to CPUID.
~Andrew
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-19 16:21 ` Andrew Cooper
0 siblings, 0 replies; 236+ messages in thread
From: Andrew Cooper @ 2023-04-19 16:21 UTC (permalink / raw)
To: Thomas Gleixner, Paul Menzel
Cc: linux-kernel, x86, David Woodhouse, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On 19/04/2023 2:50 pm, Andrew Cooper wrote:
> On 19/04/2023 2:43 pm, Thomas Gleixner wrote:
>> On Wed, Apr 19 2023 at 14:38, Thomas Gleixner wrote:
>>> On Wed, Apr 19 2023 at 11:38, Thomas Gleixner wrote:
>>> IOW, the BIOS assignes random numbers to the AP APICs for whatever
>>> raisins, which leaves the parallel startup low level code up a creek
>>> without a paddle, except for actually reading the APICID back from the
>>> APIC. *SHUDDER*
>> So Andrew just pointed out on IRC that this might be related to the
>> ancient issue of the 3-wire APIC bus where IO/APIC and APIC shared the
>> ID space, but that system is definitely post 3-wire APIC :)
> Doesn't mean the BIOS code was updated adequately following that.
>
> What I'm confused by is why this system boots in the first place. I can
> only think that's is a system which only has 4-bit APIC IDs, and happens
> to function when bit 4 gets truncated off the top of the SIPI destination...
https://www.amd.com/system/files/TechDocs/42300_15h_Mod_10h-1Fh_BKDG.pdf
This system does still require the IO-APICs to be at 0, and the LAPICs
to start at some offset, which is clearly 16 in this case. Also, this
system has configurable 4-bit or 8-bit wide APIC IDs, and I can't tell
which mode is active just from the manual.
But, it does mean that the BIOS has genuinely modified the APIC IDs of
the logic processors. This does highlight an error in reasoning with
the parallel bringup code.
For xAPIC, the APIC_ID register is writeable (at least, model
specifically), and CPUID is only the value it would have had at reset.
So the AP bringup logic can't actually use CPUID reliably.
This was changed in x2APIC, which made the x2APIC_ID immutable.
I don't see an option other than the AP bringup code query for xAPIC vs
x2APIC mode, and either looking at the real APIC_ID register, or falling
back to CPUID.
~Andrew
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
2023-04-19 12:38 ` Thomas Gleixner
@ 2023-04-19 16:45 ` Paul Menzel
-1 siblings, 0 replies; 236+ messages in thread
From: Paul Menzel @ 2023-04-19 16:45 UTC (permalink / raw)
To: Thomas Gleixner
Cc: linux-kernel, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
[-- Attachment #1: Type: text/plain, Size: 2408 bytes --]
Dear Thomas,
Am 19.04.23 um 14:38 schrieb Thomas Gleixner:
> On Wed, Apr 19 2023 at 11:38, Thomas Gleixner wrote:
>> On Tue, Apr 18 2023 at 22:10, Paul Menzel wrote:
>>> Am 18.04.23 um 10:40 schrieb Thomas Gleixner:
>>>> Can you please provide the output of cpuid?
>>>
>>> Of course. Here the top, and the whole output is attached.
>>
>> Thanks for the data. Can you please apply the debug patch below and
>> provide the dmesg output? Just the line which is added by the patch is
>> enough. You can boot with cpuhp.parallel=off so you don't have wait for
>> 10 seconds.
>
> Borislav found some a machine which also refuses to boot. It turns of
> the debug patch was spot on:
>
> [ 0.462724] .... node #0, CPUs: #1
> [ 0.462731] smpboot: Kicking AP alive: 17
> [ 0.465723] #2
> [ 0.465732] smpboot: Kicking AP alive: 18
> [ 0.467641] #3
> [ 0.467641] smpboot: Kicking AP alive: 19
>
> So the kernel gets APICID 17, 18, 19 from ACPI but CPUID leaf 0x1
> ebx[31:24], which is the initial APICID has:
>
> CPU1 0x01
> CPU2 0x02
> CPU3 0x03
>
> Which means the APICID to Linux CPU number lookup based on CPUID 0x01
> fails for all of them and stops them dead in the low level startup code.
I am attaching the logs for completeness. Linux is build from your
branch with the debug print on top. The firmware, coreboot based, is
built from [1], but it also happened non-parallel MP init. The code has
better debug prints (attached) though as far as I can see. As Borislav
is able to reproduce this too with some non-coreboot firmware, I assume
it’s unrelated to coreboot.
```
[ 0.259247] smp: Bringing up secondary CPUs ...
[ 0.259446] x86: Booting SMP configuration:
[ 0.259448] .... node #0, CPUs: #1
[ 0.259453] smpboot: Kicking AP alive: 17
[ 10.260918] CPU1 failed to report alive state
[ 10.260998] smp: Brought up 1 node, 1 CPU
[ 10.261000] smpboot: Max logical packages: 2
[ 10.261001] smpboot: Total of 1 processors activated (7801.09 BogoMIPS)
```
> IOW, the BIOS assignes random numbers to the AP APICs for whatever
> raisins, which leaves the parallel startup low level code up a creek
> without a paddle, except for actually reading the APICID back from the
> APIC. *SHUDDER*
>
> I'm leaning towards disabling the CPUID lead 0x01 based discovery and be
> done with it.
Kind regards,
Paul
[1]: https://review.coreboot.org/68169
[-- Attachment #2: kodi-linux-6.3-rc3-smp-tglx.txt --]
[-- Type: text/plain, Size: 57422 bytes --]
[ 0.000000] Linux version 6.3.0-rc3-00045-g64de4df9c80b (root@bf16f3646a84) (gcc (Debian 11.2.0-12) 11.2.0, GNU ld (GNU Binutils for Debian) 2.40) #449 SMP PREEMPT_DYNAMIC Wed Apr 19 16:13:54 UTC 2023
[ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc3-00045-g64de4df9c80b root=/dev/sda3 rw quiet noisapnp cryptomgr.notests ipv6.disable_ipv6=1 selinux=0
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256
[ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format.
[ 0.000000] signal: max sigframe size: 1776
[ 0.000000] BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
[ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005fe3cfff] usable
[ 0.000000] BIOS-e820: [mem 0x000000005fe3d000-0x000000007fffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000f8000000-0x00000000fbffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000fec10000-0x00000000fec10fff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000017effffff] usable
[ 0.000000] NX (Execute Disable) protection: active
[ 0.000000] SMBIOS 3.0.0 present.
[ 0.000000] DMI: ASUS F2A85-M_PRO/F2A85-M_PRO, BIOS 4.18-15-gc782ef4345 04/19/2023
[ 0.000000] tsc: Fast TSC calibration using PIT
[ 0.000000] tsc: Detected 3900.549 MHz processor
[ 0.000756] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
[ 0.000759] e820: remove [mem 0x000a0000-0x000fffff] usable
[ 0.000763] last_pfn = 0x17f000 max_arch_pfn = 0x400000000
[ 0.000768] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT
[ 0.000940] last_pfn = 0x5fe3d max_arch_pfn = 0x400000000
[ 0.004000] Using GB pages for direct mapping
[ 0.004000] ACPI: Early table checksum verification disabled
[ 0.004000] ACPI: RSDP 0x00000000000F6830 000024 (v02 COREv4)
[ 0.004000] ACPI: XSDT 0x000000005FE4A0E0 000074 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: FACP 0x000000005FE4BBC0 000114 (v06 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: DSDT 0x000000005FE4A280 00193A (v02 COREv4 COREBOOT 00010001 INTL 20200925)
[ 0.004000] ACPI: FACS 0x000000005FE4A240 000040
[ 0.004000] ACPI: FACS 0x000000005FE4A240 000040
[ 0.004000] ACPI: SSDT 0x000000005FE4BCE0 00008A (v02 COREv4 COREBOOT 0000002A CORE 20200925)
[ 0.004000] ACPI: MCFG 0x000000005FE4BD70 00003C (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: APIC 0x000000005FE4BDB0 000062 (v03 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: HPET 0x000000005FE4BE20 000038 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: HEST 0x000000005FE4BE60 0001D0 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: IVRS 0x000000005FE4C030 000070 (v02 AMD AMDIOMMU 00000001 AMD 00000000)
[ 0.004000] ACPI: SSDT 0x000000005FE4C0A0 00051F (v02 AMD ALIB 00000001 MSFT 04000000)
[ 0.004000] ACPI: SSDT 0x000000005FE4C5C0 0006B2 (v01 AMD POWERNOW 00000001 AMD 00000001)
[ 0.004000] ACPI: VFCT 0x000000005FE4CC80 00F269 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: Reserving FACP table memory at [mem 0x5fe4bbc0-0x5fe4bcd3]
[ 0.004000] ACPI: Reserving DSDT table memory at [mem 0x5fe4a280-0x5fe4bbb9]
[ 0.004000] ACPI: Reserving FACS table memory at [mem 0x5fe4a240-0x5fe4a27f]
[ 0.004000] ACPI: Reserving FACS table memory at [mem 0x5fe4a240-0x5fe4a27f]
[ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe4bce0-0x5fe4bd69]
[ 0.004000] ACPI: Reserving MCFG table memory at [mem 0x5fe4bd70-0x5fe4bdab]
[ 0.004000] ACPI: Reserving APIC table memory at [mem 0x5fe4bdb0-0x5fe4be11]
[ 0.004000] ACPI: Reserving HPET table memory at [mem 0x5fe4be20-0x5fe4be57]
[ 0.004000] ACPI: Reserving HEST table memory at [mem 0x5fe4be60-0x5fe4c02f]
[ 0.004000] ACPI: Reserving IVRS table memory at [mem 0x5fe4c030-0x5fe4c09f]
[ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe4c0a0-0x5fe4c5be]
[ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe4c5c0-0x5fe4cc71]
[ 0.004000] ACPI: Reserving VFCT table memory at [mem 0x5fe4cc80-0x5fe5bee8]
[ 0.004000] No NUMA configuration found
[ 0.004000] Faking a node at [mem 0x0000000000000000-0x000000017effffff]
[ 0.004000] NODE_DATA(0) allocated [mem 0x17efe9000-0x17effffff]
[ 0.004000] Zone ranges:
[ 0.004000] DMA [mem 0x0000000000001000-0x0000000000ffffff]
[ 0.004000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]
[ 0.004000] Normal [mem 0x0000000100000000-0x000000017effffff]
[ 0.004000] Device empty
[ 0.004000] Movable zone start for each node
[ 0.004000] Early memory node ranges
[ 0.004000] node 0: [mem 0x0000000000001000-0x000000000009efff]
[ 0.004000] node 0: [mem 0x0000000000100000-0x000000005fe3cfff]
[ 0.004000] node 0: [mem 0x0000000100000000-0x000000017effffff]
[ 0.004000] Initmem setup node 0 [mem 0x0000000000001000-0x000000017effffff]
[ 0.004000] On node 0, zone DMA: 1 pages in unavailable ranges
[ 0.004000] On node 0, zone DMA: 97 pages in unavailable ranges
[ 0.004000] On node 0, zone Normal: 451 pages in unavailable ranges
[ 0.004000] On node 0, zone Normal: 4096 pages in unavailable ranges
[ 0.004000] ACPI: PM-Timer IO Port: 0x818
[ 0.004000] ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1])
[ 0.004000] IOAPIC[0]: apic_id 4, version 33, address 0xfec00000, GSI 0-23
[ 0.004000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[ 0.004000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 low level)
[ 0.004000] ACPI: Using ACPI (MADT) for SMP configuration information
[ 0.004000] ACPI: HPET id: 0x10228210 base: 0xfed00000
[ 0.004000] smpboot: Allowing 2 CPUs, 0 hotplug CPUs
[ 0.004000] [mem 0x80000000-0xf7ffffff] available for PCI devices
[ 0.004000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns
[ 0.004000] setup_percpu: NR_CPUS:64 nr_cpumask_bits:2 nr_cpu_ids:2 nr_node_ids:1
[ 0.004000] percpu: Embedded 55 pages/cpu s188392 r8192 d28696 u1048576
[ 0.004000] pcpu-alloc: s188392 r8192 d28696 u1048576 alloc=1*2097152
[ 0.004000] pcpu-alloc: [0] 0 1
[ 0.004000] Fallback order for Node 0: 0
[ 0.004000] Built 1 zonelists, mobility grouping on. Total pages: 898436
[ 0.004000] Policy zone: Normal
[ 0.004000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc3-00045-g64de4df9c80b root=/dev/sda3 rw quiet noisapnp cryptomgr.notests ipv6.disable_ipv6=1 selinux=0
[ 0.004000] Unknown kernel command line parameters "noisapnp BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc3-00045-g64de4df9c80b", will be passed to user space.
[ 0.004000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[ 0.004000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[ 0.004000] mem auto-init: stack:off, heap alloc:off, heap free:off
[ 0.004000] stackdepot: allocating hash table via alloc_large_system_hash
[ 0.004000] stackdepot hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[ 0.004000] software IO TLB: area num 2.
[ 0.004000] Memory: 3477104K/3651436K available (14336K kernel code, 2340K rwdata, 5308K rodata, 2908K init, 11064K bss, 174072K reserved, 0K cma-reserved)
[ 0.004000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
[ 0.004000] ftrace: allocating 38652 entries in 151 pages
[ 0.004000] ftrace: allocated 151 pages with 5 groups
[ 0.004000] Dynamic Preempt: full
[ 0.004000] rcu: Preemptible hierarchical RCU implementation.
[ 0.004000] rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=2.
[ 0.004000] Trampoline variant of Tasks RCU enabled.
[ 0.004000] Rude variant of Tasks RCU enabled.
[ 0.004000] Tracing variant of Tasks RCU enabled.
[ 0.004000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[ 0.004000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2
[ 0.004000] NR_IRQS: 4352, nr_irqs: 440, preallocated irqs: 16
[ 0.004000] rcu: srcu_init: Setting srcu_struct sizes based on contention.
[ 0.004000] spurious 8259A interrupt: IRQ7.
[ 0.004000] Console: colour VGA+ 80x25
[ 0.004000] printk: console [tty0] enabled
[ 0.004000] ACPI: Core revision 20221020
[ 0.004000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 133484873504 ns
[ 0.004000] APIC: Switch to symmetric I/O mode setup
[ 0.004000] AMD-Vi: Using global IVHD EFR:0x0, EFR2:0x0
[ 0.004000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[ 0.004000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x7072c3c0f54, max_idle_ns: 881590752806 ns
[ 0.144910] Calibrating delay loop (skipped), value calculated using timer frequency.. 7801.09 BogoMIPS (lpj=15602196)
[ 0.144913] pid_max: default: 32768 minimum: 301
[ 0.145008] LSM: initializing lsm=capability
[ 0.145103] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[ 0.145120] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[ 0.145515] Last level iTLB entries: 4KB 512, 2MB 1024, 4MB 512
[ 0.145518] Last level dTLB entries: 4KB 1024, 2MB 1024, 4MB 512, 1GB 0
[ 0.145522] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization
[ 0.145525] Spectre V2 : Mitigation: Retpolines
[ 0.145526] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch
[ 0.145526] Spectre V2 : Spectre v2 / SpectreRSB : Filling RSB on VMEXIT
[ 0.145527] Spectre V2 : Enabling Speculation Barrier for firmware calls
[ 0.145528] RETBleed: Mitigation: untrained return thunk
[ 0.145530] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier
[ 0.145532] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl
[ 0.149973] Freeing SMP alternatives memory: 32K
[ 0.258112] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD Graphics (family: 0x15, model: 0x13, stepping: 0x1)
[ 0.258348] cblist_init_generic: Setting adjustable number of callback queues.
[ 0.258350] cblist_init_generic: Setting shift to 1 and lim to 1.
[ 0.258380] cblist_init_generic: Setting shift to 1 and lim to 1.
[ 0.258406] cblist_init_generic: Setting shift to 1 and lim to 1.
[ 0.258433] Performance Events: Fam15h core perfctr, AMD PMU driver.
[ 0.258455] ... version: 0
[ 0.258456] ... bit width: 48
[ 0.258457] ... generic registers: 6
[ 0.258458] ... value mask: 0000ffffffffffff
[ 0.258459] ... max period: 00007fffffffffff
[ 0.258460] ... fixed-purpose events: 0
[ 0.258461] ... event mask: 000000000000003f
[ 0.258581] rcu: Hierarchical SRCU implementation.
[ 0.258582] rcu: Max phase no-delay instances is 1000.
[ 0.259173] NMI watchdog: Enabled. Permanently consumes one hw-PMU counter.
[ 0.259247] smp: Bringing up secondary CPUs ...
[ 0.259446] x86: Booting SMP configuration:
[ 0.259448] .... node #0, CPUs: #1
[ 0.259453] smpboot: Kicking AP alive: 17
[ 10.260918] CPU1 failed to report alive state
[ 10.260998] smp: Brought up 1 node, 1 CPU
[ 10.261000] smpboot: Max logical packages: 2
[ 10.261001] smpboot: Total of 1 processors activated (7801.09 BogoMIPS)
[ 10.261532] devtmpfs: initialized
[ 10.261628] x86/mm: Memory block size: 128MB
[ 10.262718] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[ 10.262726] futex hash table entries: 512 (order: 3, 32768 bytes, linear)
[ 10.262817] pinctrl core: initialized pinctrl subsystem
[ 10.262889] PM: RTC time: 16:25:24, date: 2023-04-19
[ 10.263636] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[ 10.263861] audit: initializing netlink subsys (disabled)
[ 10.264118] thermal_sys: Registered thermal governor 'fair_share'
[ 10.264120] thermal_sys: Registered thermal governor 'bang_bang'
[ 10.264121] thermal_sys: Registered thermal governor 'step_wise'
[ 10.264122] thermal_sys: Registered thermal governor 'user_space'
[ 10.264141] cpuidle: using governor ladder
[ 10.264146] cpuidle: using governor menu
[ 10.264357] PCI: MMCONFIG for domain 0000 [bus 00-3f] at [mem 0xf8000000-0xfbffffff] (base 0xf8000000)
[ 10.264362] PCI: MMCONFIG at [mem 0xf8000000-0xfbffffff] reserved as E820 entry
[ 10.264374] PCI: Using configuration type 1 for base access
[ 10.264574] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.
[ 10.268993] audit: type=2000 audit(1681921524.140:1): state=initialized audit_enabled=0 res=1
[ 10.281009] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages
[ 10.281012] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page
[ 10.281014] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages
[ 10.281015] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page
[ 10.285864] cryptd: max_cpu_qlen set to 1000
[ 10.289188] ACPI: Added _OSI(Module Device)
[ 10.289191] ACPI: Added _OSI(Processor Device)
[ 10.289192] ACPI: Added _OSI(3.0 _SCP Extensions)
[ 10.289193] ACPI: Added _OSI(Processor Aggregator Device)
[ 10.295203] ACPI: 4 ACPI AML tables successfully acquired and loaded
[ 10.296702] ACPI: Interpreter enabled
[ 10.296727] ACPI: PM: (supports S0 S1 S3 S5)
[ 10.296728] ACPI: Using IOAPIC for interrupt routing
[ 10.296779] HEST: Table parsing has been initialized.
[ 10.296800] GHES: Failed to enable APEI firmware first mode.
[ 10.296803] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[ 10.296804] PCI: Ignoring E820 reservations for host bridge windows
[ 10.297084] ACPI: Enabled 8 GPEs in block 00 to 1F
[ 10.302898] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[ 10.302909] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]
[ 10.302987] acpi PNP0A03:00: _OSC: OS now controls [PME AER PCIeCapability LTR]
[ 10.303000] acpi PNP0A03:00: [Firmware Info]: MMCONFIG for domain 0000 [bus 00-3f] only partially covers this bridge
[ 10.303079] acpi PNP0A03:00: host bridge window expanded to [io 0x0000-0x0cf7 window]; [io 0x03b0-0x03df window] ignored
[ 10.303311] PCI host bridge to bus 0000:00
[ 10.303313] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]
[ 10.303316] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]
[ 10.303318] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000dffff]
[ 10.303320] pci_bus 0000:00: root bus resource [mem 0x80000000-0xffffffff]
[ 10.303322] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 10.303345] pci 0000:00:00.0: [1022:1410] type 00 class 0x060000
[ 10.303490] pci 0000:00:00.2: [1022:1419] type 00 class 0x080600
[ 10.303580] pci 0000:00:01.0: [1002:9996] type 00 class 0x030000
[ 10.303588] pci 0000:00:01.0: reg 0x10: [mem 0xe0000000-0xefffffff pref]
[ 10.303593] pci 0000:00:01.0: reg 0x14: [io 0x1000-0x10ff]
[ 10.303598] pci 0000:00:01.0: reg 0x18: [mem 0xf0180000-0xf01bffff]
[ 10.303614] pci 0000:00:01.0: enabling Extended Tags
[ 10.303625] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]
[ 10.303642] pci 0000:00:01.0: supports D1 D2
[ 10.303706] pci 0000:00:01.1: [1002:9902] type 00 class 0x040300
[ 10.303714] pci 0000:00:01.1: reg 0x10: [mem 0xf01c0000-0xf01c3fff]
[ 10.303735] pci 0000:00:01.1: enabling Extended Tags
[ 10.303759] pci 0000:00:01.1: supports D1 D2
[ 10.303842] pci 0000:00:11.0: [1022:7801] type 00 class 0x010601
[ 10.303855] pci 0000:00:11.0: reg 0x10: [io 0x1410-0x1417]
[ 10.303863] pci 0000:00:11.0: reg 0x14: [io 0x1420-0x1423]
[ 10.303870] pci 0000:00:11.0: reg 0x18: [io 0x1418-0x141f]
[ 10.303878] pci 0000:00:11.0: reg 0x1c: [io 0x1424-0x1427]
[ 10.303885] pci 0000:00:11.0: reg 0x20: [io 0x1400-0x140f]
[ 10.303892] pci 0000:00:11.0: reg 0x24: [mem 0xf01cc000-0xf01cc7ff]
[ 10.304038] pci 0000:00:12.0: [1022:7807] type 00 class 0x0c0310
[ 10.304051] pci 0000:00:12.0: reg 0x10: [mem 0xf01c8000-0xf01c8fff]
[ 10.304220] pci 0000:00:12.2: [1022:7808] type 00 class 0x0c0320
[ 10.304234] pci 0000:00:12.2: reg 0x10: [mem 0xf01cd000-0xf01cd0ff]
[ 10.304299] pci 0000:00:12.2: supports D1 D2
[ 10.304300] pci 0000:00:12.2: PME# supported from D0 D1 D2 D3hot
[ 10.304432] pci 0000:00:13.0: [1022:7807] type 00 class 0x0c0310
[ 10.304446] pci 0000:00:13.0: reg 0x10: [mem 0xf01c9000-0xf01c9fff]
[ 10.304614] pci 0000:00:13.2: [1022:7808] type 00 class 0x0c0320
[ 10.304627] pci 0000:00:13.2: reg 0x10: [mem 0xf01ce000-0xf01ce0ff]
[ 10.304692] pci 0000:00:13.2: supports D1 D2
[ 10.304693] pci 0000:00:13.2: PME# supported from D0 D1 D2 D3hot
[ 10.304823] pci 0000:00:14.0: [1022:780b] type 00 class 0x0c0500
[ 10.305554] pci 0000:00:14.2: [1022:780d] type 00 class 0x040300
[ 10.305573] pci 0000:00:14.2: reg 0x10: [mem 0xf01c4000-0xf01c7fff 64bit]
[ 10.305633] pci 0000:00:14.2: PME# supported from D0 D3hot D3cold
[ 10.305769] pci 0000:00:14.3: [1022:780e] type 00 class 0x060100
[ 10.305945] pci 0000:00:14.4: [1022:780f] type 01 class 0x060401
[ 10.306090] pci 0000:00:14.5: [1022:7809] type 00 class 0x0c0310
[ 10.306104] pci 0000:00:14.5: reg 0x10: [mem 0xf01ca000-0xf01cafff]
[ 10.306269] pci 0000:00:15.0: [1022:43a0] type 01 class 0x060400
[ 10.306298] pci 0000:00:15.0: enabling Extended Tags
[ 10.306338] pci 0000:00:15.0: supports D1 D2
[ 10.306499] pci 0000:00:15.1: [1022:43a1] type 01 class 0x060400
[ 10.306530] pci 0000:00:15.1: enabling Extended Tags
[ 10.306569] pci 0000:00:15.1: supports D1 D2
[ 10.306725] pci 0000:00:15.2: [1022:43a2] type 01 class 0x060400
[ 10.306752] pci 0000:00:15.2: enabling Extended Tags
[ 10.306791] pci 0000:00:15.2: supports D1 D2
[ 10.306866] pci 0000:00:16.0: [1022:7807] type 00 class 0x0c0310
[ 10.306879] pci 0000:00:16.0: reg 0x10: [mem 0xf01cb000-0xf01cbfff]
[ 10.307057] pci 0000:00:16.2: [1022:7808] type 00 class 0x0c0320
[ 10.307070] pci 0000:00:16.2: reg 0x10: [mem 0xf01cf000-0xf01cf0ff]
[ 10.307135] pci 0000:00:16.2: supports D1 D2
[ 10.307136] pci 0000:00:16.2: PME# supported from D0 D1 D2 D3hot
[ 10.307278] pci 0000:00:18.0: [1022:1400] type 00 class 0x060000
[ 10.307342] pci 0000:00:18.1: [1022:1401] type 00 class 0x060000
[ 10.307402] pci 0000:00:18.2: [1022:1402] type 00 class 0x060000
[ 10.307467] pci 0000:00:18.3: [1022:1403] type 00 class 0x060000
[ 10.307600] pci 0000:00:18.4: [1022:1404] type 00 class 0x060000
[ 10.307663] pci 0000:00:18.5: [1022:1405] type 00 class 0x060000
[ 10.307738] pci_bus 0000:01: extended config space not accessible
[ 10.307804] pci 0000:00:14.4: PCI bridge to [bus 01] (subtractive decode)
[ 10.307813] pci 0000:00:14.4: bridge window [io 0x0000-0x0cf7 window] (subtractive decode)
[ 10.307816] pci 0000:00:14.4: bridge window [io 0x0d00-0xffff window] (subtractive decode)
[ 10.307818] pci 0000:00:14.4: bridge window [mem 0x000a0000-0x000dffff] (subtractive decode)
[ 10.307820] pci 0000:00:14.4: bridge window [mem 0x80000000-0xffffffff] (subtractive decode)
[ 10.307868] pci 0000:00:15.0: PCI bridge to [bus 02]
[ 10.307953] pci 0000:03:00.0: [1b21:1042] type 00 class 0x0c0330
[ 10.307989] pci 0000:03:00.0: reg 0x10: [mem 0xf0000000-0xf0007fff 64bit]
[ 10.308167] pci 0000:03:00.0: PME# supported from D3hot D3cold
[ 10.308210] pci 0000:03:00.0: 2.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s PCIe x1 link at 0000:00:15.1 (capable of 4.000 Gb/s with 5.0 GT/s PCIe x1 link)
[ 10.317047] pci 0000:00:15.1: PCI bridge to [bus 03]
[ 10.317059] pci 0000:00:15.1: bridge window [mem 0xf0000000-0xf00fffff]
[ 10.317068] pci 0000:00:15.2: bridge configuration invalid ([bus 00-00]), reconfiguring
[ 10.317188] pci 0000:04:00.0: [10ec:8168] type 00 class 0x020000
[ 10.317206] pci 0000:04:00.0: reg 0x10: [io 0x0000-0x00ff]
[ 10.317227] pci 0000:04:00.0: reg 0x18: [mem 0x00000000-0x00000fff 64bit pref]
[ 10.317241] pci 0000:04:00.0: reg 0x20: [mem 0x00000000-0x00003fff 64bit pref]
[ 10.317349] pci 0000:04:00.0: supports D1 D2
[ 10.317351] pci 0000:04:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[ 10.328970] pci 0000:00:15.2: PCI bridge to [bus 04-ff]
[ 10.328981] pci 0000:00:15.2: bridge window [io 0x0000-0x0fff]
[ 10.328985] pci 0000:00:15.2: bridge window [mem 0x00000000-0x000fffff]
[ 10.328989] pci 0000:00:15.2: bridge window [mem 0x00000000-0x000fffff 64bit pref]
[ 10.328993] pci_bus 0000:04: busn_res: [bus 04-ff] end is updated to 04
[ 10.329502] ACPI: PCI: Interrupt link INTA configured for IRQ 0
[ 10.329596] ACPI: PCI: Interrupt link INTB configured for IRQ 0
[ 10.329687] ACPI: PCI: Interrupt link INTC configured for IRQ 0
[ 10.329777] ACPI: PCI: Interrupt link INTD configured for IRQ 0
[ 10.329869] ACPI: PCI: Interrupt link INTE configured for IRQ 0
[ 10.329959] ACPI: PCI: Interrupt link INTF configured for IRQ 0
[ 10.330051] ACPI: PCI: Interrupt link INTG configured for IRQ 0
[ 10.330141] ACPI: PCI: Interrupt link INTH configured for IRQ 0
[ 10.330370] iommu: Default domain type: Translated
[ 10.330372] iommu: DMA domain TLB invalidation policy: lazy mode
[ 10.330551] SCSI subsystem initialized
[ 10.330635] libata version 3.00 loaded.
[ 10.330668] ACPI: bus type USB registered
[ 10.330692] usbcore: registered new interface driver usbfs
[ 10.330702] usbcore: registered new interface driver hub
[ 10.330715] usbcore: registered new device driver usb
[ 10.331044] PCI: Using ACPI for IRQ routing
[ 10.332613] PCI: pci_cache_line_size set to 64 bytes
[ 10.332664] e820: reserve RAM buffer [mem 0x0009fc00-0x0009ffff]
[ 10.332667] e820: reserve RAM buffer [mem 0x5fe3d000-0x5fffffff]
[ 10.332669] e820: reserve RAM buffer [mem 0x17f000000-0x17fffffff]
[ 10.332714] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0
[ 10.332719] hpet0: 3 comparators, 32-bit 14.318180 MHz counter
[ 10.333992] clocksource: Switched to clocksource tsc-early
[ 10.350982] VFS: Disk quotas dquot_6.6.0
[ 10.351012] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[ 10.351123] pnp: PnP ACPI init
[ 10.351416] system 00:00: [mem 0xfec10002-0xfec11001] could not be reserved
[ 10.351719] pnp: PnP ACPI: found 2 devices
[ 10.358870] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
[ 10.359003] NET: Registered PF_INET protocol family
[ 10.359150] IP idents hash table entries: 65536 (order: 7, 524288 bytes, linear)
[ 10.360716] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear)
[ 10.360731] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
[ 10.360739] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear)
[ 10.360805] TCP bind hash table entries: 32768 (order: 8, 1048576 bytes, linear)
[ 10.361256] TCP: Hash tables configured (established 32768 bind 32768)
[ 10.361327] UDP hash table entries: 2048 (order: 4, 65536 bytes, linear)
[ 10.361347] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear)
[ 10.361453] NET: Registered PF_UNIX/PF_LOCAL protocol family
[ 10.361487] pci 0000:00:15.2: BAR 15: assigned [mem 0x80000000-0x800fffff 64bit pref]
[ 10.361492] pci 0000:00:15.2: BAR 13: assigned [io 0x2000-0x2fff]
[ 10.361496] pci 0000:00:14.4: PCI bridge to [bus 01]
[ 10.361507] pci 0000:00:15.0: PCI bridge to [bus 02]
[ 10.361515] pci 0000:00:15.1: PCI bridge to [bus 03]
[ 10.361519] pci 0000:00:15.1: bridge window [mem 0xf0000000-0xf00fffff]
[ 10.361526] pci 0000:04:00.0: BAR 4: assigned [mem 0x80000000-0x80003fff 64bit pref]
[ 10.361539] pci 0000:04:00.0: BAR 2: assigned [mem 0x80004000-0x80004fff 64bit pref]
[ 10.361550] pci 0000:04:00.0: BAR 0: assigned [io 0x2000-0x20ff]
[ 10.361555] pci 0000:00:15.2: PCI bridge to [bus 04]
[ 10.361557] pci 0000:00:15.2: bridge window [io 0x2000-0x2fff]
[ 10.361562] pci 0000:00:15.2: bridge window [mem 0x80000000-0x800fffff 64bit pref]
[ 10.361569] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]
[ 10.361570] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]
[ 10.361572] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000dffff]
[ 10.361574] pci_bus 0000:00: resource 7 [mem 0x80000000-0xffffffff]
[ 10.361575] pci_bus 0000:01: resource 4 [io 0x0000-0x0cf7 window]
[ 10.361577] pci_bus 0000:01: resource 5 [io 0x0d00-0xffff window]
[ 10.361578] pci_bus 0000:01: resource 6 [mem 0x000a0000-0x000dffff]
[ 10.361580] pci_bus 0000:01: resource 7 [mem 0x80000000-0xffffffff]
[ 10.361582] pci_bus 0000:03: resource 1 [mem 0xf0000000-0xf00fffff]
[ 10.361583] pci_bus 0000:04: resource 0 [io 0x2000-0x2fff]
[ 10.361584] pci_bus 0000:04: resource 2 [mem 0x80000000-0x800fffff 64bit pref]
[ 10.361673] pci 0000:00:01.1: D0 power state depends on 0000:00:01.0
[ 10.362350] pci 0000:00:12.2: PME# does not work under D3, disabling it
[ 10.362915] pci 0000:00:13.2: PME# does not work under D3, disabling it
[ 10.363777] pci 0000:00:16.2: PME# does not work under D3, disabling it
[ 10.364048] PCI: CLS 64 bytes, default 64
[ 10.364162] pci 0000:00:00.2: AMD-Vi: Applying erratum 746 workaround
[ 10.364245] pci 0000:00:01.0: Adding to iommu group 0
[ 10.364267] pci 0000:00:01.1: Adding to iommu group 0
[ 10.364290] pci 0000:00:11.0: Adding to iommu group 1
[ 10.364323] pci 0000:00:12.0: Adding to iommu group 2
[ 10.364345] pci 0000:00:12.2: Adding to iommu group 2
[ 10.364377] pci 0000:00:13.0: Adding to iommu group 3
[ 10.364395] pci 0000:00:13.2: Adding to iommu group 3
[ 10.364433] pci 0000:00:14.0: Adding to iommu group 4
[ 10.364450] pci 0000:00:14.2: Adding to iommu group 4
[ 10.364468] pci 0000:00:14.3: Adding to iommu group 4
[ 10.364488] pci 0000:00:14.4: Adding to iommu group 5
[ 10.364512] pci 0000:00:14.5: Adding to iommu group 6
[ 10.364549] pci 0000:00:15.0: Adding to iommu group 7
[ 10.364566] pci 0000:00:15.1: Adding to iommu group 7
[ 10.364585] pci 0000:00:15.2: Adding to iommu group 7
[ 10.364617] pci 0000:00:16.0: Adding to iommu group 8
[ 10.364634] pci 0000:00:16.2: Adding to iommu group 8
[ 10.364690] pci 0000:00:18.0: Adding to iommu group 9
[ 10.364708] pci 0000:00:18.1: Adding to iommu group 9
[ 10.364729] pci 0000:00:18.2: Adding to iommu group 9
[ 10.364746] pci 0000:00:18.3: Adding to iommu group 9
[ 10.364766] pci 0000:00:18.4: Adding to iommu group 9
[ 10.364784] pci 0000:00:18.5: Adding to iommu group 9
[ 10.364792] pci 0000:03:00.0: Adding to iommu group 7
[ 10.364806] pci 0000:04:00.0: Adding to iommu group 7
[ 10.366937] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[ 10.366942] AMD-Vi: Extended features (0x800000853, 0x0): PreF PPR GT IA
[ 10.366948] AMD-Vi: Interrupt remapping enabled
[ 10.367131] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[ 10.367132] software IO TLB: mapped [mem 0x000000005be3d000-0x000000005fe3d000] (64MB)
[ 10.367182] LVT offset 0 assigned for vector 0x400
[ 10.367203] perf: AMD IBS detected (0x000000ff)
[ 10.367211] amd_uncore: 4 amd_nb counters detected
[ 10.368017] workingset: timestamp_bits=37 max_order=20 bucket_order=0
[ 10.368047] zbud: loaded
[ 10.368508] NET: Registered PF_ALG protocol family
[ 10.368512] Key type asymmetric registered
[ 10.368514] Asymmetric key parser 'x509' registered
[ 10.368783] alg: self-tests disabled
[ 10.368876] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 251)
[ 10.369320] io scheduler mq-deadline registered
[ 10.369322] io scheduler kyber registered
[ 10.370447] pcieport 0000:00:15.0: PME: Signaling with IRQ 25
[ 10.370608] pcieport 0000:00:15.1: PME: Signaling with IRQ 26
[ 10.370680] pcieport 0000:00:15.2: enabling device (0000 -> 0003)
[ 10.370883] pcieport 0000:00:15.2: PME: Signaling with IRQ 27
[ 10.371138] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
[ 10.371201] ACPI: button: Power Button [PWRF]
[ 10.371253] ACPI: \_SB_.P000: Found 2 idle states
[ 10.371369] ACPI: \_SB_.P001: Found 2 idle states
[ 10.372255] thermal LNXTHERM:00: registered as thermal_zone0
[ 10.372258] ACPI: thermal: Thermal Zone [TZ00] (0 C)
[ 10.372573] Non-volatile memory driver v1.3
[ 10.372648] AMD-Vi: AMD IOMMUv2 loaded and initialized
[ 10.372667] ACPI: bus type drm_connector registered
[ 10.372799] ahci 0000:00:11.0: version 3.0
[ 10.373109] ahci 0000:00:11.0: AHCI 0001.0300 32 slots 8 ports 6 Gbps 0x40 impl SATA mode
[ 10.373113] ahci 0000:00:11.0: flags: 64bit ncq sntf ilck led clo pio
[ 10.374331] scsi host0: ahci
[ 10.374531] scsi host1: ahci
[ 10.374704] scsi host2: ahci
[ 10.374896] scsi host3: ahci
[ 10.375076] scsi host4: ahci
[ 10.375265] scsi host5: ahci
[ 10.375445] scsi host6: ahci
[ 10.375622] scsi host7: ahci
[ 10.375714] ata1: DUMMY
[ 10.375715] ata2: DUMMY
[ 10.375716] ata3: DUMMY
[ 10.375717] ata4: DUMMY
[ 10.375718] ata5: DUMMY
[ 10.375718] ata6: DUMMY
[ 10.375720] ata7: SATA max UDMA/133 abar m2048@0xf01cc000 port 0xf01cc400 irq 19
[ 10.375722] ata8: DUMMY
[ 10.375982] i8042: PNP: No PS/2 controller found.
[ 10.375983] i8042: Probing ports directly.
[ 10.378529] serio: i8042 KBD port at 0x60,0x64 irq 1
[ 10.378595] serio: i8042 AUX port at 0x60,0x64 irq 12
[ 10.378708] mousedev: PS/2 mouse device common for all mice
[ 10.378760] rtc_cmos 00:01: RTC can wake from S4
[ 10.379002] rtc_cmos 00:01: registered as rtc0
[ 10.379026] rtc_cmos 00:01: setting system clock to 2023-04-19T16:25:24 UTC (1681921524)
[ 10.379067] rtc_cmos 00:01: alarms up to one day, y3k, 114 bytes nvram, hpet irqs
[ 10.379101] device-mapper: uevent: version 1.0.3
[ 10.379167] device-mapper: ioctl: 4.47.0-ioctl (2022-07-28) initialised: dm-devel@redhat.com
[ 10.379323] hid: raw HID events driver (C) Jiri Kosina
[ 10.379363] usbcore: registered new interface driver usbhid
[ 10.379364] usbhid: USB HID core driver
[ 10.379457] Initializing XFRM netlink socket
[ 10.379465] NET: Registered PF_PACKET protocol family
[ 10.379467] x86/pm: family 0x15 cpu detected, MSR saving is needed during suspending.
[ 10.379633] microcode: CPU0: patch_level=0x0600111f
[ 10.379642] microcode: Microcode Update Driver: v2.2.
[ 10.379646] IPI shorthand broadcast: enabled
[ 10.379654] AVX version of gcm_enc/dec engaged.
[ 10.379670] AES CTR mode by8 optimization enabled
[ 10.383650] sched_clock: Marking stable (10264006663, 116905762)->(10383417207, -2504782)
[ 10.383836] registered taskstats version 1
[ 10.384079] zswap: loaded using pool lzo/zbud
[ 10.388276] kmemleak: Kernel memory leak detector initialized (mem pool available: 15679)
[ 10.388282] debug_vm_pgtable: [debug_vm_pgtable ]: Validating architecture page table helpers
[ 10.391487] kmemleak: Automatic memory scanning thread started
[ 10.393329] Key type encrypted registered
[ 10.396095] PM: Magic number: 3:487:439
[ 10.488448] ata7: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[ 10.488603] ata7.00: ATA-9: SanDisk SDSSDP064G, 2.0.0, max UDMA/133
[ 10.488606] ata7.00: 125045424 sectors, multi 1: LBA48 NCQ (depth 32)
[ 10.488805] ata7.00: configured for UDMA/133
[ 10.489003] scsi 6:0:0:0: Direct-Access ATA SanDisk SDSSDP06 0 PQ: 0 ANSI: 5
[ 10.490061] sd 6:0:0:0: [sda] 125045424 512-byte logical blocks: (64.0 GB/59.6 GiB)
[ 10.490078] sd 6:0:0:0: [sda] Write Protect is off
[ 10.490083] sd 6:0:0:0: [sda] Mode Sense: 00 3a 00 00
[ 10.490105] sd 6:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 10.490136] sd 6:0:0:0: [sda] Preferred minimum I/O size 512 bytes
[ 10.491510] sda: sda1 sda2 sda3
[ 10.491950] sd 6:0:0:0: [sda] Attached SCSI disk
[ 10.504047] EXT4-fs (sda3): mounted filesystem fe29e0dc-6303-4401-987c-8472bc1b9516 with ordered data mode. Quota mode: none.
[ 10.504090] VFS: Mounted root (ext4 filesystem) on device 8:3.
[ 10.506047] devtmpfs: mounted
[ 10.510012] Freeing unused kernel image (initmem) memory: 2908K
[ 10.517060] Write protecting the kernel read-only data: 20480k
[ 10.517319] Freeing unused kernel image (rodata/data gap) memory: 836K
[ 10.554385] x86/mm: Checked W+X mappings: passed, no W+X pages found.
[ 10.554391] rodata_test: all tests were successful
[ 10.554413] Run /sbin/init as init process
[ 10.554415] with arguments:
[ 10.554416] /sbin/init
[ 10.554417] noisapnp
[ 10.554418] with environment:
[ 10.554419] HOME=/
[ 10.554420] TERM=linux
[ 10.554420] BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc3-00045-g64de4df9c80b
[ 10.757895] systemd[1]: Inserted module 'autofs4'
[ 10.785886] NET: Registered PF_INET6 protocol family
[ 10.786731] Segment Routing with IPv6
[ 10.786760] In-situ OAM (IOAM) with IPv6
[ 10.812460] systemd[1]: systemd 252.6-1 running in system mode (+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
[ 10.812470] systemd[1]: Detected architecture x86-64.
[ 10.816810] systemd[1]: Hostname set to <kodi>.
[ 11.113206] systemd[1]: Queued start job for default target graphical.target.
[ 11.123550] systemd[1]: Created slice system-getty.slice - Slice /system/getty.
[ 11.124636] systemd[1]: Created slice system-modprobe.slice - Slice /system/modprobe.
[ 11.125478] systemd[1]: Created slice user.slice - User and Session Slice.
[ 11.125653] systemd[1]: Started systemd-ask-password-console.path - Dispatch Password Requests to Console Directory Watch.
[ 11.125780] systemd[1]: Started systemd-ask-password-wall.path - Forward Password Requests to Wall Directory Watch.
[ 11.126204] systemd[1]: Set up automount proc-sys-fs-binfmt_misc.automount - Arbitrary Executable File Formats File System Automount Point.
[ 11.126240] systemd[1]: Reached target cryptsetup.target - Local Encrypted Volumes.
[ 11.126284] systemd[1]: Reached target integritysetup.target - Local Integrity Protected Volumes.
[ 11.126331] systemd[1]: Reached target paths.target - Path Units.
[ 11.126362] systemd[1]: Reached target remote-fs.target - Remote File Systems.
[ 11.126389] systemd[1]: Reached target slices.target - Slice Units.
[ 11.126427] systemd[1]: Reached target swap.target - Swaps.
[ 11.126468] systemd[1]: Reached target veritysetup.target - Local Verity Protected Volumes.
[ 11.128962] systemd[1]: Listening on systemd-coredump.socket - Process Core Dump Socket.
[ 11.129221] systemd[1]: Listening on systemd-fsckd.socket - fsck to fsckd communication Socket.
[ 11.129381] systemd[1]: Listening on systemd-initctl.socket - initctl Compatibility Named Pipe.
[ 11.129690] systemd[1]: Listening on systemd-journald-audit.socket - Journal Audit Socket.
[ 11.129960] systemd[1]: Listening on systemd-journald-dev-log.socket - Journal Socket (/dev/log).
[ 11.130239] systemd[1]: Listening on systemd-journald.socket - Journal Socket.
[ 11.130488] systemd[1]: Listening on systemd-networkd.socket - Network Service Netlink Socket.
[ 11.131308] systemd[1]: Listening on systemd-udevd-control.socket - udev Control Socket.
[ 11.131580] systemd[1]: Listening on systemd-udevd-kernel.socket - udev Kernel Socket.
[ 11.134358] systemd[1]: Mounting dev-hugepages.mount - Huge Pages File System...
[ 11.136807] systemd[1]: Mounting dev-mqueue.mount - POSIX Message Queue File System...
[ 11.140733] systemd[1]: Mounting sys-kernel-debug.mount - Kernel Debug File System...
[ 11.152693] systemd[1]: Mounting sys-kernel-tracing.mount - Kernel Trace File System...
[ 11.159215] systemd[1]: Starting kmod-static-nodes.service - Create List of Static Device Nodes...
[ 11.167969] systemd[1]: Starting modprobe@configfs.service - Load Kernel Module configfs...
[ 11.185265] systemd[1]: Starting modprobe@dm_mod.service - Load Kernel Module dm_mod...
[ 11.187904] systemd[1]: Starting modprobe@drm.service - Load Kernel Module drm...
[ 11.194981] systemd[1]: Starting modprobe@efi_pstore.service - Load Kernel Module efi_pstore...
[ 11.202200] systemd[1]: Starting modprobe@fuse.service - Load Kernel Module fuse...
[ 11.213235] systemd[1]: Starting modprobe@loop.service - Load Kernel Module loop...
[ 11.213303] systemd[1]: systemd-firstboot.service - First Boot Wizard was skipped because of an unmet condition check (ConditionFirstBoot=yes).
[ 11.213372] systemd[1]: systemd-fsck-root.service - File System Check on Root Device was skipped because of an unmet condition check (ConditionPathIsReadWrite=!/).
[ 11.213409] systemd[1]: Reached target local-fs.target - Local File Systems.
[ 11.213471] systemd[1]: apparmor.service - Load AppArmor profiles was skipped because of an unmet condition check (ConditionSecurity=apparmor).
[ 11.217387] systemd[1]: Starting systemd-binfmt.service - Set Up Additional Binary Formats...
[ 11.225611] loop: module loaded
[ 11.227933] fuse: init (API version 7.38)
[ 11.241394] systemd[1]: Starting systemd-journald.service - Journal Service...
[ 11.244008] systemd[1]: Starting systemd-random-seed.service - Load/Save Random Seed...
[ 11.265213] systemd[1]: Starting systemd-sysctl.service - Apply Kernel Variables...
[ 11.267873] systemd[1]: Starting systemd-sysusers.service - Create System Users...
[ 11.294059] systemd[1]: Starting systemd-udev-trigger.service - Coldplug All udev Devices...
[ 11.316096] systemd[1]: Mounted dev-hugepages.mount - Huge Pages File System.
[ 11.316293] systemd[1]: Mounted dev-mqueue.mount - POSIX Message Queue File System.
[ 11.316467] systemd[1]: Mounted sys-kernel-debug.mount - Kernel Debug File System.
[ 11.316640] systemd[1]: Mounted sys-kernel-tracing.mount - Kernel Trace File System.
[ 11.337994] systemd[1]: Finished kmod-static-nodes.service - Create List of Static Device Nodes.
[ 11.338790] systemd[1]: modprobe@configfs.service: Deactivated successfully.
[ 11.341622] systemd[1]: Finished modprobe@configfs.service - Load Kernel Module configfs.
[ 11.342285] systemd[1]: modprobe@dm_mod.service: Deactivated successfully.
[ 11.342546] systemd[1]: Finished modprobe@dm_mod.service - Load Kernel Module dm_mod.
[ 11.343122] systemd[1]: modprobe@drm.service: Deactivated successfully.
[ 11.343364] systemd[1]: Finished modprobe@drm.service - Load Kernel Module drm.
[ 11.343928] systemd[1]: modprobe@efi_pstore.service: Deactivated successfully.
[ 11.344176] systemd[1]: Finished modprobe@efi_pstore.service - Load Kernel Module efi_pstore.
[ 11.344734] systemd[1]: modprobe@fuse.service: Deactivated successfully.
[ 11.358905] systemd[1]: Finished modprobe@fuse.service - Load Kernel Module fuse.
[ 11.359616] systemd[1]: modprobe@loop.service: Deactivated successfully.
[ 11.365358] systemd[1]: Finished modprobe@loop.service - Load Kernel Module loop.
[ 11.366685] systemd[1]: Finished systemd-sysctl.service - Apply Kernel Variables.
[ 11.367634] systemd[1]: Finished systemd-sysusers.service - Create System Users.
[ 11.368150] systemd[1]: proc-sys-fs-binfmt_misc.automount: Got automount request for /proc/sys/fs/binfmt_misc, triggered by 133 (systemd-binfmt)
[ 11.382441] systemd[1]: Mounting proc-sys-fs-binfmt_misc.mount - Arbitrary Executable File Formats File System...
[ 11.385250] tsc: Refined TSC clocksource calibration: 3900.225 MHz
[ 11.385258] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x70705d7a9ae, max_idle_ns: 881590495532 ns
[ 11.385270] clocksource: Switched to clocksource tsc
[ 11.429139] systemd[1]: Mounting sys-fs-fuse-connections.mount - FUSE Control File System...
[ 11.442192] systemd[1]: Mounting sys-kernel-config.mount - Kernel Configuration File System...
[ 11.442320] systemd[1]: systemd-pstore.service - Platform Persistent Storage Archival was skipped because of an unmet condition check (ConditionDirectoryNotEmpty=/sys/fs/pstore).
[ 11.442474] systemd[1]: systemd-repart.service - Repartition Root Disk was skipped because no trigger condition checks were met.
[ 11.465930] systemd[1]: Starting systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev...
[ 11.467355] systemd[1]: Mounted proc-sys-fs-binfmt_misc.mount - Arbitrary Executable File Formats File System.
[ 11.485939] systemd[1]: Finished systemd-binfmt.service - Set Up Additional Binary Formats.
[ 11.487783] systemd[1]: Mounted sys-fs-fuse-connections.mount - FUSE Control File System.
[ 11.487969] systemd[1]: Mounted sys-kernel-config.mount - Kernel Configuration File System.
[ 11.554727] systemd[1]: Finished systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev.
[ 11.565290] systemd[1]: Starting systemd-udevd.service - Rule-based Manager for Device Events and Files...
[ 11.570809] systemd[1]: Started systemd-journald.service - Journal Service.
[ 11.617282] systemd-journald[134]: Received client request to flush runtime journal.
[ 12.118033] sd 6:0:0:0: Attached scsi generic sg0 type 0
[ 12.224922] random: crng init done
[ 12.412498] acpi_cpufreq: overriding BIOS provided _PSD data
[ 12.561374] QUIRK: Enable AMD PLL fix
[ 12.561422] ehci-pci 0000:00:12.2: EHCI Host Controller
[ 12.561449] ehci-pci 0000:00:12.2: new USB bus registered, assigned bus number 1
[ 12.561461] ehci-pci 0000:00:12.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
[ 12.561469] ehci-pci 0000:00:12.2: debug port 1
[ 12.561640] ehci-pci 0000:00:12.2: irq 17, io mem 0xf01cd000
[ 12.576943] ehci-pci 0000:00:12.2: USB 2.0 started, EHCI 1.00
[ 12.577230] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 12.577233] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.577235] usb usb1: Product: EHCI Host Controller
[ 12.577236] usb usb1: Manufacturer: Linux 6.3.0-rc3-00045-g64de4df9c80b ehci_hcd
[ 12.577238] usb usb1: SerialNumber: 0000:00:12.2
[ 12.577680] hub 1-0:1.0: USB hub found
[ 12.577708] hub 1-0:1.0: 5 ports detected
[ 12.578363] ehci-pci 0000:00:13.2: EHCI Host Controller
[ 12.578382] ehci-pci 0000:00:13.2: new USB bus registered, assigned bus number 2
[ 12.578393] ehci-pci 0000:00:13.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
[ 12.578402] ehci-pci 0000:00:13.2: debug port 1
[ 12.578529] ehci-pci 0000:00:13.2: irq 17, io mem 0xf01ce000
[ 12.587170] piix4_smbus 0000:00:14.0: SMBus Host Controller at 0xb00, revision 0
[ 12.587177] piix4_smbus 0000:00:14.0: Using register 0x2e for SMBus port selection
[ 12.587654] piix4_smbus 0000:00:14.0: Auxiliary SMBus Host Controller at 0xb20
[ 12.592939] ehci-pci 0000:00:13.2: USB 2.0 started, EHCI 1.00
[ 12.593429] usb usb2: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 12.593433] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.593435] usb usb2: Product: EHCI Host Controller
[ 12.593437] usb usb2: Manufacturer: Linux 6.3.0-rc3-00045-g64de4df9c80b ehci_hcd
[ 12.593438] usb usb2: SerialNumber: 0000:00:13.2
[ 12.593929] hub 2-0:1.0: USB hub found
[ 12.593956] hub 2-0:1.0: 5 ports detected
[ 12.594622] ehci-pci 0000:00:16.2: EHCI Host Controller
[ 12.594640] ehci-pci 0000:00:16.2: new USB bus registered, assigned bus number 3
[ 12.594650] ehci-pci 0000:00:16.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
[ 12.594659] ehci-pci 0000:00:16.2: debug port 1
[ 12.594791] ehci-pci 0000:00:16.2: irq 17, io mem 0xf01cf000
[ 12.608936] ehci-pci 0000:00:16.2: USB 2.0 started, EHCI 1.00
[ 12.609220] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 12.609223] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.609225] usb usb3: Product: EHCI Host Controller
[ 12.609226] usb usb3: Manufacturer: Linux 6.3.0-rc3-00045-g64de4df9c80b ehci_hcd
[ 12.609228] usb usb3: SerialNumber: 0000:00:16.2
[ 12.609667] hub 3-0:1.0: USB hub found
[ 12.609694] hub 3-0:1.0: 4 ports detected
[ 12.611387] ohci-pci 0000:00:12.0: OHCI PCI host controller
[ 12.611420] ohci-pci 0000:00:12.0: new USB bus registered, assigned bus number 4
[ 12.611596] ohci-pci 0000:00:12.0: irq 18, io mem 0xf01c8000
[ 12.660463] r8169 0000:04:00.0: enabling device (0000 -> 0003)
[ 12.690367] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 12.690374] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.690376] usb usb4: Product: OHCI PCI host controller
[ 12.690378] usb usb4: Manufacturer: Linux 6.3.0-rc3-00045-g64de4df9c80b ohci_hcd
[ 12.690380] usb usb4: SerialNumber: 0000:00:12.0
[ 12.699141] hub 4-0:1.0: USB hub found
[ 12.703266] hub 4-0:1.0: 5 ports detected
[ 12.725731] ohci-pci 0000:00:13.0: OHCI PCI host controller
[ 12.725756] ohci-pci 0000:00:13.0: new USB bus registered, assigned bus number 5
[ 12.725865] ohci-pci 0000:00:13.0: irq 18, io mem 0xf01c9000
[ 12.727540] r8169 0000:04:00.0 eth0: RTL8168f/8111f, 08:60:6e:74:7a:51, XID 480, IRQ 28
[ 12.727545] r8169 0000:04:00.0 eth0: jumbo features [frames: 9194 bytes, tx checksumming: ko]
[ 12.741582] xhci_hcd 0000:03:00.0: xHCI Host Controller
[ 12.741607] xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 6
[ 12.748200] snd_hda_intel 0000:00:01.1: Force to non-snoop mode
[ 12.802855] xhci_hcd 0000:03:00.0: hcc params 0x0200f180 hci version 0x96 quirks 0x0000000000080010
[ 12.803664] xhci_hcd 0000:03:00.0: xHCI Host Controller
[ 12.803679] xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 7
[ 12.803692] xhci_hcd 0000:03:00.0: Host supports USB 3.0 SuperSpeed
[ 12.803860] usb usb6: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 12.803863] usb usb6: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.803865] usb usb6: Product: xHCI Host Controller
[ 12.803867] usb usb6: Manufacturer: Linux 6.3.0-rc3-00045-g64de4df9c80b xhci-hcd
[ 12.803868] usb usb6: SerialNumber: 0000:03:00.0
[ 12.807644] hub 6-0:1.0: USB hub found
[ 12.807677] hub 6-0:1.0: 2 ports detected
[ 12.809751] usb usb7: We don't know the algorithms for LPM for this host, disabling LPM.
[ 12.809885] usb usb7: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 6.03
[ 12.809888] usb usb7: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.809890] usb usb7: Product: xHCI Host Controller
[ 12.809891] usb usb7: Manufacturer: Linux 6.3.0-rc3-00045-g64de4df9c80b xhci-hcd
[ 12.809893] usb usb7: SerialNumber: 0000:03:00.0
[ 12.810353] hub 7-0:1.0: USB hub found
[ 12.810382] hub 7-0:1.0: 2 ports detected
[ 12.812712] input: HDA ATI HDMI HDMI/DP,pcm=3 as /devices/pci0000:00/0000:00:01.1/sound/card0/input1
[ 12.813027] input: HDA ATI HDMI HDMI/DP,pcm=7 as /devices/pci0000:00/0000:00:01.1/sound/card0/input2
[ 12.825660] snd_hda_codec_realtek hdaudioC1D0: ALC892: SKU not ready 0x00000100
[ 12.826527] snd_hda_codec_realtek hdaudioC1D0: autoconfig for ALC892: line_outs=4 (0x14/0x16/0x15/0x17/0x0) type:line
[ 12.826534] snd_hda_codec_realtek hdaudioC1D0: speaker_outs=0 (0x0/0x0/0x0/0x0/0x0)
[ 12.826536] snd_hda_codec_realtek hdaudioC1D0: hp_outs=1 (0x1b/0x0/0x0/0x0/0x0)
[ 12.826539] snd_hda_codec_realtek hdaudioC1D0: mono: mono_out=0x0
[ 12.826540] snd_hda_codec_realtek hdaudioC1D0: dig-out=0x1e/0x0
[ 12.826541] snd_hda_codec_realtek hdaudioC1D0: inputs:
[ 12.826543] snd_hda_codec_realtek hdaudioC1D0: Rear Mic=0x18
[ 12.826545] snd_hda_codec_realtek hdaudioC1D0: Front Mic=0x19
[ 12.826546] snd_hda_codec_realtek hdaudioC1D0: Line=0x1a
[ 12.826547] snd_hda_codec_realtek hdaudioC1D0: CD=0x1c
[ 12.854744] input: HD-Audio Generic Rear Mic as /devices/pci0000:00/0000:00:14.2/sound/card1/input3
[ 12.855037] input: HD-Audio Generic Front Mic as /devices/pci0000:00/0000:00:14.2/sound/card1/input4
[ 12.855316] input: HD-Audio Generic Line as /devices/pci0000:00/0000:00:14.2/sound/card1/input5
[ 12.855602] input: HD-Audio Generic Line Out Front as /devices/pci0000:00/0000:00:14.2/sound/card1/input6
[ 12.855878] input: HD-Audio Generic Line Out Surround as /devices/pci0000:00/0000:00:14.2/sound/card1/input7
[ 12.856162] input: HD-Audio Generic Line Out CLFE as /devices/pci0000:00/0000:00:14.2/sound/card1/input8
[ 12.856433] input: HD-Audio Generic Line Out Side as /devices/pci0000:00/0000:00:14.2/sound/card1/input9
[ 12.856704] input: HD-Audio Generic Front Headphone as /devices/pci0000:00/0000:00:14.2/sound/card1/input10
[ 12.861030] usb usb5: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 12.861037] usb usb5: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.861039] usb usb5: Product: OHCI PCI host controller
[ 12.861041] usb usb5: Manufacturer: Linux 6.3.0-rc3-00045-g64de4df9c80b ohci_hcd
[ 12.861043] usb usb5: SerialNumber: 0000:00:13.0
[ 12.867101] hub 5-0:1.0: USB hub found
[ 12.874104] hub 5-0:1.0: 5 ports detected
[ 12.902021] ohci-pci 0000:00:14.5: OHCI PCI host controller
[ 12.902052] ohci-pci 0000:00:14.5: new USB bus registered, assigned bus number 8
[ 12.902175] ohci-pci 0000:00:14.5: irq 18, io mem 0xf01ca000
[ 12.914899] r8169 0000:04:00.0 enp4s0: renamed from eth0
[ 12.981592] usb usb8: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 12.981599] usb usb8: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.981601] usb usb8: Product: OHCI PCI host controller
[ 12.981603] usb usb8: Manufacturer: Linux 6.3.0-rc3-00045-g64de4df9c80b ohci_hcd
[ 12.981604] usb usb8: SerialNumber: 0000:00:14.5
[ 12.983308] hub 8-0:1.0: USB hub found
[ 12.998172] hub 8-0:1.0: 2 ports detected
[ 13.004457] ohci-pci 0000:00:16.0: OHCI PCI host controller
[ 13.004481] ohci-pci 0000:00:16.0: new USB bus registered, assigned bus number 9
[ 13.004586] ohci-pci 0000:00:16.0: irq 18, io mem 0xf01cb000
[ 13.028929] usb 4-1: new low-speed USB device number 2 using ohci-pci
[ 13.117515] usb usb9: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 13.117522] usb usb9: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 13.117524] usb usb9: Product: OHCI PCI host controller
[ 13.117526] usb usb9: Manufacturer: Linux 6.3.0-rc3-00045-g64de4df9c80b ohci_hcd
[ 13.117527] usb usb9: SerialNumber: 0000:00:16.0
[ 13.122823] hub 9-0:1.0: USB hub found
[ 13.122855] hub 9-0:1.0: 4 ports detected
[ 13.150187] r8169 0000:04:00.0: Direct firmware load for rtl_nic/rtl8168f-1.fw failed with error -2
[ 13.150198] r8169 0000:04:00.0: Unable to load firmware rtl_nic/rtl8168f-1.fw (-2)
[ 13.150688] RTL8211E Gigabit Ethernet r8169-0-400:00: attached PHY driver (mii_bus:phy_addr=r8169-0-400:00, irq=MAC)
[ 13.231387] r8169 0000:04:00.0 enp4s0: Link is Down
[ 13.256051] [drm] radeon kernel modesetting enabled.
[ 13.262109] [drm] initializing kernel modesetting (ARUBA 0x1002:0x9996 0x1002:0x9996 0x00).
[ 13.262177] ATOM BIOS: 113
[ 13.262284] radeon 0000:00:01.0: VRAM: 512M 0x0000000000000000 - 0x000000001FFFFFFF (512M used)
[ 13.262288] radeon 0000:00:01.0: GTT: 1024M 0x0000000020000000 - 0x000000005FFFFFFF
[ 13.262296] [drm] Detected VRAM RAM=512M, BAR=256M
[ 13.262297] [drm] RAM width 64bits DDR
[ 13.262484] [drm] radeon: 512M of VRAM memory ready
[ 13.262490] [drm] radeon: 1024M of GTT memory ready.
[ 13.262530] [drm] Loading ARUBA Microcode
[ 13.270667] [drm] Internal thermal controller without fan control
[ 13.271087] [drm] radeon: dpm initialized
[ 13.275714] [drm] Found VCE firmware/feedback version 50.0.1 / 17!
[ 13.275771] [drm] GART: num cpu pages 262144, num gpu pages 262144
[ 13.279095] usb 4-1: New USB device found, idVendor=413c, idProduct=2106, bcdDevice= 1.01
[ 13.279101] usb 4-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 13.279103] usb 4-1: Product: Dell QuietKey Keyboard
[ 13.279105] usb 4-1: Manufacturer: DELL
[ 13.286833] input: DELL Dell QuietKey Keyboard as /devices/pci0000:00/0000:00:12.0/usb4/4-1/4-1:1.0/0003:413C:2106.0001/input/input11
[ 13.317348] [drm] PCIE GART of 1024M enabled (table at 0x00000000001D6000).
[ 13.317596] radeon 0000:00:01.0: WB enabled
[ 13.317599] radeon 0000:00:01.0: fence driver on ring 0 use gpu addr 0x0000000020000c00
[ 13.317977] radeon 0000:00:01.0: fence driver on ring 5 use gpu addr 0x0000000000075a18
[ 13.338850] radeon 0000:00:01.0: fence driver on ring 6 use gpu addr 0x0000000020000c18
[ 13.338854] radeon 0000:00:01.0: fence driver on ring 7 use gpu addr 0x0000000020000c1c
[ 13.338856] radeon 0000:00:01.0: fence driver on ring 1 use gpu addr 0x0000000020000c04
[ 13.338858] radeon 0000:00:01.0: fence driver on ring 2 use gpu addr 0x0000000020000c08
[ 13.338860] radeon 0000:00:01.0: fence driver on ring 3 use gpu addr 0x0000000020000c0c
[ 13.338861] radeon 0000:00:01.0: fence driver on ring 4 use gpu addr 0x0000000020000c10
[ 13.339153] radeon 0000:00:01.0: radeon: MSI limited to 32-bit
[ 13.339342] radeon 0000:00:01.0: radeon: using MSI.
[ 13.339414] [drm] radeon: irq initialized.
[ 13.345571] hid-generic 0003:413C:2106.0001: input,hidraw0: USB HID v1.10 Keyboard [DELL Dell QuietKey Keyboard] on usb-0000:00:12.0-1/input0
[ 13.390412] [drm] ring test on 0 succeeded in 3 usecs
[ 13.390423] [drm] ring test on 3 succeeded in 4 usecs
[ 13.390431] [drm] ring test on 4 succeeded in 4 usecs
[ 13.441915] [drm] ring test on 5 succeeded in 2 usecs
[ 13.461858] [drm] UVD initialized successfully.
[ 13.571355] [drm] ring test on 6 succeeded in 18 usecs
[ 13.571367] [drm] ring test on 7 succeeded in 4 usecs
[ 13.571368] [drm] VCE initialized successfully.
[ 13.571494] snd_hda_intel 0000:00:01.1: bound 0000:00:01.0 (ops radeon_audio_component_bind_ops [radeon])
[ 13.571667] [drm] ib test on ring 0 succeeded in 0 usecs
[ 13.571721] [drm] ib test on ring 3 succeeded in 0 usecs
[ 13.571771] [drm] ib test on ring 4 succeeded in 0 usecs
[ 13.768993] usb 4-2: new low-speed USB device number 3 using ohci-pci
[ 13.965196] usb 4-2: New USB device found, idVendor=046d, idProduct=c016, bcdDevice= 3.40
[ 13.965208] usb 4-2: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 13.965212] usb 4-2: Product: Optical USB Mouse
[ 13.965216] usb 4-2: Manufacturer: Logitech
[ 13.976218] input: Logitech Optical USB Mouse as /devices/pci0000:00/0000:00:12.0/usb4/4-2/4-2:1.0/0003:046D:C016.0002/input/input12
[ 13.977837] hid-generic 0003:046D:C016.0002: input,hidraw1: USB HID v1.10 Mouse [Logitech Optical USB Mouse] on usb-0000:00:12.0-2/input0
[ 14.105109] [drm] ib test on ring 5 succeeded
[ 14.649036] [drm] ib test on ring 6 succeeded
[ 15.161027] [drm] ib test on ring 7 succeeded
[ 15.173369] [drm] Radeon Display Connectors
[ 15.173376] [drm] Connector 0:
[ 15.173378] [drm] DP-1
[ 15.173380] [drm] HPD1
[ 15.173382] [drm] DDC: 0x6530 0x6530 0x6534 0x6534 0x6538 0x6538 0x653c 0x653c
[ 15.173387] [drm] Encoders:
[ 15.173388] [drm] DFP1: INTERNAL_UNIPHY2
[ 15.173390] [drm] Connector 1:
[ 15.173392] [drm] VGA-1
[ 15.173393] [drm] HPD2
[ 15.173395] [drm] DDC: 0x6540 0x6540 0x6544 0x6544 0x6548 0x6548 0x654c 0x654c
[ 15.173399] [drm] Encoders:
[ 15.173400] [drm] CRT1: INTERNAL_UNIPHY2
[ 15.173402] [drm] CRT1: NUTMEG
[ 15.173403] [drm] Connector 2:
[ 15.173405] [drm] HDMI-A-1
[ 15.173406] [drm] HPD3
[ 15.173408] [drm] DDC: 0x6550 0x6550 0x6554 0x6554 0x6558 0x6558 0x655c 0x655c
[ 15.173411] [drm] Encoders:
[ 15.173412] [drm] DFP2: INTERNAL_UNIPHY
[ 15.447734] [drm] fb mappable at 0xE03E9000
[ 15.447741] [drm] vram apper at 0xE0000000
[ 15.447743] [drm] size 5242880
[ 15.447745] [drm] fb depth is 24
[ 15.447747] [drm] pitch is 5120
[ 15.448342] fbcon: radeondrmfb (fb0) is primary device
[ 15.631670] Console: switching to colour frame buffer device 160x64
[ 15.633551] radeon 0000:00:01.0: [drm] fb0: radeondrmfb frame buffer device
[ 15.645230] [drm] Initialized radeon 2.50.0 20080528 for 0000:00:01.0 on minor 0
[ 15.872804] r8169 0000:04:00.0 enp4s0: Link is Up - 1Gbps/Full - flow control rx/tx
[ 15.872819] IPv6: ADDRCONF(NETDEV_CHANGE): enp4s0: link becomes ready
[ 18.185928] memfd_create() without MFD_EXEC nor MFD_NOEXEC_SEAL, pid=246 'systemd'
[ 18.983836] [drm] amdgpu kernel modesetting enabled.
[-- Attachment #3: 20230419-coreboot-cbmem-log-cb-68169.txt --]
[-- Type: text/plain, Size: 32882 bytes --]
[NOTE ] coreboot-4.18-15-gc782ef4345 Wed Apr 19 16:04:54 UTC 2023 bootblock starting (log level: 7)...
[DEBUG] FMAP: Found "FLASH" version 1.1 at 0x10000.
[DEBUG] FMAP: base = 0xffc00000 size = 0x400000 #areas = 4
[DEBUG] FMAP: area COREBOOT found @ 10200 (4128256 bytes)
[INFO ] CBFS: mcache @0x00034e00 built for 15 files, used 0x330 of 0x4000 bytes
[INFO ] CBFS: Found 'fallback/romstage' @0x20e00 size 0x4cf90 in mcache @0x00034fc0
[DEBUG] BS: bootblock times (exec / console): total (unknown) / 1 ms
[NOTE ] coreboot-4.18-15-gc782ef4345 Wed Apr 19 16:04:54 UTC 2023 romstage starting (log level: 7)...
[DEBUG] APIC 00: CPU Family_Model = 00610f31
[DEBUG] APIC 00: ** Enter AmdInitReset [00020007]
[DEBUG] Fch OEM config in INIT RESET
[DEBUG] AmdInitReset() returned AGESA_SUCCESS
[DEBUG] APIC 00: Heap in LocalCache (2) at 0x00400000
[DEBUG] APIC 00: ** Exit AmdInitReset [00020007]
[DEBUG] APIC 00: ** Enter AmdInitEarly [00020002]
[DEBUG] AmdInitEarly() returned AGESA_SUCCESS
[DEBUG] APIC 00: Heap in LocalCache (2) at 0x00400000
[DEBUG] APIC 00: ** Exit AmdInitEarly [00020002]
[DEBUG] APIC 00: ** Enter AmdInitPost [00020006]
[ERROR] -------------SPD READ ERROR-----------
[ERROR] -------------SPD READ ERROR-----------
[DEBUG] AmdInitPost() returned AGESA_SUCCESS
[DEBUG] APIC 00: Heap in TempMem (3) at 0x000b0000
[DEBUG] APIC 00: ** Exit AmdInitPost [00020006]
[DEBUG] CBMEM:
[DEBUG] IMD: root @ 0x5ffff000 254 entries.
[DEBUG] IMD: root @ 0x5fffec00 62 entries.
[DEBUG] FMAP: area COREBOOT found @ 10200 (4128256 bytes)
[DEBUG] Normal boot
[INFO ] CBFS: Found 'fallback/postcar' @0x2340 size 0x51a4 in mcache @0x00034ee8
[DEBUG] Loading module at 0x5ffd0000 with entry 0x5ffd0031. filesize: 0x4ee0 memsize: 0xb1f0
[DEBUG] Processing 161 relocs. Offset value of 0x5dfd0000
[DEBUG] BS: romstage times (exec / console): total (unknown) / 2 ms
[NOTE ] coreboot-4.18-15-gc782ef4345 Wed Apr 19 16:04:54 UTC 2023 postcar starting (log level: 7)...
[DEBUG] FMAP: area COREBOOT found @ 10200 (4128256 bytes)
[INFO ] CBFS: Found 'fallback/ramstage' @0x6de40 size 0x217fe in mcache @0x5ffdd240
[DEBUG] Loading module at 0x5feb8000 with entry 0x5feb8000. filesize: 0x46038 memsize: 0x116828
[DEBUG] Processing 3978 relocs. Offset value of 0x5beb8000
[DEBUG] BS: postcar times (exec / console): total (unknown) / 0 ms
[NOTE ] coreboot-4.18-15-gc782ef4345 Wed Apr 19 16:04:54 UTC 2023 ramstage starting (log level: 7)...
[DEBUG] Normal boot
[DEBUG] APIC 00: ** Enter AmdInitEnv [00020003]
[DEBUG] Wiped HEAP at [10000000 - 1002ffff]
[DEBUG] Fch OEM config in INIT ENV
[DEBUG] AmdInitEnv() returned AGESA_SUCCESS
[DEBUG] APIC 00: Heap in SystemMem (4) at 0x10000014
[DEBUG] APIC 00: ** Exit AmdInitEnv [00020003]
[DEBUG] BS: BS_PRE_DEVICE entry times (exec / console): 25 / 0 ms
[INFO ] Enumerating buses...
[DEBUG] Root Device scanning...
[DEBUG] CPU_CLUSTER: 0 enabled
[DEBUG] DOMAIN: 0000 enabled
[DEBUG] DOMAIN: 0000 scanning...
[DEBUG] PCI: pci_scan_bus for bus 00
[DEBUG] PCI: 00:00.0 [1022/1410] enabled
[DEBUG] PCI: 00:00.2 [1022/1419] enabled
[DEBUG] PCI: 00:01.0 [1002/9996] enabled
[DEBUG] PCI: 00:01.1 [1002/9902] enabled
[INFO ] PCI: Static device PCI: 00:02.0 not found, disabling it.
[DEBUG] hudson_enable()
[INFO ] PCI: Static device PCI: 00:10.0 not found, disabling it.
[DEBUG] hudson_enable()
[INFO ] PCI: Static device PCI: 00:10.1 not found, disabling it.
[DEBUG] hudson_enable()
[DEBUG] PCI: 00:11.0 [1022/7801] enabled
[DEBUG] hudson_enable()
[DEBUG] PCI: 00:12.0 [1022/7807] enabled
[DEBUG] hudson_enable()
[DEBUG] PCI: 00:12.2 [1022/7808] enabled
[DEBUG] hudson_enable()
[DEBUG] PCI: 00:13.0 [1022/7807] enabled
[DEBUG] hudson_enable()
[DEBUG] PCI: 00:13.2 [1022/7808] enabled
[DEBUG] hudson_enable()
[DEBUG] PCI: 00:14.0 [1022/780b] enabled
[DEBUG] hudson_enable()
[DEBUG] hudson_enable()
[DEBUG] PCI: 00:14.2 [1022/780d] enabled
[DEBUG] hudson_enable()
[DEBUG] PCI: 00:14.3 [1022/780e] enabled
[DEBUG] hudson_enable()
[DEBUG] PCI: 00:14.4 [1022/780f] enabled
[DEBUG] PCI: 00:14.5 [1022/7809] enabled
[DEBUG] hudson_enable()
[DEBUG] hudson_enable()
[DEBUG] PCI: 00:15.0 [1022/43a0] enabled
[DEBUG] hudson_enable()
[DEBUG] PCI: 00:15.1 [1022/43a1] enabled
[DEBUG] hudson_enable()
[DEBUG] PCI: 00:15.2 [1022/43a2] disabled
[DEBUG] PCI: 00:16.0 [1022/7807] enabled
[DEBUG] PCI: 00:16.2 [1022/7808] enabled
[DEBUG] PCI: 00:18.0 [1022/1400] enabled
[DEBUG] PCI: 00:18.1 [1022/1401] enabled
[DEBUG] PCI: 00:18.2 [1022/1402] enabled
[DEBUG] PCI: 00:18.3 [1022/1403] enabled
[DEBUG] PCI: 00:18.4 [1022/1404] enabled
[DEBUG] PCI: 00:18.5 [1022/1405] enabled
[WARN ] PCI: Leftover static devices:
[WARN ] PCI: 00:02.0
[WARN ] PCI: 00:10.0
[WARN ] PCI: 00:10.1
[WARN ] PCI: 00:14.1
[WARN ] PCI: 00:14.7
[WARN ] PCI: Check your devicetree.cb.
[DEBUG] PCI: 00:14.0 scanning...
[DEBUG] scan_bus: bus PCI: 00:14.0 finished in 0 msecs
[DEBUG] PCI: 00:14.3 scanning...
[DEBUG] PNP: 002e.0 disabled
[DEBUG] PNP: 002e.1 disabled
[DEBUG] PNP: 002e.2 enabled
[DEBUG] PNP: 002e.3 disabled
[DEBUG] PNP: 002e.5 enabled
[DEBUG] PNP: 002e.6 disabled
[DEBUG] PNP: 002e.7 enabled
[DEBUG] PNP: 002e.8 disabled
[DEBUG] PNP: 002e.108 enabled
[DEBUG] PNP: 002e.9 disabled
[DEBUG] PNP: 002e.109 enabled
[DEBUG] PNP: 002e.209 enabled
[DEBUG] PNP: 002e.309 enabled
[DEBUG] PNP: 002e.409 enabled
[DEBUG] PNP: 002e.509 enabled
[DEBUG] PNP: 002e.609 enabled
[DEBUG] PNP: 002e.709 enabled
[DEBUG] PNP: 002e.a enabled
[DEBUG] PNP: 002e.b enabled
[DEBUG] PNP: 002e.d disabled
[DEBUG] PNP: 002e.e disabled
[DEBUG] PNP: 002e.f enabled
[DEBUG] PNP: 002e.14 enabled
[DEBUG] PNP: 002e.16 disabled
[DEBUG] scan_bus: bus PCI: 00:14.3 finished in 0 msecs
[DEBUG] PCI: 00:14.4 scanning...
[DEBUG] PCI: pci_scan_bus for bus 01
[DEBUG] scan_bus: bus PCI: 00:14.4 finished in 0 msecs
[DEBUG] PCI: 00:15.0 scanning...
[DEBUG] PCI: pci_scan_bus for bus 02
[DEBUG] scan_bus: bus PCI: 00:15.0 finished in 0 msecs
[DEBUG] PCI: 00:15.1 scanning...
[DEBUG] PCI: pci_scan_bus for bus 03
[DEBUG] PCI: 03:00.0 [1b21/1042] enabled
[DEBUG] scan_bus: bus PCI: 00:15.1 finished in 0 msecs
[DEBUG] scan_bus: bus DOMAIN: 0000 finished in 0 msecs
[DEBUG] scan_bus: bus Root Device finished in 0 msecs
[INFO ] done
[DEBUG] BS: BS_DEV_ENUMERATE run times (exec / console): 1 / 0 ms
[DEBUG] found VGA at PCI: 00:01.0
[DEBUG] Setting up VGA for PCI: 00:01.0
[DEBUG] Setting PCI_BRIDGE_CTL_VGA for bridge DOMAIN: 0000
[DEBUG] Setting PCI_BRIDGE_CTL_VGA for bridge Root Device
[INFO ] Allocating resources...
[INFO ] Reading resources...
[DEBUG] fx_devs=0x1
[ERROR] PNP: 002e.7 missing read_resources
[DEBUG] Adding PCIe enhanced config space BAR 0xf8000000-0xfc000000.
[INFO ] Done reading resources.
[ERROR] skipping PNP: 002e.7@f4 fixed resource, size=0!
[ERROR] skipping PNP: 002e.7@e0 fixed resource, size=0!
[ERROR] skipping PNP: 002e.7@e1 fixed resource, size=0!
[ERROR] skipping PNP: 002e.108@e0 fixed resource, size=0!
[ERROR] skipping PNP: 002e.108@e2 fixed resource, size=0!
[ERROR] skipping PNP: 002e.108@e4 fixed resource, size=0!
[ERROR] skipping PNP: 002e.108@f0 fixed resource, size=0!
[ERROR] skipping PNP: 002e.108@f4 fixed resource, size=0!
[ERROR] skipping PNP: 002e.108@f5 fixed resource, size=0!
[ERROR] skipping PNP: 002e.108@f6 fixed resource, size=0!
[ERROR] skipping PNP: 002e.108@f7 fixed resource, size=0!
[ERROR] skipping PNP: 002e.209@e0 fixed resource, size=0!
[ERROR] skipping PNP: 002e.309@e4 fixed resource, size=0!
[ERROR] skipping PNP: 002e.309@e5 fixed resource, size=0!
[ERROR] skipping PNP: 002e.409@f0 fixed resource, size=0!
[ERROR] skipping PNP: 002e.a@e6 fixed resource, size=0!
[ERROR] skipping PNP: 002e.a@e7 fixed resource, size=0!
[ERROR] skipping PNP: 002e.b@e2 fixed resource, size=0!
[ERROR] skipping PNP: 002e.b@e4 fixed resource, size=0!
[ERROR] skipping PNP: 002e.f@e6 fixed resource, size=0!
[ERROR] skipping PNP: 002e.14@e0 fixed resource, size=0!
[INFO ] Setting resources...
[DEBUG] node 0: mmio_basek=00200000, basek=00400000, limitk=005fc000
[INFO ] add_uma_resource_below_tolm: uma size 0x20000000, memory start 0x60000000
[DEBUG] PCI: 00:00.2 44 <- [0x00000000f0100000 - 0x00000000f017ffff] size 0x00080000 gran 0x13 mem
[DEBUG] PCI: 00:01.0 10 <- [0x00000000e0000000 - 0x00000000efffffff] size 0x10000000 gran 0x1c prefmem
[DEBUG] PCI: 00:01.0 14 <- [0x0000000000001000 - 0x00000000000010ff] size 0x00000100 gran 0x08 io
[DEBUG] PCI: 00:01.0 18 <- [0x00000000f0180000 - 0x00000000f01bffff] size 0x00040000 gran 0x12 mem
[DEBUG] PCI: 00:01.1 10 <- [0x00000000f01c0000 - 0x00000000f01c3fff] size 0x00004000 gran 0x0e mem
[DEBUG] PCI: 00:11.0 10 <- [0x0000000000001410 - 0x0000000000001417] size 0x00000008 gran 0x03 io
[DEBUG] PCI: 00:11.0 14 <- [0x0000000000001420 - 0x0000000000001423] size 0x00000004 gran 0x02 io
[DEBUG] PCI: 00:11.0 18 <- [0x0000000000001418 - 0x000000000000141f] size 0x00000008 gran 0x03 io
[DEBUG] PCI: 00:11.0 1c <- [0x0000000000001424 - 0x0000000000001427] size 0x00000004 gran 0x02 io
[DEBUG] PCI: 00:11.0 20 <- [0x0000000000001400 - 0x000000000000140f] size 0x00000010 gran 0x04 io
[DEBUG] PCI: 00:11.0 24 <- [0x00000000f01cc000 - 0x00000000f01cc7ff] size 0x00000800 gran 0x0b mem
[DEBUG] PCI: 00:12.0 10 <- [0x00000000f01c8000 - 0x00000000f01c8fff] size 0x00001000 gran 0x0c mem
[DEBUG] PCI: 00:12.2 10 <- [0x00000000f01cd000 - 0x00000000f01cd0ff] size 0x00000100 gran 0x08 mem
[DEBUG] PCI: 00:13.0 10 <- [0x00000000f01c9000 - 0x00000000f01c9fff] size 0x00001000 gran 0x0c mem
[DEBUG] PCI: 00:13.2 10 <- [0x00000000f01ce000 - 0x00000000f01ce0ff] size 0x00000100 gran 0x08 mem
[DEBUG] PCI: 00:14.2 10 <- [0x00000000f01c4000 - 0x00000000f01c7fff] size 0x00004000 gran 0x0e mem64
[DEBUG] PNP: 002e.2 60 <- [0x00000000000003f8 - 0x00000000000003ff] size 0x00000008 gran 0x03 io
[DEBUG] PNP: 002e.2 70 <- [0x0000000000000004 - 0x0000000000000004] size 0x00000001 gran 0x00 irq
[DEBUG] PNP: 002e.5 60 <- [0x0000000000000060 - 0x0000000000000060] size 0x00000001 gran 0x00 io
[DEBUG] PNP: 002e.5 62 <- [0x0000000000000064 - 0x0000000000000064] size 0x00000001 gran 0x00 io
[DEBUG] PNP: 002e.5 70 <- [0x0000000000000001 - 0x0000000000000001] size 0x00000001 gran 0x00 irq
[DEBUG] PNP: 002e.5 72 <- [0x000000000000000c - 0x000000000000000c] size 0x00000001 gran 0x00 irq
[ERROR] PNP: 002e.7 missing set_resources
[DEBUG] PNP: 002e.108 e0 <- [0x00000000000000ff - 0x00000000000000fe] size 0x00000000 gran 0x00 irq
[DEBUG] PNP: 002e.108 e2 <- [0x00000000000000ff - 0x00000000000000fe] size 0x00000000 gran 0x00 irq
[DEBUG] PNP: 002e.108 e4 <- [0x00000000000000ff - 0x00000000000000fe] size 0x00000000 gran 0x00 irq
[DEBUG] PNP: 002e.108 f0 <- [0x00000000000000ff - 0x00000000000000fe] size 0x00000000 gran 0x00 irq
[DEBUG] PNP: 002e.108 f4 <- [0x0000000000000008 - 0x0000000000000007] size 0x00000000 gran 0x00 irq
[DEBUG] PNP: 002e.108 f5 <- [0x00000000000000ff - 0x00000000000000fe] size 0x00000000 gran 0x00 irq
[DEBUG] PNP: 002e.108 f6 <- [0x0000000000000000 - 0xffffffffffffffff] size 0x00000000 gran 0x00 irq
[DEBUG] PNP: 002e.108 f7 <- [0x00000000000000ff - 0x00000000000000fe] size 0x00000000 gran 0x00 irq
[DEBUG] PNP: 002e.209 e0 <- [0x00000000000000ff - 0x00000000000000fe] size 0x00000000 gran 0x00 irq
[DEBUG] PNP: 002e.309 e4 <- [0x000000000000007f - 0x000000000000007e] size 0x00000000 gran 0x00 irq
[DEBUG] PNP: 002e.309 e5 <- [0x0000000000000000 - 0xffffffffffffffff] size 0x00000000 gran 0x00 irq
[DEBUG] PNP: 002e.409 f0 <- [0x00000000000000ff - 0x00000000000000fe] size 0x00000000 gran 0x00 irq
[DEBUG] PNP: 002e.509 f4 <- [0x00000000000000ff - 0x00000000000000ff] size 0x00000001 gran 0x00 irq
[WARN ] PNP: 002e.509 f5 irq size: 0x0000000001 not assigned in devicetree
[WARN ] PNP: 002e.609 f4 irq size: 0x0000000001 not assigned in devicetree
[WARN ] PNP: 002e.609 f5 irq size: 0x0000000001 not assigned in devicetree
[DEBUG] PNP: 002e.a e6 <- [0x000000000000004c - 0x000000000000004b] size 0x00000000 gran 0x00 irq
[DEBUG] PNP: 002e.a e7 <- [0x0000000000000011 - 0x0000000000000010] size 0x00000000 gran 0x00 irq
[DEBUG] PNP: 002e.a f2 <- [0x000000000000005d - 0x000000000000005d] size 0x00000001 gran 0x00 irq
[DEBUG] PNP: 002e.b 60 <- [0x0000000000000290 - 0x0000000000000291] size 0x00000002 gran 0x01 io
[DEBUG] PNP: 002e.b 62 <- [0x0000000000000000 - 0x0000000000000001] size 0x00000002 gran 0x01 io
[DEBUG] PNP: 002e.b 70 <- [0x0000000000000000 - 0x0000000000000000] size 0x00000001 gran 0x00 io
[DEBUG] PNP: 002e.b e2 <- [0x000000000000007f - 0x000000000000007e] size 0x00000000 gran 0x00 irq
[DEBUG] PNP: 002e.b e4 <- [0x00000000000000f1 - 0x00000000000000f0] size 0x00000000 gran 0x00 irq
[WARN ] PNP: 002e.b f0 irq size: 0x0000000001 not assigned in devicetree
[DEBUG] PNP: 002e.f e6 <- [0x0000000000000007 - 0x0000000000000006] size 0x00000000 gran 0x00 irq
[DEBUG] PNP: 002e.14 e0 <- [0x0000000000000000 - 0xffffffffffffffff] size 0x00000000 gran 0x00 irq
[DEBUG] PCI: 00:14.4 1c <- [0x000000000000ffff - 0x000000000000fffe] size 0x00000000 gran 0x0c bus 01 io
[DEBUG] PCI: 00:14.4 24 <- [0x00000000f7ffffff - 0x00000000f7fffffe] size 0x00000000 gran 0x14 bus 01 prefmem
[DEBUG] PCI: 00:14.4 20 <- [0x00000000f7ffffff - 0x00000000f7fffffe] size 0x00000000 gran 0x14 bus 01 mem
[DEBUG] PCI: 00:14.5 10 <- [0x00000000f01ca000 - 0x00000000f01cafff] size 0x00001000 gran 0x0c mem
[DEBUG] PCI: 00:15.0 1c <- [0x000000000000ffff - 0x000000000000fffe] size 0x00000000 gran 0x0c bus 02 io
[DEBUG] PCI: 00:15.0 24 <- [0x00000000f7ffffff - 0x00000000f7fffffe] size 0x00000000 gran 0x14 bus 02 prefmem
[DEBUG] PCI: 00:15.0 20 <- [0x00000000f7ffffff - 0x00000000f7fffffe] size 0x00000000 gran 0x14 bus 02 mem
[DEBUG] PCI: 00:15.1 1c <- [0x000000000000ffff - 0x000000000000fffe] size 0x00000000 gran 0x0c bus 03 io
[DEBUG] PCI: 00:15.1 24 <- [0x00000000f7ffffff - 0x00000000f7fffffe] size 0x00000000 gran 0x14 bus 03 prefmem
[DEBUG] PCI: 00:15.1 20 <- [0x00000000f0000000 - 0x00000000f00fffff] size 0x00100000 gran 0x14 bus 03 mem
[DEBUG] PCI: 03:00.0 10 <- [0x00000000f0000000 - 0x00000000f0007fff] size 0x00008000 gran 0x0f mem64
[DEBUG] PCI: 00:16.0 10 <- [0x00000000f01cb000 - 0x00000000f01cbfff] size 0x00001000 gran 0x0c mem
[DEBUG] PCI: 00:16.2 10 <- [0x00000000f01cf000 - 0x00000000f01cf0ff] size 0x00000100 gran 0x08 mem
[INFO ] Done setting resources.
[INFO ] Done allocating resources.
[DEBUG] BS: BS_DEV_RESOURCES run times (exec / console): 1 / 0 ms
[DEBUG] APIC 00: ** Enter AmdInitMid [00020005]
[DEBUG] AmdInitMid() returned AGESA_SUCCESS
[DEBUG] APIC 00: Heap in SystemMem (4) at 0x10000014
[DEBUG] APIC 00: ** Exit AmdInitMid [00020005]
[DEBUG] PCI_INTR tables: Writing registers C00/C01 for PIC mode PCI IRQ routing:
[DEBUG] PCI_INTR_INDEX PCI_INTR_DATA
[DEBUG] 0x00 INTA# : 0x1F
[DEBUG] 0x01 INTB# : 0x1F
[DEBUG] 0x02 INTC# : 0x1F
[DEBUG] 0x03 INTD# : 0x1F
[DEBUG] 0x04 INTE# : 0x1F
[DEBUG] 0x05 INTF# : 0x1F
[DEBUG] 0x06 INTG# : 0x1F
[DEBUG] 0x07 INTH# : 0x1F
[DEBUG] 0x08 Misc : 0x0A
[DEBUG] 0x09 Misc0 : 0xF1
[DEBUG] 0x0A Misc1 : 0x00
[DEBUG] 0x0B Misc2 : 0x00
[DEBUG] 0x0C Ser IRQ INTA : 0x1F
[DEBUG] 0x0D Ser IRQ INTB : 0x1F
[DEBUG] 0x0E Ser IRQ INTC : 0x1F
[DEBUG] 0x0F Ser IRQ INTD : 0x1F
[DEBUG] 0x10 SCI : 0x09
[DEBUG] 0x11 SMBUS0 : 0x1F
[DEBUG] 0x12 ASF : 0x1F
[DEBUG] 0x13 HDA : 0x1F
[DEBUG] 0x14 SD : 0x1F
[DEBUG] 0x15 GEC : 0x1F
[DEBUG] 0x16 PerMon : 0x1F
[DEBUG] 0x20 IMC INT0 : 0x1F
[DEBUG] 0x21 IMC INT1 : 0x1F
[DEBUG] 0x22 IMC INT2 : 0x1F
[DEBUG] 0x23 IMC INT3 : 0x1F
[DEBUG] 0x24 IMC INT4 : 0x1F
[DEBUG] 0x25 IMC INT5 : 0x1F
[DEBUG] 0x30 Dev18.0 INTA : 0x1F
[DEBUG] 0x31 Dev18.2 INTB : 0x1F
[DEBUG] 0x32 Dev19.0 INTA : 0x1F
[DEBUG] 0x33 Dev19.2 INTB : 0x1F
[DEBUG] 0x34 Dev22.0 INTA : 0x1F
[DEBUG] 0x35 Dev22.2 INTB : 0x1F
[DEBUG] 0x36 Dev20.5 INTC : 0x1F
[DEBUG] 0x40 IDE : 0x1F
[DEBUG] 0x41 SATA : 0x1F
[DEBUG] 0x50 GPPInt0 : 0x1F
[DEBUG] 0x51 GPPInt1 : 0x1F
[DEBUG] 0x52 GPPInt2 : 0x1F
[DEBUG] 0x53 GPPInt3 : 0x1F
[DEBUG] PCI_INTR tables: Writing registers C00/C01 for APIC mode PCI IRQ routing:
[DEBUG] PCI_INTR_INDEX PCI_INTR_DATA
[DEBUG] 0x00 INTA# : 0x10
[DEBUG] 0x01 INTB# : 0x11
[DEBUG] 0x02 INTC# : 0x12
[DEBUG] 0x03 INTD# : 0x13
[DEBUG] 0x04 INTE# : 0x14
[DEBUG] 0x05 INTF# : 0x15
[DEBUG] 0x06 INTG# : 0x16
[DEBUG] 0x07 INTH# : 0x17
[DEBUG] 0x08 Misc : 0x00
[DEBUG] 0x09 Misc0 : 0x00
[DEBUG] 0x0A Misc1 : 0x00
[DEBUG] 0x0B Misc2 : 0x00
[DEBUG] 0x0C Ser IRQ INTA : 0x1F
[DEBUG] 0x0D Ser IRQ INTB : 0x1F
[DEBUG] 0x0E Ser IRQ INTC : 0x1F
[DEBUG] 0x0F Ser IRQ INTD : 0x1F
[DEBUG] 0x10 SCI : 0x09
[DEBUG] 0x11 SMBUS0 : 0x1F
[DEBUG] 0x12 ASF : 0x1F
[DEBUG] 0x13 HDA : 0x10
[DEBUG] 0x14 SD : 0x1F
[DEBUG] 0x15 GEC : 0x10
[DEBUG] 0x16 PerMon : 0x1F
[DEBUG] 0x20 IMC INT0 : 0x05
[DEBUG] 0x21 IMC INT1 : 0x1F
[DEBUG] 0x22 IMC INT2 : 0x1F
[DEBUG] 0x23 IMC INT3 : 0x1F
[DEBUG] 0x24 IMC INT4 : 0x1F
[DEBUG] 0x25 IMC INT5 : 0x1F
[DEBUG] 0x30 Dev18.0 INTA : 0x12
[DEBUG] 0x31 Dev18.2 INTB : 0x11
[DEBUG] 0x32 Dev19.0 INTA : 0x12
[DEBUG] 0x33 Dev19.2 INTB : 0x11
[DEBUG] 0x34 Dev22.0 INTA : 0x12
[DEBUG] 0x35 Dev22.2 INTB : 0x11
[DEBUG] 0x36 Dev20.5 INTC : 0x12
[DEBUG] 0x40 IDE : 0x11
[DEBUG] 0x41 SATA : 0x13
[DEBUG] 0x50 GPPInt0 : 0x10
[DEBUG] 0x51 GPPInt1 : 0x11
[DEBUG] 0x52 GPPInt2 : 0x12
[DEBUG] 0x53 GPPInt3 : 0x13
[WARN ] Can't write PCI IRQ assignments because 'mainboard_pirq_data' structure does not exist
[DEBUG] BS: BS_DEV_ENABLE entry times (exec / console): 5 / 0 ms
[INFO ] Enabling resources...
[DEBUG] PCI: 00:00.0 subsystem <- 1022/1410
[DEBUG] PCI: 00:00.0 cmd <- 06
[DEBUG] PCI: 00:00.2 cmd <- 06
[DEBUG] PCI: 00:01.0 cmd <- 07
[DEBUG] PCI: 00:01.1 cmd <- 02
[DEBUG] PCI: 00:11.0 cmd <- 03
[DEBUG] PCI: 00:12.0 subsystem <- 1022/1410
[DEBUG] PCI: 00:12.0 cmd <- 02
[DEBUG] PCI: 00:12.2 subsystem <- 1022/1410
[DEBUG] PCI: 00:12.2 cmd <- 02
[DEBUG] PCI: 00:13.0 subsystem <- 1022/1410
[DEBUG] PCI: 00:13.0 cmd <- 02
[DEBUG] PCI: 00:13.2 subsystem <- 1022/1410
[DEBUG] PCI: 00:13.2 cmd <- 02
[DEBUG] PCI: 00:14.0 subsystem <- 1022/1410
[DEBUG] PCI: 00:14.0 cmd <- 403
[DEBUG] PCI: 00:14.2 subsystem <- 1022/1410
[DEBUG] PCI: 00:14.2 cmd <- 02
[DEBUG] PCI: 00:14.3 subsystem <- 1022/1410
[DEBUG] PCI: 00:14.3 cmd <- 0f
[DEBUG] hudson lpc decode:PNP: 002e.2, base=0x000003f8, end=0x000003ff
[DEBUG] hudson lpc decode:PNP: 002e.5, base=0x00000060, end=0x00000060
[DEBUG] hudson lpc decode:PNP: 002e.5, base=0x00000064, end=0x00000064
[DEBUG] hudson lpc decode:PNP: 002e.b, base=0x00000290, end=0x00000291
[DEBUG] PCI: 00:14.4 bridge ctrl <- 0013
[DEBUG] PCI: 00:14.4 cmd <- 00
[DEBUG] PCI: 00:14.5 cmd <- 02
[DEBUG] PCI: 00:15.0 bridge ctrl <- 0013
[DEBUG] PCI: 00:15.0 cmd <- 00
[DEBUG] PCI: 00:15.1 bridge ctrl <- 0013
[DEBUG] PCI: 00:15.1 cmd <- 06
[DEBUG] PCI: 00:16.0 cmd <- 02
[DEBUG] PCI: 00:16.2 cmd <- 02
[DEBUG] PCI: 00:18.0 cmd <- 00
[DEBUG] PCI: 00:18.1 subsystem <- 1022/1410
[DEBUG] PCI: 00:18.1 cmd <- 00
[DEBUG] PCI: 00:18.2 subsystem <- 1022/1410
[DEBUG] PCI: 00:18.2 cmd <- 00
[DEBUG] PCI: 00:18.3 subsystem <- 1022/1410
[DEBUG] PCI: 00:18.3 cmd <- 00
[DEBUG] PCI: 00:18.4 subsystem <- 1022/1410
[DEBUG] PCI: 00:18.4 cmd <- 00
[DEBUG] PCI: 00:18.5 subsystem <- 1022/1410
[DEBUG] PCI: 00:18.5 cmd <- 00
[DEBUG] PCI: 03:00.0 cmd <- 02
[INFO ] done.
[INFO ] Initializing devices...
[DEBUG] CPU_CLUSTER: 0 init
[DEBUG] MTRR check
[DEBUG] Fixed MTRRs : Disabled
[DEBUG] Variable MTRRs: Enabled
[INFO ] CPU: AMD A6-6400K APU with Radeon(tm) HD Graphics .
[INFO ] LAPIC 0x10 in XAPIC mode.
[DEBUG] Loading module at 0x00030000 with entry 0x00030000. filesize: 0x1b8 memsize: 0x1b8
[DEBUG] Processing 18 relocs. Offset value of 0x00030000
[DEBUG] Attempting to start 1 APs
[DEBUG] Waiting for 10ms after sending INIT.
[DEBUG] Waiting for SIPI to complete...
[INFO ] LAPIC 0x11 in XAPIC mode.
[DEBUG] done.
[INFO ] AP: slot 1 apic_id 11
[DEBUG] Waiting for SIPI to complete...
[DEBUG] done.
[INFO ] Initializing CPU #0
[DEBUG] CPU: vendor AMD device 610f31
[DEBUG] CPU: family 15, model 13, stepping 01
[DEBUG] Model 15 Init.
[DEBUG] siblings = 01, CPU #0 initialized
[INFO ] Initializing CPU #1
[DEBUG] CPU: vendor AMD device 610f31
[DEBUG] CPU: family 15, model 13, stepping 01
[DEBUG] Model 15 Init.
[DEBUG] siblings = 01, CPU #1 initialized
[INFO ] bsp_do_flight_plan done after 0 msecs.
[DEBUG] CPU_CLUSTER: 0 init finished in 10 msecs
[DEBUG] PCI: 00:00.0 init
[DEBUG] PCI: 00:00.0 init finished in 0 msecs
[DEBUG] PCI: 00:01.0 init
[DEBUG] PCI: 00:01.0 init finished in 0 msecs
[DEBUG] PCI: 00:01.1 init
[DEBUG] PCI: 00:01.1 init finished in 0 msecs
[DEBUG] PCI: 00:11.0 init
[DEBUG] PCI: 00:11.0 init finished in 0 msecs
[DEBUG] PCI: 00:12.0 init
[DEBUG] PCI: 00:12.0 init finished in 0 msecs
[DEBUG] PCI: 00:12.2 init
[DEBUG] PCI: 00:12.2 init finished in 0 msecs
[DEBUG] PCI: 00:13.0 init
[DEBUG] PCI: 00:13.0 init finished in 0 msecs
[DEBUG] PCI: 00:13.2 init
[DEBUG] PCI: 00:13.2 init finished in 0 msecs
[DEBUG] PCI: 00:14.0 init
[DEBUG] IOAPIC: Initializing IOAPIC at 0xfec00000
[DEBUG] IOAPIC: ID = 0x04
[DEBUG] IOAPIC: 24 interrupts
[DEBUG] IOAPIC: Clearing IOAPIC at 0xfec00000
[DEBUG] IOAPIC: Bootstrap Processor Local APIC = 0x10
[DEBUG] PCI: 00:14.0 init finished in 0 msecs
[DEBUG] PCI: 00:14.2 init
[DEBUG] PCI: 00:14.2 init finished in 0 msecs
[DEBUG] PCI: 00:14.3 init
[DEBUG] RTC Init
[DEBUG] PCI: 00:14.3 init finished in 0 msecs
[DEBUG] PCI: 00:14.5 init
[DEBUG] PCI: 00:14.5 init finished in 0 msecs
[DEBUG] PCI: 00:15.0 init
[DEBUG] PCI: 00:15.0 init finished in 0 msecs
[DEBUG] PCI: 00:15.1 init
[DEBUG] PCI: 00:15.1 init finished in 0 msecs
[DEBUG] PCI: 00:16.0 init
[DEBUG] PCI: 00:16.0 init finished in 0 msecs
[DEBUG] PCI: 00:16.2 init
[DEBUG] PCI: 00:16.2 init finished in 0 msecs
[DEBUG] PCI: 00:18.1 init
[DEBUG] PCI: 00:18.1 init finished in 0 msecs
[DEBUG] PCI: 00:18.2 init
[DEBUG] PCI: 00:18.2 init finished in 0 msecs
[DEBUG] PCI: 00:18.3 init
[DEBUG] PCI: 00:18.3 init finished in 0 msecs
[DEBUG] PCI: 00:18.4 init
[DEBUG] PCI: 00:18.4 init finished in 0 msecs
[DEBUG] PCI: 00:18.5 init
[DEBUG] PCI: 00:18.5 init finished in 0 msecs
[DEBUG] PNP: 002e.2 init
[DEBUG] PNP: 002e.2 init finished in 0 msecs
[DEBUG] PNP: 002e.5 init
[DEBUG] PNP: 002e.5 init finished in 0 msecs
[DEBUG] PNP: 002e.108 init
[DEBUG] PNP: 002e.108 init finished in 0 msecs
[DEBUG] PNP: 002e.109 init
[DEBUG] PNP: 002e.109 init finished in 0 msecs
[DEBUG] PNP: 002e.209 init
[DEBUG] PNP: 002e.209 init finished in 0 msecs
[DEBUG] PNP: 002e.309 init
[DEBUG] PNP: 002e.309 init finished in 0 msecs
[DEBUG] PNP: 002e.409 init
[DEBUG] PNP: 002e.409 init finished in 0 msecs
[DEBUG] PNP: 002e.509 init
[DEBUG] PNP: 002e.509 init finished in 0 msecs
[DEBUG] PNP: 002e.609 init
[DEBUG] PNP: 002e.609 init finished in 0 msecs
[DEBUG] PNP: 002e.709 init
[DEBUG] PNP: 002e.709 init finished in 0 msecs
[DEBUG] PNP: 002e.a init
[DEBUG] PNP: 002e.a init finished in 0 msecs
[DEBUG] PNP: 002e.b init
[DEBUG] PNP: 002e.b init finished in 0 msecs
[DEBUG] PNP: 002e.f init
[DEBUG] PNP: 002e.f init finished in 0 msecs
[DEBUG] PNP: 002e.14 init
[DEBUG] PNP: 002e.14 init finished in 0 msecs
[DEBUG] PCI: 03:00.0 init
[DEBUG] PCI: 03:00.0 init finished in 0 msecs
[INFO ] Devices initialized
[DEBUG] BS: BS_DEV_INIT run times (exec / console): 10 / 0 ms
[INFO ] Finalize devices...
[DEBUG] PCI: 00:14.3 final
[INFO ] Devices finalized
[DEBUG] APIC 00: ** Enter AmdInitLate [00020004]
[EMERG] ASSERTION ERROR: file 'src/vendorcode/amd/agesa/f15tn/Proc/Common/CommonReturns.c', line 187
[EMERG] ASSERTION ERROR: file 'src/vendorcode/amd/agesa/f15tn/Proc/CPU/cpuGeneralServices.c', line 776
[EMERG] ASSERTION ERROR: file 'src/vendorcode/amd/agesa/f15tn/Proc/Common/CommonReturns.c', line 187
[EMERG] ASSERTION ERROR: file 'src/vendorcode/amd/agesa/f15tn/Proc/CPU/cpuGeneralServices.c', line 776
[EMERG] ASSERTION ERROR: file 'src/vendorcode/amd/agesa/f15tn/Proc/Common/CommonReturns.c', line 187
[EMERG] ASSERTION ERROR: file 'src/vendorcode/amd/agesa/f15tn/Proc/CPU/cpuGeneralServices.c', line 776
[EMERG] ASSERTION ERROR: file 'src/vendorcode/amd/agesa/f15tn/Proc/Common/CommonReturns.c', line 187
[EMERG] ASSERTION ERROR: file 'src/vendorcode/amd/agesa/f15tn/Proc/CPU/cpuGeneralServices.c', line 776
[EMERG] ASSERTION ERROR: file 'src/vendorcode/amd/agesa/f15tn/Proc/Common/CommonReturns.c', line 187
[EMERG] ASSERTION ERROR: file 'src/vendorcode/amd/agesa/f15tn/Proc/CPU/cpuGeneralServices.c', line 776
[DEBUG] AmdInitLate() returned AGESA_SUCCESS
[DEBUG] APIC 00: Heap in SystemMem (4) at 0x10000014
[DEBUG] APIC 00: ** Exit AmdInitLate [00020004]
[DEBUG] APIC 00: ** Enter AmdS3Save [0002000b]
[DEBUG] FMAP: area COREBOOT found @ 10200 (4128256 bytes)
[INFO ] CBFS: Found 'pci1002,9996.rom' @0x8f6c0 size 0xf200 in mcache @0x5ffdd294
[DEBUG] FMAP: area RW_MRC_CACHE found @ 0 (65536 bytes)
[DEBUG] MRC: Checking cached data update for 'RW_MRC_CACHE'.
[INFO ] Manufacturer: ef
[INFO ] SF: Detected ef 4016 with sector size 0x1000, total 0x400000
[DEBUG] MRC: cache data 'RW_MRC_CACHE' needs update.
[DEBUG] MRC: updated 'RW_MRC_CACHE'.
[DEBUG] AmdS3Save() returned AGESA_SUCCESS
[EMERG] ASSERTION ERROR: file 'src/drivers/amd/agesa/state_machine.c', line 276
[DEBUG] APIC 00: Heap in SystemMem (4) at 0x10000014
[DEBUG] APIC 00: ** Exit AmdS3Save [0002000b]
[DEBUG] BS: BS_POST_DEVICE exit times (exec / console): 14 / 0 ms
[INFO ] Writing IRQ routing tables to 0xf0000...write_pirq_routing_table done.
[INFO ] Writing IRQ routing tables to 0x5fe6e000...write_pirq_routing_table done.
[DEBUG] PIRQ table: 48 bytes.
[INFO ] CBFS: Found 'fallback/dsdt.aml' @0x700 size 0x193a in mcache @0x5ffdd094
[WARN ] CBFS: 'fallback/slic' not found.
[INFO ] ACPI: Writing ACPI tables at 5fe4a000.
[DEBUG] ACPI: * FACS
[DEBUG] ACPI: * DSDT
[DEBUG] ACPI: * FADT
[DEBUG] pm_base: 0x0800
[DEBUG] ACPI: added table 1/32, length now 40
[DEBUG] ACPI: * SSDT
[INFO ] CBFS: Found 'pci1002,9996.rom' @0x8f6c0 size 0xf200 in mcache @0x5ffdd294
[DEBUG] In CBFS, ROM address for PCI: 00:01.0 = 0xffc9f8ec
[ERROR] PCI: 00:01.0: Missing ACPI scope
[DEBUG] ACPI: added table 2/32, length now 44
[DEBUG] ACPI: * MCFG
[DEBUG] ACPI: added table 3/32, length now 48
[DEBUG] ACPI: * MADT
[DEBUG] ACPI: added table 4/32, length now 52
[DEBUG] current = 5fe4be20
[DEBUG] ACPI: * HPET
[DEBUG] ACPI: added table 5/32, length now 56
[DEBUG] ACPI: added table 6/32, length now 60
[DEBUG] ACPI: * IVRS at 5fe4c030
[DEBUG] ACPI: added table 7/32, length now 64
[DEBUG] ACPI: * SRAT at 5fe4c0a0
[DEBUG] AGESA SRAT table NULL. Skipping.
[DEBUG] ACPI: * SLIT at 5fe4c0a0
[DEBUG] AGESA SLIT table NULL. Skipping.
[DEBUG] ACPI: * AGESA ALIB SSDT at 5fe4c0a0
[DEBUG] ACPI: added table 8/32, length now 68
[DEBUG] ACPI: * SSDT at 5fe4c5c0
[DEBUG] ACPI: added table 9/32, length now 72
[DEBUG] ACPI: * SSDT for PState at 5fe4cc72
[INFO ] CBFS: Found 'pci1002,9996.rom' @0x8f6c0 size 0xf200 in mcache @0x5ffdd294
[DEBUG] In CBFS, ROM address for PCI: 00:01.0 = 0xffc9f8ec
[DEBUG] Copying VBIOS image from 0xffc9f8ec
[DEBUG] ACPI: * VFCT at 5fe4cc80
[DEBUG] ACPI: added table 10/32, length now 76
[INFO ] ACPI: done.
[DEBUG] ACPI tables: 73456 bytes.
[DEBUG] smbios_write_tables: 5fe42000
[DEBUG] SMBIOS firmware version is set to coreboot_version: '4.18-15-gc782ef4345'
[DEBUG] SMBIOS tables: 531 bytes.
[DEBUG] Writing table forward entry at 0x00000500
[DEBUG] Wrote coreboot table at: 0x00000500, 0x10 bytes, checksum aff7
[DEBUG] Writing coreboot table at 0x5fe6f000
[DEBUG] 0. 0000000000000000-0000000000000fff: CONFIGURATION TABLES
[DEBUG] 1. 0000000000001000-000000000009ffff: RAM
[DEBUG] 2. 00000000000c0000-000000005fe41fff: RAM
[DEBUG] 3. 000000005fe42000-000000005feb7fff: CONFIGURATION TABLES
[DEBUG] 4. 000000005feb8000-000000005ffcefff: RAMSTAGE
[DEBUG] 5. 000000005ffcf000-000000005fffffff: CONFIGURATION TABLES
[DEBUG] 6. 0000000060000000-000000007fffffff: RESERVED
[DEBUG] 7. 00000000f8000000-00000000fbffffff: RESERVED
[DEBUG] 8. 00000000fec10000-00000000fec10fff: RESERVED
[DEBUG] 9. 0000000100000000-000000017effffff: RAM
[DEBUG] Wrote coreboot table at: 0x5fe6f000, 0x354 bytes, checksum d6c8
[DEBUG] coreboot table: 876 bytes.
[DEBUG] IMD ROOT 0. 0x5ffff000 0x00001000
[DEBUG] IMD SMALL 1. 0x5fffe000 0x00001000
[DEBUG] CONSOLE 2. 0x5ffde000 0x00020000
[DEBUG] RO MCACHE 3. 0x5ffdd000 0x00000330
[DEBUG] TIME STAMP 4. 0x5ffdc000 0x00000910
[DEBUG] AFTER CAR 5. 0x5ffcf000 0x0000d000
[DEBUG] RAMSTAGE 6. 0x5feb7000 0x00118000
[DEBUG] SMM BACKUP 7. 0x5fea7000 0x00010000
[DEBUG] ACPISCRATCH 8. 0x5fe77000 0x00030000
[DEBUG] COREBOOT 9. 0x5fe6f000 0x00008000
[DEBUG] IRQ TABLE 10. 0x5fe6e000 0x00001000
[DEBUG] ACPI 11. 0x5fe4a000 0x00024000
[DEBUG] SMBIOS 12. 0x5fe42000 0x00008000
[DEBUG] IMD small region:
[DEBUG] IMD ROOT 0. 0x5fffec00 0x00000400
[DEBUG] FMAP 1. 0x5fffeb20 0x000000e0
[DEBUG] ROMSTAGE 2. 0x5fffeb00 0x00000004
[DEBUG] ROMSTG STCK 3. 0x5fffea60 0x00000088
[DEBUG] AGESA MTRR 4. 0x5fffe960 0x000000f0
[DEBUG] BS: BS_WRITE_TABLES run times (exec / console): 9 / 0 ms
[INFO ] CBFS: Found 'fallback/payload' @0x9e900 size 0x118e9 in mcache @0x5ffdd2c0
[DEBUG] Checking segment from ROM address 0xffcaeb2c
[DEBUG] Checking segment from ROM address 0xffcaeb48
[DEBUG] Loading segment from ROM address 0xffcaeb2c
[DEBUG] code (compression=1)
[DEBUG] New segment dstaddr 0x000dee80 memsize 0x21180 srcaddr 0xffcaeb64 filesize 0x118b1
[DEBUG] Loading Segment: addr: 0x000dee80 memsz: 0x0000000000021180 filesz: 0x00000000000118b1
[DEBUG] using LZMA
[DEBUG] Loading segment from ROM address 0xffcaeb48
[DEBUG] Entry Point 0x000fd266
[DEBUG] BS: BS_PAYLOAD_LOAD run times (exec / console): 13 / 0 ms
[DEBUG] Jumping to boot code at 0x000fd266(0x5fe6f000)
SeaBIOS (version rel-1.16.2-0-gea1b7a0)
BUILD: gcc: (Debian 12.2.0-14) 12.2.0 binutils: (GNU Binutils for Debian) 2.40
Found coreboot cbmem console @ 5ffde000
Found mainboard ASUS F2A85-M_PRO
Relocating init from 0x000e05c0 to 0x5ee34b20 (size 54336)
Found CBFS header at 0xffc1022c
multiboot: eax=5fefb3d8, ebx=5fefb3a4
Found 26 PCI devices (max PCI bus is 03)
Copying SMBIOS from 0x5fe42000 to 0x000f6880
Copying SMBIOS 3.0 from 0x5fe42020 to 0x000f6860
Copying ACPI RSDP from 0x5fe4a000 to 0x000f6830
Copying PIR from 0x5fe6e000 to 0x000f6800
table(50434146)=0x5fe4bbc0 (via xsdt)
Using pmtimer, ioport 0x818
Scan for VGA option rom
Running option rom at c000:0003
Turning on vga text mode console
SeaBIOS (version rel-1.16.2-0-gea1b7a0)
PCI: XHCI at 03:00.0 (mmio 0xf0000000)
XHCI init: regs @ 0xf0000000, 4 ports, 32 slots, 32 byte contexts
XHCI extcap 0x1 @ 0xf0000800
XHCI protocol USB 3.00, 2 ports (offset 1), def 0
XHCI protocol USB 2.00, 2 ports (offset 3), def 1
EHCI init on dev 00:12.2 (regs=0xf01cd020)
EHCI init on dev 00:13.2 (regs=0xf01ce020)
EHCI init on dev 00:16.2 (regs=0xf01cf020)
OHCI init on dev 00:12.0 (regs=0xf01c8000)
OHCI init on dev 00:13.0 (regs=0xf01c9000)
OHCI init on dev 00:14.5 (regs=0xf01ca000)
OHCI init on dev 00:16.0 (regs=0xf01cb000)
AHCI controller at 00:11.0, iobase 0xf01cc000, irq 0
Searching bootorder for: HALT
Found 0 lpt ports
Found 1 serial ports
Searching bootorder for: /pci@i0cf8/*@11/drive@6/disk@0
AHCI/6: Set transfer mode to UDMA-6
Searching bios-geometry for: /pci@i0cf8/*@11/drive@6/disk@0
AHCI/6: registering: "AHCI/6: SanDisk SDSSDP064G ATA-9 Hard-Disk (61057 MiBytes)"
USB keyboard initialized
USB mouse initialized
XHCI no devices found
PS2 keyboard initialized
All threads complete.
Scan for option roms
Press ESC for boot menu.
Searching bootorder for: HALT
drive 0x000f6790: PCHS=16383/16/63 translation=lba LCHS=1024/255/63 s=125045424
Space available for UMB: cf800-ec000, f60a0-f6790
Returned 16756736 bytes of ZoneHigh
e820 map has 8 items:
0: 0000000000000000 - 000000000009fc00 = 1 RAM
1: 000000000009fc00 - 00000000000a0000 = 2 RESERVED
2: 00000000000f0000 - 0000000000100000 = 2 RESERVED
3: 0000000000100000 - 000000005fe3d000 = 1 RAM
4: 000000005fe3d000 - 0000000080000000 = 2 RESERVED
5: 00000000f8000000 - 00000000fc000000 = 2 RESERVED
6: 00000000fec10000 - 00000000fec11000 = 2 RESERVED
7: 0000000100000000 - 000000017f000000 = 1 RAM
enter handle_19:
NULL
Booting from Hard Disk...
Booting from 0000:7c00
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-19 16:45 ` Paul Menzel
0 siblings, 0 replies; 236+ messages in thread
From: Paul Menzel @ 2023-04-19 16:45 UTC (permalink / raw)
To: Thomas Gleixner
Cc: linux-kernel, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
[-- Attachment #1: Type: text/plain, Size: 2408 bytes --]
Dear Thomas,
Am 19.04.23 um 14:38 schrieb Thomas Gleixner:
> On Wed, Apr 19 2023 at 11:38, Thomas Gleixner wrote:
>> On Tue, Apr 18 2023 at 22:10, Paul Menzel wrote:
>>> Am 18.04.23 um 10:40 schrieb Thomas Gleixner:
>>>> Can you please provide the output of cpuid?
>>>
>>> Of course. Here the top, and the whole output is attached.
>>
>> Thanks for the data. Can you please apply the debug patch below and
>> provide the dmesg output? Just the line which is added by the patch is
>> enough. You can boot with cpuhp.parallel=off so you don't have wait for
>> 10 seconds.
>
> Borislav found some a machine which also refuses to boot. It turns of
> the debug patch was spot on:
>
> [ 0.462724] .... node #0, CPUs: #1
> [ 0.462731] smpboot: Kicking AP alive: 17
> [ 0.465723] #2
> [ 0.465732] smpboot: Kicking AP alive: 18
> [ 0.467641] #3
> [ 0.467641] smpboot: Kicking AP alive: 19
>
> So the kernel gets APICID 17, 18, 19 from ACPI but CPUID leaf 0x1
> ebx[31:24], which is the initial APICID has:
>
> CPU1 0x01
> CPU2 0x02
> CPU3 0x03
>
> Which means the APICID to Linux CPU number lookup based on CPUID 0x01
> fails for all of them and stops them dead in the low level startup code.
I am attaching the logs for completeness. Linux is build from your
branch with the debug print on top. The firmware, coreboot based, is
built from [1], but it also happened non-parallel MP init. The code has
better debug prints (attached) though as far as I can see. As Borislav
is able to reproduce this too with some non-coreboot firmware, I assume
it’s unrelated to coreboot.
```
[ 0.259247] smp: Bringing up secondary CPUs ...
[ 0.259446] x86: Booting SMP configuration:
[ 0.259448] .... node #0, CPUs: #1
[ 0.259453] smpboot: Kicking AP alive: 17
[ 10.260918] CPU1 failed to report alive state
[ 10.260998] smp: Brought up 1 node, 1 CPU
[ 10.261000] smpboot: Max logical packages: 2
[ 10.261001] smpboot: Total of 1 processors activated (7801.09 BogoMIPS)
```
> IOW, the BIOS assignes random numbers to the AP APICs for whatever
> raisins, which leaves the parallel startup low level code up a creek
> without a paddle, except for actually reading the APICID back from the
> APIC. *SHUDDER*
>
> I'm leaning towards disabling the CPUID lead 0x01 based discovery and be
> done with it.
Kind regards,
Paul
[1]: https://review.coreboot.org/68169
[-- Attachment #2: kodi-linux-6.3-rc3-smp-tglx.txt --]
[-- Type: text/plain, Size: 57422 bytes --]
[ 0.000000] Linux version 6.3.0-rc3-00045-g64de4df9c80b (root@bf16f3646a84) (gcc (Debian 11.2.0-12) 11.2.0, GNU ld (GNU Binutils for Debian) 2.40) #449 SMP PREEMPT_DYNAMIC Wed Apr 19 16:13:54 UTC 2023
[ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc3-00045-g64de4df9c80b root=/dev/sda3 rw quiet noisapnp cryptomgr.notests ipv6.disable_ipv6=1 selinux=0
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256
[ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format.
[ 0.000000] signal: max sigframe size: 1776
[ 0.000000] BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
[ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005fe3cfff] usable
[ 0.000000] BIOS-e820: [mem 0x000000005fe3d000-0x000000007fffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000f8000000-0x00000000fbffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000fec10000-0x00000000fec10fff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000017effffff] usable
[ 0.000000] NX (Execute Disable) protection: active
[ 0.000000] SMBIOS 3.0.0 present.
[ 0.000000] DMI: ASUS F2A85-M_PRO/F2A85-M_PRO, BIOS 4.18-15-gc782ef4345 04/19/2023
[ 0.000000] tsc: Fast TSC calibration using PIT
[ 0.000000] tsc: Detected 3900.549 MHz processor
[ 0.000756] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
[ 0.000759] e820: remove [mem 0x000a0000-0x000fffff] usable
[ 0.000763] last_pfn = 0x17f000 max_arch_pfn = 0x400000000
[ 0.000768] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT
[ 0.000940] last_pfn = 0x5fe3d max_arch_pfn = 0x400000000
[ 0.004000] Using GB pages for direct mapping
[ 0.004000] ACPI: Early table checksum verification disabled
[ 0.004000] ACPI: RSDP 0x00000000000F6830 000024 (v02 COREv4)
[ 0.004000] ACPI: XSDT 0x000000005FE4A0E0 000074 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: FACP 0x000000005FE4BBC0 000114 (v06 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: DSDT 0x000000005FE4A280 00193A (v02 COREv4 COREBOOT 00010001 INTL 20200925)
[ 0.004000] ACPI: FACS 0x000000005FE4A240 000040
[ 0.004000] ACPI: FACS 0x000000005FE4A240 000040
[ 0.004000] ACPI: SSDT 0x000000005FE4BCE0 00008A (v02 COREv4 COREBOOT 0000002A CORE 20200925)
[ 0.004000] ACPI: MCFG 0x000000005FE4BD70 00003C (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: APIC 0x000000005FE4BDB0 000062 (v03 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: HPET 0x000000005FE4BE20 000038 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: HEST 0x000000005FE4BE60 0001D0 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: IVRS 0x000000005FE4C030 000070 (v02 AMD AMDIOMMU 00000001 AMD 00000000)
[ 0.004000] ACPI: SSDT 0x000000005FE4C0A0 00051F (v02 AMD ALIB 00000001 MSFT 04000000)
[ 0.004000] ACPI: SSDT 0x000000005FE4C5C0 0006B2 (v01 AMD POWERNOW 00000001 AMD 00000001)
[ 0.004000] ACPI: VFCT 0x000000005FE4CC80 00F269 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: Reserving FACP table memory at [mem 0x5fe4bbc0-0x5fe4bcd3]
[ 0.004000] ACPI: Reserving DSDT table memory at [mem 0x5fe4a280-0x5fe4bbb9]
[ 0.004000] ACPI: Reserving FACS table memory at [mem 0x5fe4a240-0x5fe4a27f]
[ 0.004000] ACPI: Reserving FACS table memory at [mem 0x5fe4a240-0x5fe4a27f]
[ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe4bce0-0x5fe4bd69]
[ 0.004000] ACPI: Reserving MCFG table memory at [mem 0x5fe4bd70-0x5fe4bdab]
[ 0.004000] ACPI: Reserving APIC table memory at [mem 0x5fe4bdb0-0x5fe4be11]
[ 0.004000] ACPI: Reserving HPET table memory at [mem 0x5fe4be20-0x5fe4be57]
[ 0.004000] ACPI: Reserving HEST table memory at [mem 0x5fe4be60-0x5fe4c02f]
[ 0.004000] ACPI: Reserving IVRS table memory at [mem 0x5fe4c030-0x5fe4c09f]
[ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe4c0a0-0x5fe4c5be]
[ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe4c5c0-0x5fe4cc71]
[ 0.004000] ACPI: Reserving VFCT table memory at [mem 0x5fe4cc80-0x5fe5bee8]
[ 0.004000] No NUMA configuration found
[ 0.004000] Faking a node at [mem 0x0000000000000000-0x000000017effffff]
[ 0.004000] NODE_DATA(0) allocated [mem 0x17efe9000-0x17effffff]
[ 0.004000] Zone ranges:
[ 0.004000] DMA [mem 0x0000000000001000-0x0000000000ffffff]
[ 0.004000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]
[ 0.004000] Normal [mem 0x0000000100000000-0x000000017effffff]
[ 0.004000] Device empty
[ 0.004000] Movable zone start for each node
[ 0.004000] Early memory node ranges
[ 0.004000] node 0: [mem 0x0000000000001000-0x000000000009efff]
[ 0.004000] node 0: [mem 0x0000000000100000-0x000000005fe3cfff]
[ 0.004000] node 0: [mem 0x0000000100000000-0x000000017effffff]
[ 0.004000] Initmem setup node 0 [mem 0x0000000000001000-0x000000017effffff]
[ 0.004000] On node 0, zone DMA: 1 pages in unavailable ranges
[ 0.004000] On node 0, zone DMA: 97 pages in unavailable ranges
[ 0.004000] On node 0, zone Normal: 451 pages in unavailable ranges
[ 0.004000] On node 0, zone Normal: 4096 pages in unavailable ranges
[ 0.004000] ACPI: PM-Timer IO Port: 0x818
[ 0.004000] ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1])
[ 0.004000] IOAPIC[0]: apic_id 4, version 33, address 0xfec00000, GSI 0-23
[ 0.004000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[ 0.004000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 low level)
[ 0.004000] ACPI: Using ACPI (MADT) for SMP configuration information
[ 0.004000] ACPI: HPET id: 0x10228210 base: 0xfed00000
[ 0.004000] smpboot: Allowing 2 CPUs, 0 hotplug CPUs
[ 0.004000] [mem 0x80000000-0xf7ffffff] available for PCI devices
[ 0.004000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns
[ 0.004000] setup_percpu: NR_CPUS:64 nr_cpumask_bits:2 nr_cpu_ids:2 nr_node_ids:1
[ 0.004000] percpu: Embedded 55 pages/cpu s188392 r8192 d28696 u1048576
[ 0.004000] pcpu-alloc: s188392 r8192 d28696 u1048576 alloc=1*2097152
[ 0.004000] pcpu-alloc: [0] 0 1
[ 0.004000] Fallback order for Node 0: 0
[ 0.004000] Built 1 zonelists, mobility grouping on. Total pages: 898436
[ 0.004000] Policy zone: Normal
[ 0.004000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc3-00045-g64de4df9c80b root=/dev/sda3 rw quiet noisapnp cryptomgr.notests ipv6.disable_ipv6=1 selinux=0
[ 0.004000] Unknown kernel command line parameters "noisapnp BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc3-00045-g64de4df9c80b", will be passed to user space.
[ 0.004000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[ 0.004000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[ 0.004000] mem auto-init: stack:off, heap alloc:off, heap free:off
[ 0.004000] stackdepot: allocating hash table via alloc_large_system_hash
[ 0.004000] stackdepot hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[ 0.004000] software IO TLB: area num 2.
[ 0.004000] Memory: 3477104K/3651436K available (14336K kernel code, 2340K rwdata, 5308K rodata, 2908K init, 11064K bss, 174072K reserved, 0K cma-reserved)
[ 0.004000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
[ 0.004000] ftrace: allocating 38652 entries in 151 pages
[ 0.004000] ftrace: allocated 151 pages with 5 groups
[ 0.004000] Dynamic Preempt: full
[ 0.004000] rcu: Preemptible hierarchical RCU implementation.
[ 0.004000] rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=2.
[ 0.004000] Trampoline variant of Tasks RCU enabled.
[ 0.004000] Rude variant of Tasks RCU enabled.
[ 0.004000] Tracing variant of Tasks RCU enabled.
[ 0.004000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[ 0.004000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2
[ 0.004000] NR_IRQS: 4352, nr_irqs: 440, preallocated irqs: 16
[ 0.004000] rcu: srcu_init: Setting srcu_struct sizes based on contention.
[ 0.004000] spurious 8259A interrupt: IRQ7.
[ 0.004000] Console: colour VGA+ 80x25
[ 0.004000] printk: console [tty0] enabled
[ 0.004000] ACPI: Core revision 20221020
[ 0.004000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 133484873504 ns
[ 0.004000] APIC: Switch to symmetric I/O mode setup
[ 0.004000] AMD-Vi: Using global IVHD EFR:0x0, EFR2:0x0
[ 0.004000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[ 0.004000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x7072c3c0f54, max_idle_ns: 881590752806 ns
[ 0.144910] Calibrating delay loop (skipped), value calculated using timer frequency.. 7801.09 BogoMIPS (lpj=15602196)
[ 0.144913] pid_max: default: 32768 minimum: 301
[ 0.145008] LSM: initializing lsm=capability
[ 0.145103] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[ 0.145120] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[ 0.145515] Last level iTLB entries: 4KB 512, 2MB 1024, 4MB 512
[ 0.145518] Last level dTLB entries: 4KB 1024, 2MB 1024, 4MB 512, 1GB 0
[ 0.145522] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization
[ 0.145525] Spectre V2 : Mitigation: Retpolines
[ 0.145526] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch
[ 0.145526] Spectre V2 : Spectre v2 / SpectreRSB : Filling RSB on VMEXIT
[ 0.145527] Spectre V2 : Enabling Speculation Barrier for firmware calls
[ 0.145528] RETBleed: Mitigation: untrained return thunk
[ 0.145530] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier
[ 0.145532] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl
[ 0.149973] Freeing SMP alternatives memory: 32K
[ 0.258112] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD Graphics (family: 0x15, model: 0x13, stepping: 0x1)
[ 0.258348] cblist_init_generic: Setting adjustable number of callback queues.
[ 0.258350] cblist_init_generic: Setting shift to 1 and lim to 1.
[ 0.258380] cblist_init_generic: Setting shift to 1 and lim to 1.
[ 0.258406] cblist_init_generic: Setting shift to 1 and lim to 1.
[ 0.258433] Performance Events: Fam15h core perfctr, AMD PMU driver.
[ 0.258455] ... version: 0
[ 0.258456] ... bit width: 48
[ 0.258457] ... generic registers: 6
[ 0.258458] ... value mask: 0000ffffffffffff
[ 0.258459] ... max period: 00007fffffffffff
[ 0.258460] ... fixed-purpose events: 0
[ 0.258461] ... event mask: 000000000000003f
[ 0.258581] rcu: Hierarchical SRCU implementation.
[ 0.258582] rcu: Max phase no-delay instances is 1000.
[ 0.259173] NMI watchdog: Enabled. Permanently consumes one hw-PMU counter.
[ 0.259247] smp: Bringing up secondary CPUs ...
[ 0.259446] x86: Booting SMP configuration:
[ 0.259448] .... node #0, CPUs: #1
[ 0.259453] smpboot: Kicking AP alive: 17
[ 10.260918] CPU1 failed to report alive state
[ 10.260998] smp: Brought up 1 node, 1 CPU
[ 10.261000] smpboot: Max logical packages: 2
[ 10.261001] smpboot: Total of 1 processors activated (7801.09 BogoMIPS)
[ 10.261532] devtmpfs: initialized
[ 10.261628] x86/mm: Memory block size: 128MB
[ 10.262718] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[ 10.262726] futex hash table entries: 512 (order: 3, 32768 bytes, linear)
[ 10.262817] pinctrl core: initialized pinctrl subsystem
[ 10.262889] PM: RTC time: 16:25:24, date: 2023-04-19
[ 10.263636] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[ 10.263861] audit: initializing netlink subsys (disabled)
[ 10.264118] thermal_sys: Registered thermal governor 'fair_share'
[ 10.264120] thermal_sys: Registered thermal governor 'bang_bang'
[ 10.264121] thermal_sys: Registered thermal governor 'step_wise'
[ 10.264122] thermal_sys: Registered thermal governor 'user_space'
[ 10.264141] cpuidle: using governor ladder
[ 10.264146] cpuidle: using governor menu
[ 10.264357] PCI: MMCONFIG for domain 0000 [bus 00-3f] at [mem 0xf8000000-0xfbffffff] (base 0xf8000000)
[ 10.264362] PCI: MMCONFIG at [mem 0xf8000000-0xfbffffff] reserved as E820 entry
[ 10.264374] PCI: Using configuration type 1 for base access
[ 10.264574] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.
[ 10.268993] audit: type=2000 audit(1681921524.140:1): state=initialized audit_enabled=0 res=1
[ 10.281009] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages
[ 10.281012] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page
[ 10.281014] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages
[ 10.281015] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page
[ 10.285864] cryptd: max_cpu_qlen set to 1000
[ 10.289188] ACPI: Added _OSI(Module Device)
[ 10.289191] ACPI: Added _OSI(Processor Device)
[ 10.289192] ACPI: Added _OSI(3.0 _SCP Extensions)
[ 10.289193] ACPI: Added _OSI(Processor Aggregator Device)
[ 10.295203] ACPI: 4 ACPI AML tables successfully acquired and loaded
[ 10.296702] ACPI: Interpreter enabled
[ 10.296727] ACPI: PM: (supports S0 S1 S3 S5)
[ 10.296728] ACPI: Using IOAPIC for interrupt routing
[ 10.296779] HEST: Table parsing has been initialized.
[ 10.296800] GHES: Failed to enable APEI firmware first mode.
[ 10.296803] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[ 10.296804] PCI: Ignoring E820 reservations for host bridge windows
[ 10.297084] ACPI: Enabled 8 GPEs in block 00 to 1F
[ 10.302898] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[ 10.302909] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]
[ 10.302987] acpi PNP0A03:00: _OSC: OS now controls [PME AER PCIeCapability LTR]
[ 10.303000] acpi PNP0A03:00: [Firmware Info]: MMCONFIG for domain 0000 [bus 00-3f] only partially covers this bridge
[ 10.303079] acpi PNP0A03:00: host bridge window expanded to [io 0x0000-0x0cf7 window]; [io 0x03b0-0x03df window] ignored
[ 10.303311] PCI host bridge to bus 0000:00
[ 10.303313] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]
[ 10.303316] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]
[ 10.303318] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000dffff]
[ 10.303320] pci_bus 0000:00: root bus resource [mem 0x80000000-0xffffffff]
[ 10.303322] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 10.303345] pci 0000:00:00.0: [1022:1410] type 00 class 0x060000
[ 10.303490] pci 0000:00:00.2: [1022:1419] type 00 class 0x080600
[ 10.303580] pci 0000:00:01.0: [1002:9996] type 00 class 0x030000
[ 10.303588] pci 0000:00:01.0: reg 0x10: [mem 0xe0000000-0xefffffff pref]
[ 10.303593] pci 0000:00:01.0: reg 0x14: [io 0x1000-0x10ff]
[ 10.303598] pci 0000:00:01.0: reg 0x18: [mem 0xf0180000-0xf01bffff]
[ 10.303614] pci 0000:00:01.0: enabling Extended Tags
[ 10.303625] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]
[ 10.303642] pci 0000:00:01.0: supports D1 D2
[ 10.303706] pci 0000:00:01.1: [1002:9902] type 00 class 0x040300
[ 10.303714] pci 0000:00:01.1: reg 0x10: [mem 0xf01c0000-0xf01c3fff]
[ 10.303735] pci 0000:00:01.1: enabling Extended Tags
[ 10.303759] pci 0000:00:01.1: supports D1 D2
[ 10.303842] pci 0000:00:11.0: [1022:7801] type 00 class 0x010601
[ 10.303855] pci 0000:00:11.0: reg 0x10: [io 0x1410-0x1417]
[ 10.303863] pci 0000:00:11.0: reg 0x14: [io 0x1420-0x1423]
[ 10.303870] pci 0000:00:11.0: reg 0x18: [io 0x1418-0x141f]
[ 10.303878] pci 0000:00:11.0: reg 0x1c: [io 0x1424-0x1427]
[ 10.303885] pci 0000:00:11.0: reg 0x20: [io 0x1400-0x140f]
[ 10.303892] pci 0000:00:11.0: reg 0x24: [mem 0xf01cc000-0xf01cc7ff]
[ 10.304038] pci 0000:00:12.0: [1022:7807] type 00 class 0x0c0310
[ 10.304051] pci 0000:00:12.0: reg 0x10: [mem 0xf01c8000-0xf01c8fff]
[ 10.304220] pci 0000:00:12.2: [1022:7808] type 00 class 0x0c0320
[ 10.304234] pci 0000:00:12.2: reg 0x10: [mem 0xf01cd000-0xf01cd0ff]
[ 10.304299] pci 0000:00:12.2: supports D1 D2
[ 10.304300] pci 0000:00:12.2: PME# supported from D0 D1 D2 D3hot
[ 10.304432] pci 0000:00:13.0: [1022:7807] type 00 class 0x0c0310
[ 10.304446] pci 0000:00:13.0: reg 0x10: [mem 0xf01c9000-0xf01c9fff]
[ 10.304614] pci 0000:00:13.2: [1022:7808] type 00 class 0x0c0320
[ 10.304627] pci 0000:00:13.2: reg 0x10: [mem 0xf01ce000-0xf01ce0ff]
[ 10.304692] pci 0000:00:13.2: supports D1 D2
[ 10.304693] pci 0000:00:13.2: PME# supported from D0 D1 D2 D3hot
[ 10.304823] pci 0000:00:14.0: [1022:780b] type 00 class 0x0c0500
[ 10.305554] pci 0000:00:14.2: [1022:780d] type 00 class 0x040300
[ 10.305573] pci 0000:00:14.2: reg 0x10: [mem 0xf01c4000-0xf01c7fff 64bit]
[ 10.305633] pci 0000:00:14.2: PME# supported from D0 D3hot D3cold
[ 10.305769] pci 0000:00:14.3: [1022:780e] type 00 class 0x060100
[ 10.305945] pci 0000:00:14.4: [1022:780f] type 01 class 0x060401
[ 10.306090] pci 0000:00:14.5: [1022:7809] type 00 class 0x0c0310
[ 10.306104] pci 0000:00:14.5: reg 0x10: [mem 0xf01ca000-0xf01cafff]
[ 10.306269] pci 0000:00:15.0: [1022:43a0] type 01 class 0x060400
[ 10.306298] pci 0000:00:15.0: enabling Extended Tags
[ 10.306338] pci 0000:00:15.0: supports D1 D2
[ 10.306499] pci 0000:00:15.1: [1022:43a1] type 01 class 0x060400
[ 10.306530] pci 0000:00:15.1: enabling Extended Tags
[ 10.306569] pci 0000:00:15.1: supports D1 D2
[ 10.306725] pci 0000:00:15.2: [1022:43a2] type 01 class 0x060400
[ 10.306752] pci 0000:00:15.2: enabling Extended Tags
[ 10.306791] pci 0000:00:15.2: supports D1 D2
[ 10.306866] pci 0000:00:16.0: [1022:7807] type 00 class 0x0c0310
[ 10.306879] pci 0000:00:16.0: reg 0x10: [mem 0xf01cb000-0xf01cbfff]
[ 10.307057] pci 0000:00:16.2: [1022:7808] type 00 class 0x0c0320
[ 10.307070] pci 0000:00:16.2: reg 0x10: [mem 0xf01cf000-0xf01cf0ff]
[ 10.307135] pci 0000:00:16.2: supports D1 D2
[ 10.307136] pci 0000:00:16.2: PME# supported from D0 D1 D2 D3hot
[ 10.307278] pci 0000:00:18.0: [1022:1400] type 00 class 0x060000
[ 10.307342] pci 0000:00:18.1: [1022:1401] type 00 class 0x060000
[ 10.307402] pci 0000:00:18.2: [1022:1402] type 00 class 0x060000
[ 10.307467] pci 0000:00:18.3: [1022:1403] type 00 class 0x060000
[ 10.307600] pci 0000:00:18.4: [1022:1404] type 00 class 0x060000
[ 10.307663] pci 0000:00:18.5: [1022:1405] type 00 class 0x060000
[ 10.307738] pci_bus 0000:01: extended config space not accessible
[ 10.307804] pci 0000:00:14.4: PCI bridge to [bus 01] (subtractive decode)
[ 10.307813] pci 0000:00:14.4: bridge window [io 0x0000-0x0cf7 window] (subtractive decode)
[ 10.307816] pci 0000:00:14.4: bridge window [io 0x0d00-0xffff window] (subtractive decode)
[ 10.307818] pci 0000:00:14.4: bridge window [mem 0x000a0000-0x000dffff] (subtractive decode)
[ 10.307820] pci 0000:00:14.4: bridge window [mem 0x80000000-0xffffffff] (subtractive decode)
[ 10.307868] pci 0000:00:15.0: PCI bridge to [bus 02]
[ 10.307953] pci 0000:03:00.0: [1b21:1042] type 00 class 0x0c0330
[ 10.307989] pci 0000:03:00.0: reg 0x10: [mem 0xf0000000-0xf0007fff 64bit]
[ 10.308167] pci 0000:03:00.0: PME# supported from D3hot D3cold
[ 10.308210] pci 0000:03:00.0: 2.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s PCIe x1 link at 0000:00:15.1 (capable of 4.000 Gb/s with 5.0 GT/s PCIe x1 link)
[ 10.317047] pci 0000:00:15.1: PCI bridge to [bus 03]
[ 10.317059] pci 0000:00:15.1: bridge window [mem 0xf0000000-0xf00fffff]
[ 10.317068] pci 0000:00:15.2: bridge configuration invalid ([bus 00-00]), reconfiguring
[ 10.317188] pci 0000:04:00.0: [10ec:8168] type 00 class 0x020000
[ 10.317206] pci 0000:04:00.0: reg 0x10: [io 0x0000-0x00ff]
[ 10.317227] pci 0000:04:00.0: reg 0x18: [mem 0x00000000-0x00000fff 64bit pref]
[ 10.317241] pci 0000:04:00.0: reg 0x20: [mem 0x00000000-0x00003fff 64bit pref]
[ 10.317349] pci 0000:04:00.0: supports D1 D2
[ 10.317351] pci 0000:04:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[ 10.328970] pci 0000:00:15.2: PCI bridge to [bus 04-ff]
[ 10.328981] pci 0000:00:15.2: bridge window [io 0x0000-0x0fff]
[ 10.328985] pci 0000:00:15.2: bridge window [mem 0x00000000-0x000fffff]
[ 10.328989] pci 0000:00:15.2: bridge window [mem 0x00000000-0x000fffff 64bit pref]
[ 10.328993] pci_bus 0000:04: busn_res: [bus 04-ff] end is updated to 04
[ 10.329502] ACPI: PCI: Interrupt link INTA configured for IRQ 0
[ 10.329596] ACPI: PCI: Interrupt link INTB configured for IRQ 0
[ 10.329687] ACPI: PCI: Interrupt link INTC configured for IRQ 0
[ 10.329777] ACPI: PCI: Interrupt link INTD configured for IRQ 0
[ 10.329869] ACPI: PCI: Interrupt link INTE configured for IRQ 0
[ 10.329959] ACPI: PCI: Interrupt link INTF configured for IRQ 0
[ 10.330051] ACPI: PCI: Interrupt link INTG configured for IRQ 0
[ 10.330141] ACPI: PCI: Interrupt link INTH configured for IRQ 0
[ 10.330370] iommu: Default domain type: Translated
[ 10.330372] iommu: DMA domain TLB invalidation policy: lazy mode
[ 10.330551] SCSI subsystem initialized
[ 10.330635] libata version 3.00 loaded.
[ 10.330668] ACPI: bus type USB registered
[ 10.330692] usbcore: registered new interface driver usbfs
[ 10.330702] usbcore: registered new interface driver hub
[ 10.330715] usbcore: registered new device driver usb
[ 10.331044] PCI: Using ACPI for IRQ routing
[ 10.332613] PCI: pci_cache_line_size set to 64 bytes
[ 10.332664] e820: reserve RAM buffer [mem 0x0009fc00-0x0009ffff]
[ 10.332667] e820: reserve RAM buffer [mem 0x5fe3d000-0x5fffffff]
[ 10.332669] e820: reserve RAM buffer [mem 0x17f000000-0x17fffffff]
[ 10.332714] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0
[ 10.332719] hpet0: 3 comparators, 32-bit 14.318180 MHz counter
[ 10.333992] clocksource: Switched to clocksource tsc-early
[ 10.350982] VFS: Disk quotas dquot_6.6.0
[ 10.351012] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[ 10.351123] pnp: PnP ACPI init
[ 10.351416] system 00:00: [mem 0xfec10002-0xfec11001] could not be reserved
[ 10.351719] pnp: PnP ACPI: found 2 devices
[ 10.358870] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
[ 10.359003] NET: Registered PF_INET protocol family
[ 10.359150] IP idents hash table entries: 65536 (order: 7, 524288 bytes, linear)
[ 10.360716] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear)
[ 10.360731] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
[ 10.360739] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear)
[ 10.360805] TCP bind hash table entries: 32768 (order: 8, 1048576 bytes, linear)
[ 10.361256] TCP: Hash tables configured (established 32768 bind 32768)
[ 10.361327] UDP hash table entries: 2048 (order: 4, 65536 bytes, linear)
[ 10.361347] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear)
[ 10.361453] NET: Registered PF_UNIX/PF_LOCAL protocol family
[ 10.361487] pci 0000:00:15.2: BAR 15: assigned [mem 0x80000000-0x800fffff 64bit pref]
[ 10.361492] pci 0000:00:15.2: BAR 13: assigned [io 0x2000-0x2fff]
[ 10.361496] pci 0000:00:14.4: PCI bridge to [bus 01]
[ 10.361507] pci 0000:00:15.0: PCI bridge to [bus 02]
[ 10.361515] pci 0000:00:15.1: PCI bridge to [bus 03]
[ 10.361519] pci 0000:00:15.1: bridge window [mem 0xf0000000-0xf00fffff]
[ 10.361526] pci 0000:04:00.0: BAR 4: assigned [mem 0x80000000-0x80003fff 64bit pref]
[ 10.361539] pci 0000:04:00.0: BAR 2: assigned [mem 0x80004000-0x80004fff 64bit pref]
[ 10.361550] pci 0000:04:00.0: BAR 0: assigned [io 0x2000-0x20ff]
[ 10.361555] pci 0000:00:15.2: PCI bridge to [bus 04]
[ 10.361557] pci 0000:00:15.2: bridge window [io 0x2000-0x2fff]
[ 10.361562] pci 0000:00:15.2: bridge window [mem 0x80000000-0x800fffff 64bit pref]
[ 10.361569] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]
[ 10.361570] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]
[ 10.361572] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000dffff]
[ 10.361574] pci_bus 0000:00: resource 7 [mem 0x80000000-0xffffffff]
[ 10.361575] pci_bus 0000:01: resource 4 [io 0x0000-0x0cf7 window]
[ 10.361577] pci_bus 0000:01: resource 5 [io 0x0d00-0xffff window]
[ 10.361578] pci_bus 0000:01: resource 6 [mem 0x000a0000-0x000dffff]
[ 10.361580] pci_bus 0000:01: resource 7 [mem 0x80000000-0xffffffff]
[ 10.361582] pci_bus 0000:03: resource 1 [mem 0xf0000000-0xf00fffff]
[ 10.361583] pci_bus 0000:04: resource 0 [io 0x2000-0x2fff]
[ 10.361584] pci_bus 0000:04: resource 2 [mem 0x80000000-0x800fffff 64bit pref]
[ 10.361673] pci 0000:00:01.1: D0 power state depends on 0000:00:01.0
[ 10.362350] pci 0000:00:12.2: PME# does not work under D3, disabling it
[ 10.362915] pci 0000:00:13.2: PME# does not work under D3, disabling it
[ 10.363777] pci 0000:00:16.2: PME# does not work under D3, disabling it
[ 10.364048] PCI: CLS 64 bytes, default 64
[ 10.364162] pci 0000:00:00.2: AMD-Vi: Applying erratum 746 workaround
[ 10.364245] pci 0000:00:01.0: Adding to iommu group 0
[ 10.364267] pci 0000:00:01.1: Adding to iommu group 0
[ 10.364290] pci 0000:00:11.0: Adding to iommu group 1
[ 10.364323] pci 0000:00:12.0: Adding to iommu group 2
[ 10.364345] pci 0000:00:12.2: Adding to iommu group 2
[ 10.364377] pci 0000:00:13.0: Adding to iommu group 3
[ 10.364395] pci 0000:00:13.2: Adding to iommu group 3
[ 10.364433] pci 0000:00:14.0: Adding to iommu group 4
[ 10.364450] pci 0000:00:14.2: Adding to iommu group 4
[ 10.364468] pci 0000:00:14.3: Adding to iommu group 4
[ 10.364488] pci 0000:00:14.4: Adding to iommu group 5
[ 10.364512] pci 0000:00:14.5: Adding to iommu group 6
[ 10.364549] pci 0000:00:15.0: Adding to iommu group 7
[ 10.364566] pci 0000:00:15.1: Adding to iommu group 7
[ 10.364585] pci 0000:00:15.2: Adding to iommu group 7
[ 10.364617] pci 0000:00:16.0: Adding to iommu group 8
[ 10.364634] pci 0000:00:16.2: Adding to iommu group 8
[ 10.364690] pci 0000:00:18.0: Adding to iommu group 9
[ 10.364708] pci 0000:00:18.1: Adding to iommu group 9
[ 10.364729] pci 0000:00:18.2: Adding to iommu group 9
[ 10.364746] pci 0000:00:18.3: Adding to iommu group 9
[ 10.364766] pci 0000:00:18.4: Adding to iommu group 9
[ 10.364784] pci 0000:00:18.5: Adding to iommu group 9
[ 10.364792] pci 0000:03:00.0: Adding to iommu group 7
[ 10.364806] pci 0000:04:00.0: Adding to iommu group 7
[ 10.366937] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[ 10.366942] AMD-Vi: Extended features (0x800000853, 0x0): PreF PPR GT IA
[ 10.366948] AMD-Vi: Interrupt remapping enabled
[ 10.367131] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[ 10.367132] software IO TLB: mapped [mem 0x000000005be3d000-0x000000005fe3d000] (64MB)
[ 10.367182] LVT offset 0 assigned for vector 0x400
[ 10.367203] perf: AMD IBS detected (0x000000ff)
[ 10.367211] amd_uncore: 4 amd_nb counters detected
[ 10.368017] workingset: timestamp_bits=37 max_order=20 bucket_order=0
[ 10.368047] zbud: loaded
[ 10.368508] NET: Registered PF_ALG protocol family
[ 10.368512] Key type asymmetric registered
[ 10.368514] Asymmetric key parser 'x509' registered
[ 10.368783] alg: self-tests disabled
[ 10.368876] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 251)
[ 10.369320] io scheduler mq-deadline registered
[ 10.369322] io scheduler kyber registered
[ 10.370447] pcieport 0000:00:15.0: PME: Signaling with IRQ 25
[ 10.370608] pcieport 0000:00:15.1: PME: Signaling with IRQ 26
[ 10.370680] pcieport 0000:00:15.2: enabling device (0000 -> 0003)
[ 10.370883] pcieport 0000:00:15.2: PME: Signaling with IRQ 27
[ 10.371138] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
[ 10.371201] ACPI: button: Power Button [PWRF]
[ 10.371253] ACPI: \_SB_.P000: Found 2 idle states
[ 10.371369] ACPI: \_SB_.P001: Found 2 idle states
[ 10.372255] thermal LNXTHERM:00: registered as thermal_zone0
[ 10.372258] ACPI: thermal: Thermal Zone [TZ00] (0 C)
[ 10.372573] Non-volatile memory driver v1.3
[ 10.372648] AMD-Vi: AMD IOMMUv2 loaded and initialized
[ 10.372667] ACPI: bus type drm_connector registered
[ 10.372799] ahci 0000:00:11.0: version 3.0
[ 10.373109] ahci 0000:00:11.0: AHCI 0001.0300 32 slots 8 ports 6 Gbps 0x40 impl SATA mode
[ 10.373113] ahci 0000:00:11.0: flags: 64bit ncq sntf ilck led clo pio
[ 10.374331] scsi host0: ahci
[ 10.374531] scsi host1: ahci
[ 10.374704] scsi host2: ahci
[ 10.374896] scsi host3: ahci
[ 10.375076] scsi host4: ahci
[ 10.375265] scsi host5: ahci
[ 10.375445] scsi host6: ahci
[ 10.375622] scsi host7: ahci
[ 10.375714] ata1: DUMMY
[ 10.375715] ata2: DUMMY
[ 10.375716] ata3: DUMMY
[ 10.375717] ata4: DUMMY
[ 10.375718] ata5: DUMMY
[ 10.375718] ata6: DUMMY
[ 10.375720] ata7: SATA max UDMA/133 abar m2048@0xf01cc000 port 0xf01cc400 irq 19
[ 10.375722] ata8: DUMMY
[ 10.375982] i8042: PNP: No PS/2 controller found.
[ 10.375983] i8042: Probing ports directly.
[ 10.378529] serio: i8042 KBD port at 0x60,0x64 irq 1
[ 10.378595] serio: i8042 AUX port at 0x60,0x64 irq 12
[ 10.378708] mousedev: PS/2 mouse device common for all mice
[ 10.378760] rtc_cmos 00:01: RTC can wake from S4
[ 10.379002] rtc_cmos 00:01: registered as rtc0
[ 10.379026] rtc_cmos 00:01: setting system clock to 2023-04-19T16:25:24 UTC (1681921524)
[ 10.379067] rtc_cmos 00:01: alarms up to one day, y3k, 114 bytes nvram, hpet irqs
[ 10.379101] device-mapper: uevent: version 1.0.3
[ 10.379167] device-mapper: ioctl: 4.47.0-ioctl (2022-07-28) initialised: dm-devel@redhat.com
[ 10.379323] hid: raw HID events driver (C) Jiri Kosina
[ 10.379363] usbcore: registered new interface driver usbhid
[ 10.379364] usbhid: USB HID core driver
[ 10.379457] Initializing XFRM netlink socket
[ 10.379465] NET: Registered PF_PACKET protocol family
[ 10.379467] x86/pm: family 0x15 cpu detected, MSR saving is needed during suspending.
[ 10.379633] microcode: CPU0: patch_level=0x0600111f
[ 10.379642] microcode: Microcode Update Driver: v2.2.
[ 10.379646] IPI shorthand broadcast: enabled
[ 10.379654] AVX version of gcm_enc/dec engaged.
[ 10.379670] AES CTR mode by8 optimization enabled
[ 10.383650] sched_clock: Marking stable (10264006663, 116905762)->(10383417207, -2504782)
[ 10.383836] registered taskstats version 1
[ 10.384079] zswap: loaded using pool lzo/zbud
[ 10.388276] kmemleak: Kernel memory leak detector initialized (mem pool available: 15679)
[ 10.388282] debug_vm_pgtable: [debug_vm_pgtable ]: Validating architecture page table helpers
[ 10.391487] kmemleak: Automatic memory scanning thread started
[ 10.393329] Key type encrypted registered
[ 10.396095] PM: Magic number: 3:487:439
[ 10.488448] ata7: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[ 10.488603] ata7.00: ATA-9: SanDisk SDSSDP064G, 2.0.0, max UDMA/133
[ 10.488606] ata7.00: 125045424 sectors, multi 1: LBA48 NCQ (depth 32)
[ 10.488805] ata7.00: configured for UDMA/133
[ 10.489003] scsi 6:0:0:0: Direct-Access ATA SanDisk SDSSDP06 0 PQ: 0 ANSI: 5
[ 10.490061] sd 6:0:0:0: [sda] 125045424 512-byte logical blocks: (64.0 GB/59.6 GiB)
[ 10.490078] sd 6:0:0:0: [sda] Write Protect is off
[ 10.490083] sd 6:0:0:0: [sda] Mode Sense: 00 3a 00 00
[ 10.490105] sd 6:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 10.490136] sd 6:0:0:0: [sda] Preferred minimum I/O size 512 bytes
[ 10.491510] sda: sda1 sda2 sda3
[ 10.491950] sd 6:0:0:0: [sda] Attached SCSI disk
[ 10.504047] EXT4-fs (sda3): mounted filesystem fe29e0dc-6303-4401-987c-8472bc1b9516 with ordered data mode. Quota mode: none.
[ 10.504090] VFS: Mounted root (ext4 filesystem) on device 8:3.
[ 10.506047] devtmpfs: mounted
[ 10.510012] Freeing unused kernel image (initmem) memory: 2908K
[ 10.517060] Write protecting the kernel read-only data: 20480k
[ 10.517319] Freeing unused kernel image (rodata/data gap) memory: 836K
[ 10.554385] x86/mm: Checked W+X mappings: passed, no W+X pages found.
[ 10.554391] rodata_test: all tests were successful
[ 10.554413] Run /sbin/init as init process
[ 10.554415] with arguments:
[ 10.554416] /sbin/init
[ 10.554417] noisapnp
[ 10.554418] with environment:
[ 10.554419] HOME=/
[ 10.554420] TERM=linux
[ 10.554420] BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc3-00045-g64de4df9c80b
[ 10.757895] systemd[1]: Inserted module 'autofs4'
[ 10.785886] NET: Registered PF_INET6 protocol family
[ 10.786731] Segment Routing with IPv6
[ 10.786760] In-situ OAM (IOAM) with IPv6
[ 10.812460] systemd[1]: systemd 252.6-1 running in system mode (+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
[ 10.812470] systemd[1]: Detected architecture x86-64.
[ 10.816810] systemd[1]: Hostname set to <kodi>.
[ 11.113206] systemd[1]: Queued start job for default target graphical.target.
[ 11.123550] systemd[1]: Created slice system-getty.slice - Slice /system/getty.
[ 11.124636] systemd[1]: Created slice system-modprobe.slice - Slice /system/modprobe.
[ 11.125478] systemd[1]: Created slice user.slice - User and Session Slice.
[ 11.125653] systemd[1]: Started systemd-ask-password-console.path - Dispatch Password Requests to Console Directory Watch.
[ 11.125780] systemd[1]: Started systemd-ask-password-wall.path - Forward Password Requests to Wall Directory Watch.
[ 11.126204] systemd[1]: Set up automount proc-sys-fs-binfmt_misc.automount - Arbitrary Executable File Formats File System Automount Point.
[ 11.126240] systemd[1]: Reached target cryptsetup.target - Local Encrypted Volumes.
[ 11.126284] systemd[1]: Reached target integritysetup.target - Local Integrity Protected Volumes.
[ 11.126331] systemd[1]: Reached target paths.target - Path Units.
[ 11.126362] systemd[1]: Reached target remote-fs.target - Remote File Systems.
[ 11.126389] systemd[1]: Reached target slices.target - Slice Units.
[ 11.126427] systemd[1]: Reached target swap.target - Swaps.
[ 11.126468] systemd[1]: Reached target veritysetup.target - Local Verity Protected Volumes.
[ 11.128962] systemd[1]: Listening on systemd-coredump.socket - Process Core Dump Socket.
[ 11.129221] systemd[1]: Listening on systemd-fsckd.socket - fsck to fsckd communication Socket.
[ 11.129381] systemd[1]: Listening on systemd-initctl.socket - initctl Compatibility Named Pipe.
[ 11.129690] systemd[1]: Listening on systemd-journald-audit.socket - Journal Audit Socket.
[ 11.129960] systemd[1]: Listening on systemd-journald-dev-log.socket - Journal Socket (/dev/log).
[ 11.130239] systemd[1]: Listening on systemd-journald.socket - Journal Socket.
[ 11.130488] systemd[1]: Listening on systemd-networkd.socket - Network Service Netlink Socket.
[ 11.131308] systemd[1]: Listening on systemd-udevd-control.socket - udev Control Socket.
[ 11.131580] systemd[1]: Listening on systemd-udevd-kernel.socket - udev Kernel Socket.
[ 11.134358] systemd[1]: Mounting dev-hugepages.mount - Huge Pages File System...
[ 11.136807] systemd[1]: Mounting dev-mqueue.mount - POSIX Message Queue File System...
[ 11.140733] systemd[1]: Mounting sys-kernel-debug.mount - Kernel Debug File System...
[ 11.152693] systemd[1]: Mounting sys-kernel-tracing.mount - Kernel Trace File System...
[ 11.159215] systemd[1]: Starting kmod-static-nodes.service - Create List of Static Device Nodes...
[ 11.167969] systemd[1]: Starting modprobe@configfs.service - Load Kernel Module configfs...
[ 11.185265] systemd[1]: Starting modprobe@dm_mod.service - Load Kernel Module dm_mod...
[ 11.187904] systemd[1]: Starting modprobe@drm.service - Load Kernel Module drm...
[ 11.194981] systemd[1]: Starting modprobe@efi_pstore.service - Load Kernel Module efi_pstore...
[ 11.202200] systemd[1]: Starting modprobe@fuse.service - Load Kernel Module fuse...
[ 11.213235] systemd[1]: Starting modprobe@loop.service - Load Kernel Module loop...
[ 11.213303] systemd[1]: systemd-firstboot.service - First Boot Wizard was skipped because of an unmet condition check (ConditionFirstBoot=yes).
[ 11.213372] systemd[1]: systemd-fsck-root.service - File System Check on Root Device was skipped because of an unmet condition check (ConditionPathIsReadWrite=!/).
[ 11.213409] systemd[1]: Reached target local-fs.target - Local File Systems.
[ 11.213471] systemd[1]: apparmor.service - Load AppArmor profiles was skipped because of an unmet condition check (ConditionSecurity=apparmor).
[ 11.217387] systemd[1]: Starting systemd-binfmt.service - Set Up Additional Binary Formats...
[ 11.225611] loop: module loaded
[ 11.227933] fuse: init (API version 7.38)
[ 11.241394] systemd[1]: Starting systemd-journald.service - Journal Service...
[ 11.244008] systemd[1]: Starting systemd-random-seed.service - Load/Save Random Seed...
[ 11.265213] systemd[1]: Starting systemd-sysctl.service - Apply Kernel Variables...
[ 11.267873] systemd[1]: Starting systemd-sysusers.service - Create System Users...
[ 11.294059] systemd[1]: Starting systemd-udev-trigger.service - Coldplug All udev Devices...
[ 11.316096] systemd[1]: Mounted dev-hugepages.mount - Huge Pages File System.
[ 11.316293] systemd[1]: Mounted dev-mqueue.mount - POSIX Message Queue File System.
[ 11.316467] systemd[1]: Mounted sys-kernel-debug.mount - Kernel Debug File System.
[ 11.316640] systemd[1]: Mounted sys-kernel-tracing.mount - Kernel Trace File System.
[ 11.337994] systemd[1]: Finished kmod-static-nodes.service - Create List of Static Device Nodes.
[ 11.338790] systemd[1]: modprobe@configfs.service: Deactivated successfully.
[ 11.341622] systemd[1]: Finished modprobe@configfs.service - Load Kernel Module configfs.
[ 11.342285] systemd[1]: modprobe@dm_mod.service: Deactivated successfully.
[ 11.342546] systemd[1]: Finished modprobe@dm_mod.service - Load Kernel Module dm_mod.
[ 11.343122] systemd[1]: modprobe@drm.service: Deactivated successfully.
[ 11.343364] systemd[1]: Finished modprobe@drm.service - Load Kernel Module drm.
[ 11.343928] systemd[1]: modprobe@efi_pstore.service: Deactivated successfully.
[ 11.344176] systemd[1]: Finished modprobe@efi_pstore.service - Load Kernel Module efi_pstore.
[ 11.344734] systemd[1]: modprobe@fuse.service: Deactivated successfully.
[ 11.358905] systemd[1]: Finished modprobe@fuse.service - Load Kernel Module fuse.
[ 11.359616] systemd[1]: modprobe@loop.service: Deactivated successfully.
[ 11.365358] systemd[1]: Finished modprobe@loop.service - Load Kernel Module loop.
[ 11.366685] systemd[1]: Finished systemd-sysctl.service - Apply Kernel Variables.
[ 11.367634] systemd[1]: Finished systemd-sysusers.service - Create System Users.
[ 11.368150] systemd[1]: proc-sys-fs-binfmt_misc.automount: Got automount request for /proc/sys/fs/binfmt_misc, triggered by 133 (systemd-binfmt)
[ 11.382441] systemd[1]: Mounting proc-sys-fs-binfmt_misc.mount - Arbitrary Executable File Formats File System...
[ 11.385250] tsc: Refined TSC clocksource calibration: 3900.225 MHz
[ 11.385258] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x70705d7a9ae, max_idle_ns: 881590495532 ns
[ 11.385270] clocksource: Switched to clocksource tsc
[ 11.429139] systemd[1]: Mounting sys-fs-fuse-connections.mount - FUSE Control File System...
[ 11.442192] systemd[1]: Mounting sys-kernel-config.mount - Kernel Configuration File System...
[ 11.442320] systemd[1]: systemd-pstore.service - Platform Persistent Storage Archival was skipped because of an unmet condition check (ConditionDirectoryNotEmpty=/sys/fs/pstore).
[ 11.442474] systemd[1]: systemd-repart.service - Repartition Root Disk was skipped because no trigger condition checks were met.
[ 11.465930] systemd[1]: Starting systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev...
[ 11.467355] systemd[1]: Mounted proc-sys-fs-binfmt_misc.mount - Arbitrary Executable File Formats File System.
[ 11.485939] systemd[1]: Finished systemd-binfmt.service - Set Up Additional Binary Formats.
[ 11.487783] systemd[1]: Mounted sys-fs-fuse-connections.mount - FUSE Control File System.
[ 11.487969] systemd[1]: Mounted sys-kernel-config.mount - Kernel Configuration File System.
[ 11.554727] systemd[1]: Finished systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev.
[ 11.565290] systemd[1]: Starting systemd-udevd.service - Rule-based Manager for Device Events and Files...
[ 11.570809] systemd[1]: Started systemd-journald.service - Journal Service.
[ 11.617282] systemd-journald[134]: Received client request to flush runtime journal.
[ 12.118033] sd 6:0:0:0: Attached scsi generic sg0 type 0
[ 12.224922] random: crng init done
[ 12.412498] acpi_cpufreq: overriding BIOS provided _PSD data
[ 12.561374] QUIRK: Enable AMD PLL fix
[ 12.561422] ehci-pci 0000:00:12.2: EHCI Host Controller
[ 12.561449] ehci-pci 0000:00:12.2: new USB bus registered, assigned bus number 1
[ 12.561461] ehci-pci 0000:00:12.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
[ 12.561469] ehci-pci 0000:00:12.2: debug port 1
[ 12.561640] ehci-pci 0000:00:12.2: irq 17, io mem 0xf01cd000
[ 12.576943] ehci-pci 0000:00:12.2: USB 2.0 started, EHCI 1.00
[ 12.577230] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 12.577233] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.577235] usb usb1: Product: EHCI Host Controller
[ 12.577236] usb usb1: Manufacturer: Linux 6.3.0-rc3-00045-g64de4df9c80b ehci_hcd
[ 12.577238] usb usb1: SerialNumber: 0000:00:12.2
[ 12.577680] hub 1-0:1.0: USB hub found
[ 12.577708] hub 1-0:1.0: 5 ports detected
[ 12.578363] ehci-pci 0000:00:13.2: EHCI Host Controller
[ 12.578382] ehci-pci 0000:00:13.2: new USB bus registered, assigned bus number 2
[ 12.578393] ehci-pci 0000:00:13.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
[ 12.578402] ehci-pci 0000:00:13.2: debug port 1
[ 12.578529] ehci-pci 0000:00:13.2: irq 17, io mem 0xf01ce000
[ 12.587170] piix4_smbus 0000:00:14.0: SMBus Host Controller at 0xb00, revision 0
[ 12.587177] piix4_smbus 0000:00:14.0: Using register 0x2e for SMBus port selection
[ 12.587654] piix4_smbus 0000:00:14.0: Auxiliary SMBus Host Controller at 0xb20
[ 12.592939] ehci-pci 0000:00:13.2: USB 2.0 started, EHCI 1.00
[ 12.593429] usb usb2: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 12.593433] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.593435] usb usb2: Product: EHCI Host Controller
[ 12.593437] usb usb2: Manufacturer: Linux 6.3.0-rc3-00045-g64de4df9c80b ehci_hcd
[ 12.593438] usb usb2: SerialNumber: 0000:00:13.2
[ 12.593929] hub 2-0:1.0: USB hub found
[ 12.593956] hub 2-0:1.0: 5 ports detected
[ 12.594622] ehci-pci 0000:00:16.2: EHCI Host Controller
[ 12.594640] ehci-pci 0000:00:16.2: new USB bus registered, assigned bus number 3
[ 12.594650] ehci-pci 0000:00:16.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
[ 12.594659] ehci-pci 0000:00:16.2: debug port 1
[ 12.594791] ehci-pci 0000:00:16.2: irq 17, io mem 0xf01cf000
[ 12.608936] ehci-pci 0000:00:16.2: USB 2.0 started, EHCI 1.00
[ 12.609220] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 12.609223] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.609225] usb usb3: Product: EHCI Host Controller
[ 12.609226] usb usb3: Manufacturer: Linux 6.3.0-rc3-00045-g64de4df9c80b ehci_hcd
[ 12.609228] usb usb3: SerialNumber: 0000:00:16.2
[ 12.609667] hub 3-0:1.0: USB hub found
[ 12.609694] hub 3-0:1.0: 4 ports detected
[ 12.611387] ohci-pci 0000:00:12.0: OHCI PCI host controller
[ 12.611420] ohci-pci 0000:00:12.0: new USB bus registered, assigned bus number 4
[ 12.611596] ohci-pci 0000:00:12.0: irq 18, io mem 0xf01c8000
[ 12.660463] r8169 0000:04:00.0: enabling device (0000 -> 0003)
[ 12.690367] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 12.690374] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.690376] usb usb4: Product: OHCI PCI host controller
[ 12.690378] usb usb4: Manufacturer: Linux 6.3.0-rc3-00045-g64de4df9c80b ohci_hcd
[ 12.690380] usb usb4: SerialNumber: 0000:00:12.0
[ 12.699141] hub 4-0:1.0: USB hub found
[ 12.703266] hub 4-0:1.0: 5 ports detected
[ 12.725731] ohci-pci 0000:00:13.0: OHCI PCI host controller
[ 12.725756] ohci-pci 0000:00:13.0: new USB bus registered, assigned bus number 5
[ 12.725865] ohci-pci 0000:00:13.0: irq 18, io mem 0xf01c9000
[ 12.727540] r8169 0000:04:00.0 eth0: RTL8168f/8111f, 08:60:6e:74:7a:51, XID 480, IRQ 28
[ 12.727545] r8169 0000:04:00.0 eth0: jumbo features [frames: 9194 bytes, tx checksumming: ko]
[ 12.741582] xhci_hcd 0000:03:00.0: xHCI Host Controller
[ 12.741607] xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 6
[ 12.748200] snd_hda_intel 0000:00:01.1: Force to non-snoop mode
[ 12.802855] xhci_hcd 0000:03:00.0: hcc params 0x0200f180 hci version 0x96 quirks 0x0000000000080010
[ 12.803664] xhci_hcd 0000:03:00.0: xHCI Host Controller
[ 12.803679] xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 7
[ 12.803692] xhci_hcd 0000:03:00.0: Host supports USB 3.0 SuperSpeed
[ 12.803860] usb usb6: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 12.803863] usb usb6: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.803865] usb usb6: Product: xHCI Host Controller
[ 12.803867] usb usb6: Manufacturer: Linux 6.3.0-rc3-00045-g64de4df9c80b xhci-hcd
[ 12.803868] usb usb6: SerialNumber: 0000:03:00.0
[ 12.807644] hub 6-0:1.0: USB hub found
[ 12.807677] hub 6-0:1.0: 2 ports detected
[ 12.809751] usb usb7: We don't know the algorithms for LPM for this host, disabling LPM.
[ 12.809885] usb usb7: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 6.03
[ 12.809888] usb usb7: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.809890] usb usb7: Product: xHCI Host Controller
[ 12.809891] usb usb7: Manufacturer: Linux 6.3.0-rc3-00045-g64de4df9c80b xhci-hcd
[ 12.809893] usb usb7: SerialNumber: 0000:03:00.0
[ 12.810353] hub 7-0:1.0: USB hub found
[ 12.810382] hub 7-0:1.0: 2 ports detected
[ 12.812712] input: HDA ATI HDMI HDMI/DP,pcm=3 as /devices/pci0000:00/0000:00:01.1/sound/card0/input1
[ 12.813027] input: HDA ATI HDMI HDMI/DP,pcm=7 as /devices/pci0000:00/0000:00:01.1/sound/card0/input2
[ 12.825660] snd_hda_codec_realtek hdaudioC1D0: ALC892: SKU not ready 0x00000100
[ 12.826527] snd_hda_codec_realtek hdaudioC1D0: autoconfig for ALC892: line_outs=4 (0x14/0x16/0x15/0x17/0x0) type:line
[ 12.826534] snd_hda_codec_realtek hdaudioC1D0: speaker_outs=0 (0x0/0x0/0x0/0x0/0x0)
[ 12.826536] snd_hda_codec_realtek hdaudioC1D0: hp_outs=1 (0x1b/0x0/0x0/0x0/0x0)
[ 12.826539] snd_hda_codec_realtek hdaudioC1D0: mono: mono_out=0x0
[ 12.826540] snd_hda_codec_realtek hdaudioC1D0: dig-out=0x1e/0x0
[ 12.826541] snd_hda_codec_realtek hdaudioC1D0: inputs:
[ 12.826543] snd_hda_codec_realtek hdaudioC1D0: Rear Mic=0x18
[ 12.826545] snd_hda_codec_realtek hdaudioC1D0: Front Mic=0x19
[ 12.826546] snd_hda_codec_realtek hdaudioC1D0: Line=0x1a
[ 12.826547] snd_hda_codec_realtek hdaudioC1D0: CD=0x1c
[ 12.854744] input: HD-Audio Generic Rear Mic as /devices/pci0000:00/0000:00:14.2/sound/card1/input3
[ 12.855037] input: HD-Audio Generic Front Mic as /devices/pci0000:00/0000:00:14.2/sound/card1/input4
[ 12.855316] input: HD-Audio Generic Line as /devices/pci0000:00/0000:00:14.2/sound/card1/input5
[ 12.855602] input: HD-Audio Generic Line Out Front as /devices/pci0000:00/0000:00:14.2/sound/card1/input6
[ 12.855878] input: HD-Audio Generic Line Out Surround as /devices/pci0000:00/0000:00:14.2/sound/card1/input7
[ 12.856162] input: HD-Audio Generic Line Out CLFE as /devices/pci0000:00/0000:00:14.2/sound/card1/input8
[ 12.856433] input: HD-Audio Generic Line Out Side as /devices/pci0000:00/0000:00:14.2/sound/card1/input9
[ 12.856704] input: HD-Audio Generic Front Headphone as /devices/pci0000:00/0000:00:14.2/sound/card1/input10
[ 12.861030] usb usb5: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 12.861037] usb usb5: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.861039] usb usb5: Product: OHCI PCI host controller
[ 12.861041] usb usb5: Manufacturer: Linux 6.3.0-rc3-00045-g64de4df9c80b ohci_hcd
[ 12.861043] usb usb5: SerialNumber: 0000:00:13.0
[ 12.867101] hub 5-0:1.0: USB hub found
[ 12.874104] hub 5-0:1.0: 5 ports detected
[ 12.902021] ohci-pci 0000:00:14.5: OHCI PCI host controller
[ 12.902052] ohci-pci 0000:00:14.5: new USB bus registered, assigned bus number 8
[ 12.902175] ohci-pci 0000:00:14.5: irq 18, io mem 0xf01ca000
[ 12.914899] r8169 0000:04:00.0 enp4s0: renamed from eth0
[ 12.981592] usb usb8: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 12.981599] usb usb8: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 12.981601] usb usb8: Product: OHCI PCI host controller
[ 12.981603] usb usb8: Manufacturer: Linux 6.3.0-rc3-00045-g64de4df9c80b ohci_hcd
[ 12.981604] usb usb8: SerialNumber: 0000:00:14.5
[ 12.983308] hub 8-0:1.0: USB hub found
[ 12.998172] hub 8-0:1.0: 2 ports detected
[ 13.004457] ohci-pci 0000:00:16.0: OHCI PCI host controller
[ 13.004481] ohci-pci 0000:00:16.0: new USB bus registered, assigned bus number 9
[ 13.004586] ohci-pci 0000:00:16.0: irq 18, io mem 0xf01cb000
[ 13.028929] usb 4-1: new low-speed USB device number 2 using ohci-pci
[ 13.117515] usb usb9: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 13.117522] usb usb9: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 13.117524] usb usb9: Product: OHCI PCI host controller
[ 13.117526] usb usb9: Manufacturer: Linux 6.3.0-rc3-00045-g64de4df9c80b ohci_hcd
[ 13.117527] usb usb9: SerialNumber: 0000:00:16.0
[ 13.122823] hub 9-0:1.0: USB hub found
[ 13.122855] hub 9-0:1.0: 4 ports detected
[ 13.150187] r8169 0000:04:00.0: Direct firmware load for rtl_nic/rtl8168f-1.fw failed with error -2
[ 13.150198] r8169 0000:04:00.0: Unable to load firmware rtl_nic/rtl8168f-1.fw (-2)
[ 13.150688] RTL8211E Gigabit Ethernet r8169-0-400:00: attached PHY driver (mii_bus:phy_addr=r8169-0-400:00, irq=MAC)
[ 13.231387] r8169 0000:04:00.0 enp4s0: Link is Down
[ 13.256051] [drm] radeon kernel modesetting enabled.
[ 13.262109] [drm] initializing kernel modesetting (ARUBA 0x1002:0x9996 0x1002:0x9996 0x00).
[ 13.262177] ATOM BIOS: 113
[ 13.262284] radeon 0000:00:01.0: VRAM: 512M 0x0000000000000000 - 0x000000001FFFFFFF (512M used)
[ 13.262288] radeon 0000:00:01.0: GTT: 1024M 0x0000000020000000 - 0x000000005FFFFFFF
[ 13.262296] [drm] Detected VRAM RAM=512M, BAR=256M
[ 13.262297] [drm] RAM width 64bits DDR
[ 13.262484] [drm] radeon: 512M of VRAM memory ready
[ 13.262490] [drm] radeon: 1024M of GTT memory ready.
[ 13.262530] [drm] Loading ARUBA Microcode
[ 13.270667] [drm] Internal thermal controller without fan control
[ 13.271087] [drm] radeon: dpm initialized
[ 13.275714] [drm] Found VCE firmware/feedback version 50.0.1 / 17!
[ 13.275771] [drm] GART: num cpu pages 262144, num gpu pages 262144
[ 13.279095] usb 4-1: New USB device found, idVendor=413c, idProduct=2106, bcdDevice= 1.01
[ 13.279101] usb 4-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 13.279103] usb 4-1: Product: Dell QuietKey Keyboard
[ 13.279105] usb 4-1: Manufacturer: DELL
[ 13.286833] input: DELL Dell QuietKey Keyboard as /devices/pci0000:00/0000:00:12.0/usb4/4-1/4-1:1.0/0003:413C:2106.0001/input/input11
[ 13.317348] [drm] PCIE GART of 1024M enabled (table at 0x00000000001D6000).
[ 13.317596] radeon 0000:00:01.0: WB enabled
[ 13.317599] radeon 0000:00:01.0: fence driver on ring 0 use gpu addr 0x0000000020000c00
[ 13.317977] radeon 0000:00:01.0: fence driver on ring 5 use gpu addr 0x0000000000075a18
[ 13.338850] radeon 0000:00:01.0: fence driver on ring 6 use gpu addr 0x0000000020000c18
[ 13.338854] radeon 0000:00:01.0: fence driver on ring 7 use gpu addr 0x0000000020000c1c
[ 13.338856] radeon 0000:00:01.0: fence driver on ring 1 use gpu addr 0x0000000020000c04
[ 13.338858] radeon 0000:00:01.0: fence driver on ring 2 use gpu addr 0x0000000020000c08
[ 13.338860] radeon 0000:00:01.0: fence driver on ring 3 use gpu addr 0x0000000020000c0c
[ 13.338861] radeon 0000:00:01.0: fence driver on ring 4 use gpu addr 0x0000000020000c10
[ 13.339153] radeon 0000:00:01.0: radeon: MSI limited to 32-bit
[ 13.339342] radeon 0000:00:01.0: radeon: using MSI.
[ 13.339414] [drm] radeon: irq initialized.
[ 13.345571] hid-generic 0003:413C:2106.0001: input,hidraw0: USB HID v1.10 Keyboard [DELL Dell QuietKey Keyboard] on usb-0000:00:12.0-1/input0
[ 13.390412] [drm] ring test on 0 succeeded in 3 usecs
[ 13.390423] [drm] ring test on 3 succeeded in 4 usecs
[ 13.390431] [drm] ring test on 4 succeeded in 4 usecs
[ 13.441915] [drm] ring test on 5 succeeded in 2 usecs
[ 13.461858] [drm] UVD initialized successfully.
[ 13.571355] [drm] ring test on 6 succeeded in 18 usecs
[ 13.571367] [drm] ring test on 7 succeeded in 4 usecs
[ 13.571368] [drm] VCE initialized successfully.
[ 13.571494] snd_hda_intel 0000:00:01.1: bound 0000:00:01.0 (ops radeon_audio_component_bind_ops [radeon])
[ 13.571667] [drm] ib test on ring 0 succeeded in 0 usecs
[ 13.571721] [drm] ib test on ring 3 succeeded in 0 usecs
[ 13.571771] [drm] ib test on ring 4 succeeded in 0 usecs
[ 13.768993] usb 4-2: new low-speed USB device number 3 using ohci-pci
[ 13.965196] usb 4-2: New USB device found, idVendor=046d, idProduct=c016, bcdDevice= 3.40
[ 13.965208] usb 4-2: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 13.965212] usb 4-2: Product: Optical USB Mouse
[ 13.965216] usb 4-2: Manufacturer: Logitech
[ 13.976218] input: Logitech Optical USB Mouse as /devices/pci0000:00/0000:00:12.0/usb4/4-2/4-2:1.0/0003:046D:C016.0002/input/input12
[ 13.977837] hid-generic 0003:046D:C016.0002: input,hidraw1: USB HID v1.10 Mouse [Logitech Optical USB Mouse] on usb-0000:00:12.0-2/input0
[ 14.105109] [drm] ib test on ring 5 succeeded
[ 14.649036] [drm] ib test on ring 6 succeeded
[ 15.161027] [drm] ib test on ring 7 succeeded
[ 15.173369] [drm] Radeon Display Connectors
[ 15.173376] [drm] Connector 0:
[ 15.173378] [drm] DP-1
[ 15.173380] [drm] HPD1
[ 15.173382] [drm] DDC: 0x6530 0x6530 0x6534 0x6534 0x6538 0x6538 0x653c 0x653c
[ 15.173387] [drm] Encoders:
[ 15.173388] [drm] DFP1: INTERNAL_UNIPHY2
[ 15.173390] [drm] Connector 1:
[ 15.173392] [drm] VGA-1
[ 15.173393] [drm] HPD2
[ 15.173395] [drm] DDC: 0x6540 0x6540 0x6544 0x6544 0x6548 0x6548 0x654c 0x654c
[ 15.173399] [drm] Encoders:
[ 15.173400] [drm] CRT1: INTERNAL_UNIPHY2
[ 15.173402] [drm] CRT1: NUTMEG
[ 15.173403] [drm] Connector 2:
[ 15.173405] [drm] HDMI-A-1
[ 15.173406] [drm] HPD3
[ 15.173408] [drm] DDC: 0x6550 0x6550 0x6554 0x6554 0x6558 0x6558 0x655c 0x655c
[ 15.173411] [drm] Encoders:
[ 15.173412] [drm] DFP2: INTERNAL_UNIPHY
[ 15.447734] [drm] fb mappable at 0xE03E9000
[ 15.447741] [drm] vram apper at 0xE0000000
[ 15.447743] [drm] size 5242880
[ 15.447745] [drm] fb depth is 24
[ 15.447747] [drm] pitch is 5120
[ 15.448342] fbcon: radeondrmfb (fb0) is primary device
[ 15.631670] Console: switching to colour frame buffer device 160x64
[ 15.633551] radeon 0000:00:01.0: [drm] fb0: radeondrmfb frame buffer device
[ 15.645230] [drm] Initialized radeon 2.50.0 20080528 for 0000:00:01.0 on minor 0
[ 15.872804] r8169 0000:04:00.0 enp4s0: Link is Up - 1Gbps/Full - flow control rx/tx
[ 15.872819] IPv6: ADDRCONF(NETDEV_CHANGE): enp4s0: link becomes ready
[ 18.185928] memfd_create() without MFD_EXEC nor MFD_NOEXEC_SEAL, pid=246 'systemd'
[ 18.983836] [drm] amdgpu kernel modesetting enabled.
[-- Attachment #3: 20230419-coreboot-cbmem-log-cb-68169.txt --]
[-- Type: text/plain, Size: 32882 bytes --]
[NOTE ] coreboot-4.18-15-gc782ef4345 Wed Apr 19 16:04:54 UTC 2023 bootblock starting (log level: 7)...
[DEBUG] FMAP: Found "FLASH" version 1.1 at 0x10000.
[DEBUG] FMAP: base = 0xffc00000 size = 0x400000 #areas = 4
[DEBUG] FMAP: area COREBOOT found @ 10200 (4128256 bytes)
[INFO ] CBFS: mcache @0x00034e00 built for 15 files, used 0x330 of 0x4000 bytes
[INFO ] CBFS: Found 'fallback/romstage' @0x20e00 size 0x4cf90 in mcache @0x00034fc0
[DEBUG] BS: bootblock times (exec / console): total (unknown) / 1 ms
[NOTE ] coreboot-4.18-15-gc782ef4345 Wed Apr 19 16:04:54 UTC 2023 romstage starting (log level: 7)...
[DEBUG] APIC 00: CPU Family_Model = 00610f31
[DEBUG] APIC 00: ** Enter AmdInitReset [00020007]
[DEBUG] Fch OEM config in INIT RESET
[DEBUG] AmdInitReset() returned AGESA_SUCCESS
[DEBUG] APIC 00: Heap in LocalCache (2) at 0x00400000
[DEBUG] APIC 00: ** Exit AmdInitReset [00020007]
[DEBUG] APIC 00: ** Enter AmdInitEarly [00020002]
[DEBUG] AmdInitEarly() returned AGESA_SUCCESS
[DEBUG] APIC 00: Heap in LocalCache (2) at 0x00400000
[DEBUG] APIC 00: ** Exit AmdInitEarly [00020002]
[DEBUG] APIC 00: ** Enter AmdInitPost [00020006]
[ERROR] -------------SPD READ ERROR-----------
[ERROR] -------------SPD READ ERROR-----------
[DEBUG] AmdInitPost() returned AGESA_SUCCESS
[DEBUG] APIC 00: Heap in TempMem (3) at 0x000b0000
[DEBUG] APIC 00: ** Exit AmdInitPost [00020006]
[DEBUG] CBMEM:
[DEBUG] IMD: root @ 0x5ffff000 254 entries.
[DEBUG] IMD: root @ 0x5fffec00 62 entries.
[DEBUG] FMAP: area COREBOOT found @ 10200 (4128256 bytes)
[DEBUG] Normal boot
[INFO ] CBFS: Found 'fallback/postcar' @0x2340 size 0x51a4 in mcache @0x00034ee8
[DEBUG] Loading module at 0x5ffd0000 with entry 0x5ffd0031. filesize: 0x4ee0 memsize: 0xb1f0
[DEBUG] Processing 161 relocs. Offset value of 0x5dfd0000
[DEBUG] BS: romstage times (exec / console): total (unknown) / 2 ms
[NOTE ] coreboot-4.18-15-gc782ef4345 Wed Apr 19 16:04:54 UTC 2023 postcar starting (log level: 7)...
[DEBUG] FMAP: area COREBOOT found @ 10200 (4128256 bytes)
[INFO ] CBFS: Found 'fallback/ramstage' @0x6de40 size 0x217fe in mcache @0x5ffdd240
[DEBUG] Loading module at 0x5feb8000 with entry 0x5feb8000. filesize: 0x46038 memsize: 0x116828
[DEBUG] Processing 3978 relocs. Offset value of 0x5beb8000
[DEBUG] BS: postcar times (exec / console): total (unknown) / 0 ms
[NOTE ] coreboot-4.18-15-gc782ef4345 Wed Apr 19 16:04:54 UTC 2023 ramstage starting (log level: 7)...
[DEBUG] Normal boot
[DEBUG] APIC 00: ** Enter AmdInitEnv [00020003]
[DEBUG] Wiped HEAP at [10000000 - 1002ffff]
[DEBUG] Fch OEM config in INIT ENV
[DEBUG] AmdInitEnv() returned AGESA_SUCCESS
[DEBUG] APIC 00: Heap in SystemMem (4) at 0x10000014
[DEBUG] APIC 00: ** Exit AmdInitEnv [00020003]
[DEBUG] BS: BS_PRE_DEVICE entry times (exec / console): 25 / 0 ms
[INFO ] Enumerating buses...
[DEBUG] Root Device scanning...
[DEBUG] CPU_CLUSTER: 0 enabled
[DEBUG] DOMAIN: 0000 enabled
[DEBUG] DOMAIN: 0000 scanning...
[DEBUG] PCI: pci_scan_bus for bus 00
[DEBUG] PCI: 00:00.0 [1022/1410] enabled
[DEBUG] PCI: 00:00.2 [1022/1419] enabled
[DEBUG] PCI: 00:01.0 [1002/9996] enabled
[DEBUG] PCI: 00:01.1 [1002/9902] enabled
[INFO ] PCI: Static device PCI: 00:02.0 not found, disabling it.
[DEBUG] hudson_enable()
[INFO ] PCI: Static device PCI: 00:10.0 not found, disabling it.
[DEBUG] hudson_enable()
[INFO ] PCI: Static device PCI: 00:10.1 not found, disabling it.
[DEBUG] hudson_enable()
[DEBUG] PCI: 00:11.0 [1022/7801] enabled
[DEBUG] hudson_enable()
[DEBUG] PCI: 00:12.0 [1022/7807] enabled
[DEBUG] hudson_enable()
[DEBUG] PCI: 00:12.2 [1022/7808] enabled
[DEBUG] hudson_enable()
[DEBUG] PCI: 00:13.0 [1022/7807] enabled
[DEBUG] hudson_enable()
[DEBUG] PCI: 00:13.2 [1022/7808] enabled
[DEBUG] hudson_enable()
[DEBUG] PCI: 00:14.0 [1022/780b] enabled
[DEBUG] hudson_enable()
[DEBUG] hudson_enable()
[DEBUG] PCI: 00:14.2 [1022/780d] enabled
[DEBUG] hudson_enable()
[DEBUG] PCI: 00:14.3 [1022/780e] enabled
[DEBUG] hudson_enable()
[DEBUG] PCI: 00:14.4 [1022/780f] enabled
[DEBUG] PCI: 00:14.5 [1022/7809] enabled
[DEBUG] hudson_enable()
[DEBUG] hudson_enable()
[DEBUG] PCI: 00:15.0 [1022/43a0] enabled
[DEBUG] hudson_enable()
[DEBUG] PCI: 00:15.1 [1022/43a1] enabled
[DEBUG] hudson_enable()
[DEBUG] PCI: 00:15.2 [1022/43a2] disabled
[DEBUG] PCI: 00:16.0 [1022/7807] enabled
[DEBUG] PCI: 00:16.2 [1022/7808] enabled
[DEBUG] PCI: 00:18.0 [1022/1400] enabled
[DEBUG] PCI: 00:18.1 [1022/1401] enabled
[DEBUG] PCI: 00:18.2 [1022/1402] enabled
[DEBUG] PCI: 00:18.3 [1022/1403] enabled
[DEBUG] PCI: 00:18.4 [1022/1404] enabled
[DEBUG] PCI: 00:18.5 [1022/1405] enabled
[WARN ] PCI: Leftover static devices:
[WARN ] PCI: 00:02.0
[WARN ] PCI: 00:10.0
[WARN ] PCI: 00:10.1
[WARN ] PCI: 00:14.1
[WARN ] PCI: 00:14.7
[WARN ] PCI: Check your devicetree.cb.
[DEBUG] PCI: 00:14.0 scanning...
[DEBUG] scan_bus: bus PCI: 00:14.0 finished in 0 msecs
[DEBUG] PCI: 00:14.3 scanning...
[DEBUG] PNP: 002e.0 disabled
[DEBUG] PNP: 002e.1 disabled
[DEBUG] PNP: 002e.2 enabled
[DEBUG] PNP: 002e.3 disabled
[DEBUG] PNP: 002e.5 enabled
[DEBUG] PNP: 002e.6 disabled
[DEBUG] PNP: 002e.7 enabled
[DEBUG] PNP: 002e.8 disabled
[DEBUG] PNP: 002e.108 enabled
[DEBUG] PNP: 002e.9 disabled
[DEBUG] PNP: 002e.109 enabled
[DEBUG] PNP: 002e.209 enabled
[DEBUG] PNP: 002e.309 enabled
[DEBUG] PNP: 002e.409 enabled
[DEBUG] PNP: 002e.509 enabled
[DEBUG] PNP: 002e.609 enabled
[DEBUG] PNP: 002e.709 enabled
[DEBUG] PNP: 002e.a enabled
[DEBUG] PNP: 002e.b enabled
[DEBUG] PNP: 002e.d disabled
[DEBUG] PNP: 002e.e disabled
[DEBUG] PNP: 002e.f enabled
[DEBUG] PNP: 002e.14 enabled
[DEBUG] PNP: 002e.16 disabled
[DEBUG] scan_bus: bus PCI: 00:14.3 finished in 0 msecs
[DEBUG] PCI: 00:14.4 scanning...
[DEBUG] PCI: pci_scan_bus for bus 01
[DEBUG] scan_bus: bus PCI: 00:14.4 finished in 0 msecs
[DEBUG] PCI: 00:15.0 scanning...
[DEBUG] PCI: pci_scan_bus for bus 02
[DEBUG] scan_bus: bus PCI: 00:15.0 finished in 0 msecs
[DEBUG] PCI: 00:15.1 scanning...
[DEBUG] PCI: pci_scan_bus for bus 03
[DEBUG] PCI: 03:00.0 [1b21/1042] enabled
[DEBUG] scan_bus: bus PCI: 00:15.1 finished in 0 msecs
[DEBUG] scan_bus: bus DOMAIN: 0000 finished in 0 msecs
[DEBUG] scan_bus: bus Root Device finished in 0 msecs
[INFO ] done
[DEBUG] BS: BS_DEV_ENUMERATE run times (exec / console): 1 / 0 ms
[DEBUG] found VGA at PCI: 00:01.0
[DEBUG] Setting up VGA for PCI: 00:01.0
[DEBUG] Setting PCI_BRIDGE_CTL_VGA for bridge DOMAIN: 0000
[DEBUG] Setting PCI_BRIDGE_CTL_VGA for bridge Root Device
[INFO ] Allocating resources...
[INFO ] Reading resources...
[DEBUG] fx_devs=0x1
[ERROR] PNP: 002e.7 missing read_resources
[DEBUG] Adding PCIe enhanced config space BAR 0xf8000000-0xfc000000.
[INFO ] Done reading resources.
[ERROR] skipping PNP: 002e.7@f4 fixed resource, size=0!
[ERROR] skipping PNP: 002e.7@e0 fixed resource, size=0!
[ERROR] skipping PNP: 002e.7@e1 fixed resource, size=0!
[ERROR] skipping PNP: 002e.108@e0 fixed resource, size=0!
[ERROR] skipping PNP: 002e.108@e2 fixed resource, size=0!
[ERROR] skipping PNP: 002e.108@e4 fixed resource, size=0!
[ERROR] skipping PNP: 002e.108@f0 fixed resource, size=0!
[ERROR] skipping PNP: 002e.108@f4 fixed resource, size=0!
[ERROR] skipping PNP: 002e.108@f5 fixed resource, size=0!
[ERROR] skipping PNP: 002e.108@f6 fixed resource, size=0!
[ERROR] skipping PNP: 002e.108@f7 fixed resource, size=0!
[ERROR] skipping PNP: 002e.209@e0 fixed resource, size=0!
[ERROR] skipping PNP: 002e.309@e4 fixed resource, size=0!
[ERROR] skipping PNP: 002e.309@e5 fixed resource, size=0!
[ERROR] skipping PNP: 002e.409@f0 fixed resource, size=0!
[ERROR] skipping PNP: 002e.a@e6 fixed resource, size=0!
[ERROR] skipping PNP: 002e.a@e7 fixed resource, size=0!
[ERROR] skipping PNP: 002e.b@e2 fixed resource, size=0!
[ERROR] skipping PNP: 002e.b@e4 fixed resource, size=0!
[ERROR] skipping PNP: 002e.f@e6 fixed resource, size=0!
[ERROR] skipping PNP: 002e.14@e0 fixed resource, size=0!
[INFO ] Setting resources...
[DEBUG] node 0: mmio_basek=00200000, basek=00400000, limitk=005fc000
[INFO ] add_uma_resource_below_tolm: uma size 0x20000000, memory start 0x60000000
[DEBUG] PCI: 00:00.2 44 <- [0x00000000f0100000 - 0x00000000f017ffff] size 0x00080000 gran 0x13 mem
[DEBUG] PCI: 00:01.0 10 <- [0x00000000e0000000 - 0x00000000efffffff] size 0x10000000 gran 0x1c prefmem
[DEBUG] PCI: 00:01.0 14 <- [0x0000000000001000 - 0x00000000000010ff] size 0x00000100 gran 0x08 io
[DEBUG] PCI: 00:01.0 18 <- [0x00000000f0180000 - 0x00000000f01bffff] size 0x00040000 gran 0x12 mem
[DEBUG] PCI: 00:01.1 10 <- [0x00000000f01c0000 - 0x00000000f01c3fff] size 0x00004000 gran 0x0e mem
[DEBUG] PCI: 00:11.0 10 <- [0x0000000000001410 - 0x0000000000001417] size 0x00000008 gran 0x03 io
[DEBUG] PCI: 00:11.0 14 <- [0x0000000000001420 - 0x0000000000001423] size 0x00000004 gran 0x02 io
[DEBUG] PCI: 00:11.0 18 <- [0x0000000000001418 - 0x000000000000141f] size 0x00000008 gran 0x03 io
[DEBUG] PCI: 00:11.0 1c <- [0x0000000000001424 - 0x0000000000001427] size 0x00000004 gran 0x02 io
[DEBUG] PCI: 00:11.0 20 <- [0x0000000000001400 - 0x000000000000140f] size 0x00000010 gran 0x04 io
[DEBUG] PCI: 00:11.0 24 <- [0x00000000f01cc000 - 0x00000000f01cc7ff] size 0x00000800 gran 0x0b mem
[DEBUG] PCI: 00:12.0 10 <- [0x00000000f01c8000 - 0x00000000f01c8fff] size 0x00001000 gran 0x0c mem
[DEBUG] PCI: 00:12.2 10 <- [0x00000000f01cd000 - 0x00000000f01cd0ff] size 0x00000100 gran 0x08 mem
[DEBUG] PCI: 00:13.0 10 <- [0x00000000f01c9000 - 0x00000000f01c9fff] size 0x00001000 gran 0x0c mem
[DEBUG] PCI: 00:13.2 10 <- [0x00000000f01ce000 - 0x00000000f01ce0ff] size 0x00000100 gran 0x08 mem
[DEBUG] PCI: 00:14.2 10 <- [0x00000000f01c4000 - 0x00000000f01c7fff] size 0x00004000 gran 0x0e mem64
[DEBUG] PNP: 002e.2 60 <- [0x00000000000003f8 - 0x00000000000003ff] size 0x00000008 gran 0x03 io
[DEBUG] PNP: 002e.2 70 <- [0x0000000000000004 - 0x0000000000000004] size 0x00000001 gran 0x00 irq
[DEBUG] PNP: 002e.5 60 <- [0x0000000000000060 - 0x0000000000000060] size 0x00000001 gran 0x00 io
[DEBUG] PNP: 002e.5 62 <- [0x0000000000000064 - 0x0000000000000064] size 0x00000001 gran 0x00 io
[DEBUG] PNP: 002e.5 70 <- [0x0000000000000001 - 0x0000000000000001] size 0x00000001 gran 0x00 irq
[DEBUG] PNP: 002e.5 72 <- [0x000000000000000c - 0x000000000000000c] size 0x00000001 gran 0x00 irq
[ERROR] PNP: 002e.7 missing set_resources
[DEBUG] PNP: 002e.108 e0 <- [0x00000000000000ff - 0x00000000000000fe] size 0x00000000 gran 0x00 irq
[DEBUG] PNP: 002e.108 e2 <- [0x00000000000000ff - 0x00000000000000fe] size 0x00000000 gran 0x00 irq
[DEBUG] PNP: 002e.108 e4 <- [0x00000000000000ff - 0x00000000000000fe] size 0x00000000 gran 0x00 irq
[DEBUG] PNP: 002e.108 f0 <- [0x00000000000000ff - 0x00000000000000fe] size 0x00000000 gran 0x00 irq
[DEBUG] PNP: 002e.108 f4 <- [0x0000000000000008 - 0x0000000000000007] size 0x00000000 gran 0x00 irq
[DEBUG] PNP: 002e.108 f5 <- [0x00000000000000ff - 0x00000000000000fe] size 0x00000000 gran 0x00 irq
[DEBUG] PNP: 002e.108 f6 <- [0x0000000000000000 - 0xffffffffffffffff] size 0x00000000 gran 0x00 irq
[DEBUG] PNP: 002e.108 f7 <- [0x00000000000000ff - 0x00000000000000fe] size 0x00000000 gran 0x00 irq
[DEBUG] PNP: 002e.209 e0 <- [0x00000000000000ff - 0x00000000000000fe] size 0x00000000 gran 0x00 irq
[DEBUG] PNP: 002e.309 e4 <- [0x000000000000007f - 0x000000000000007e] size 0x00000000 gran 0x00 irq
[DEBUG] PNP: 002e.309 e5 <- [0x0000000000000000 - 0xffffffffffffffff] size 0x00000000 gran 0x00 irq
[DEBUG] PNP: 002e.409 f0 <- [0x00000000000000ff - 0x00000000000000fe] size 0x00000000 gran 0x00 irq
[DEBUG] PNP: 002e.509 f4 <- [0x00000000000000ff - 0x00000000000000ff] size 0x00000001 gran 0x00 irq
[WARN ] PNP: 002e.509 f5 irq size: 0x0000000001 not assigned in devicetree
[WARN ] PNP: 002e.609 f4 irq size: 0x0000000001 not assigned in devicetree
[WARN ] PNP: 002e.609 f5 irq size: 0x0000000001 not assigned in devicetree
[DEBUG] PNP: 002e.a e6 <- [0x000000000000004c - 0x000000000000004b] size 0x00000000 gran 0x00 irq
[DEBUG] PNP: 002e.a e7 <- [0x0000000000000011 - 0x0000000000000010] size 0x00000000 gran 0x00 irq
[DEBUG] PNP: 002e.a f2 <- [0x000000000000005d - 0x000000000000005d] size 0x00000001 gran 0x00 irq
[DEBUG] PNP: 002e.b 60 <- [0x0000000000000290 - 0x0000000000000291] size 0x00000002 gran 0x01 io
[DEBUG] PNP: 002e.b 62 <- [0x0000000000000000 - 0x0000000000000001] size 0x00000002 gran 0x01 io
[DEBUG] PNP: 002e.b 70 <- [0x0000000000000000 - 0x0000000000000000] size 0x00000001 gran 0x00 io
[DEBUG] PNP: 002e.b e2 <- [0x000000000000007f - 0x000000000000007e] size 0x00000000 gran 0x00 irq
[DEBUG] PNP: 002e.b e4 <- [0x00000000000000f1 - 0x00000000000000f0] size 0x00000000 gran 0x00 irq
[WARN ] PNP: 002e.b f0 irq size: 0x0000000001 not assigned in devicetree
[DEBUG] PNP: 002e.f e6 <- [0x0000000000000007 - 0x0000000000000006] size 0x00000000 gran 0x00 irq
[DEBUG] PNP: 002e.14 e0 <- [0x0000000000000000 - 0xffffffffffffffff] size 0x00000000 gran 0x00 irq
[DEBUG] PCI: 00:14.4 1c <- [0x000000000000ffff - 0x000000000000fffe] size 0x00000000 gran 0x0c bus 01 io
[DEBUG] PCI: 00:14.4 24 <- [0x00000000f7ffffff - 0x00000000f7fffffe] size 0x00000000 gran 0x14 bus 01 prefmem
[DEBUG] PCI: 00:14.4 20 <- [0x00000000f7ffffff - 0x00000000f7fffffe] size 0x00000000 gran 0x14 bus 01 mem
[DEBUG] PCI: 00:14.5 10 <- [0x00000000f01ca000 - 0x00000000f01cafff] size 0x00001000 gran 0x0c mem
[DEBUG] PCI: 00:15.0 1c <- [0x000000000000ffff - 0x000000000000fffe] size 0x00000000 gran 0x0c bus 02 io
[DEBUG] PCI: 00:15.0 24 <- [0x00000000f7ffffff - 0x00000000f7fffffe] size 0x00000000 gran 0x14 bus 02 prefmem
[DEBUG] PCI: 00:15.0 20 <- [0x00000000f7ffffff - 0x00000000f7fffffe] size 0x00000000 gran 0x14 bus 02 mem
[DEBUG] PCI: 00:15.1 1c <- [0x000000000000ffff - 0x000000000000fffe] size 0x00000000 gran 0x0c bus 03 io
[DEBUG] PCI: 00:15.1 24 <- [0x00000000f7ffffff - 0x00000000f7fffffe] size 0x00000000 gran 0x14 bus 03 prefmem
[DEBUG] PCI: 00:15.1 20 <- [0x00000000f0000000 - 0x00000000f00fffff] size 0x00100000 gran 0x14 bus 03 mem
[DEBUG] PCI: 03:00.0 10 <- [0x00000000f0000000 - 0x00000000f0007fff] size 0x00008000 gran 0x0f mem64
[DEBUG] PCI: 00:16.0 10 <- [0x00000000f01cb000 - 0x00000000f01cbfff] size 0x00001000 gran 0x0c mem
[DEBUG] PCI: 00:16.2 10 <- [0x00000000f01cf000 - 0x00000000f01cf0ff] size 0x00000100 gran 0x08 mem
[INFO ] Done setting resources.
[INFO ] Done allocating resources.
[DEBUG] BS: BS_DEV_RESOURCES run times (exec / console): 1 / 0 ms
[DEBUG] APIC 00: ** Enter AmdInitMid [00020005]
[DEBUG] AmdInitMid() returned AGESA_SUCCESS
[DEBUG] APIC 00: Heap in SystemMem (4) at 0x10000014
[DEBUG] APIC 00: ** Exit AmdInitMid [00020005]
[DEBUG] PCI_INTR tables: Writing registers C00/C01 for PIC mode PCI IRQ routing:
[DEBUG] PCI_INTR_INDEX PCI_INTR_DATA
[DEBUG] 0x00 INTA# : 0x1F
[DEBUG] 0x01 INTB# : 0x1F
[DEBUG] 0x02 INTC# : 0x1F
[DEBUG] 0x03 INTD# : 0x1F
[DEBUG] 0x04 INTE# : 0x1F
[DEBUG] 0x05 INTF# : 0x1F
[DEBUG] 0x06 INTG# : 0x1F
[DEBUG] 0x07 INTH# : 0x1F
[DEBUG] 0x08 Misc : 0x0A
[DEBUG] 0x09 Misc0 : 0xF1
[DEBUG] 0x0A Misc1 : 0x00
[DEBUG] 0x0B Misc2 : 0x00
[DEBUG] 0x0C Ser IRQ INTA : 0x1F
[DEBUG] 0x0D Ser IRQ INTB : 0x1F
[DEBUG] 0x0E Ser IRQ INTC : 0x1F
[DEBUG] 0x0F Ser IRQ INTD : 0x1F
[DEBUG] 0x10 SCI : 0x09
[DEBUG] 0x11 SMBUS0 : 0x1F
[DEBUG] 0x12 ASF : 0x1F
[DEBUG] 0x13 HDA : 0x1F
[DEBUG] 0x14 SD : 0x1F
[DEBUG] 0x15 GEC : 0x1F
[DEBUG] 0x16 PerMon : 0x1F
[DEBUG] 0x20 IMC INT0 : 0x1F
[DEBUG] 0x21 IMC INT1 : 0x1F
[DEBUG] 0x22 IMC INT2 : 0x1F
[DEBUG] 0x23 IMC INT3 : 0x1F
[DEBUG] 0x24 IMC INT4 : 0x1F
[DEBUG] 0x25 IMC INT5 : 0x1F
[DEBUG] 0x30 Dev18.0 INTA : 0x1F
[DEBUG] 0x31 Dev18.2 INTB : 0x1F
[DEBUG] 0x32 Dev19.0 INTA : 0x1F
[DEBUG] 0x33 Dev19.2 INTB : 0x1F
[DEBUG] 0x34 Dev22.0 INTA : 0x1F
[DEBUG] 0x35 Dev22.2 INTB : 0x1F
[DEBUG] 0x36 Dev20.5 INTC : 0x1F
[DEBUG] 0x40 IDE : 0x1F
[DEBUG] 0x41 SATA : 0x1F
[DEBUG] 0x50 GPPInt0 : 0x1F
[DEBUG] 0x51 GPPInt1 : 0x1F
[DEBUG] 0x52 GPPInt2 : 0x1F
[DEBUG] 0x53 GPPInt3 : 0x1F
[DEBUG] PCI_INTR tables: Writing registers C00/C01 for APIC mode PCI IRQ routing:
[DEBUG] PCI_INTR_INDEX PCI_INTR_DATA
[DEBUG] 0x00 INTA# : 0x10
[DEBUG] 0x01 INTB# : 0x11
[DEBUG] 0x02 INTC# : 0x12
[DEBUG] 0x03 INTD# : 0x13
[DEBUG] 0x04 INTE# : 0x14
[DEBUG] 0x05 INTF# : 0x15
[DEBUG] 0x06 INTG# : 0x16
[DEBUG] 0x07 INTH# : 0x17
[DEBUG] 0x08 Misc : 0x00
[DEBUG] 0x09 Misc0 : 0x00
[DEBUG] 0x0A Misc1 : 0x00
[DEBUG] 0x0B Misc2 : 0x00
[DEBUG] 0x0C Ser IRQ INTA : 0x1F
[DEBUG] 0x0D Ser IRQ INTB : 0x1F
[DEBUG] 0x0E Ser IRQ INTC : 0x1F
[DEBUG] 0x0F Ser IRQ INTD : 0x1F
[DEBUG] 0x10 SCI : 0x09
[DEBUG] 0x11 SMBUS0 : 0x1F
[DEBUG] 0x12 ASF : 0x1F
[DEBUG] 0x13 HDA : 0x10
[DEBUG] 0x14 SD : 0x1F
[DEBUG] 0x15 GEC : 0x10
[DEBUG] 0x16 PerMon : 0x1F
[DEBUG] 0x20 IMC INT0 : 0x05
[DEBUG] 0x21 IMC INT1 : 0x1F
[DEBUG] 0x22 IMC INT2 : 0x1F
[DEBUG] 0x23 IMC INT3 : 0x1F
[DEBUG] 0x24 IMC INT4 : 0x1F
[DEBUG] 0x25 IMC INT5 : 0x1F
[DEBUG] 0x30 Dev18.0 INTA : 0x12
[DEBUG] 0x31 Dev18.2 INTB : 0x11
[DEBUG] 0x32 Dev19.0 INTA : 0x12
[DEBUG] 0x33 Dev19.2 INTB : 0x11
[DEBUG] 0x34 Dev22.0 INTA : 0x12
[DEBUG] 0x35 Dev22.2 INTB : 0x11
[DEBUG] 0x36 Dev20.5 INTC : 0x12
[DEBUG] 0x40 IDE : 0x11
[DEBUG] 0x41 SATA : 0x13
[DEBUG] 0x50 GPPInt0 : 0x10
[DEBUG] 0x51 GPPInt1 : 0x11
[DEBUG] 0x52 GPPInt2 : 0x12
[DEBUG] 0x53 GPPInt3 : 0x13
[WARN ] Can't write PCI IRQ assignments because 'mainboard_pirq_data' structure does not exist
[DEBUG] BS: BS_DEV_ENABLE entry times (exec / console): 5 / 0 ms
[INFO ] Enabling resources...
[DEBUG] PCI: 00:00.0 subsystem <- 1022/1410
[DEBUG] PCI: 00:00.0 cmd <- 06
[DEBUG] PCI: 00:00.2 cmd <- 06
[DEBUG] PCI: 00:01.0 cmd <- 07
[DEBUG] PCI: 00:01.1 cmd <- 02
[DEBUG] PCI: 00:11.0 cmd <- 03
[DEBUG] PCI: 00:12.0 subsystem <- 1022/1410
[DEBUG] PCI: 00:12.0 cmd <- 02
[DEBUG] PCI: 00:12.2 subsystem <- 1022/1410
[DEBUG] PCI: 00:12.2 cmd <- 02
[DEBUG] PCI: 00:13.0 subsystem <- 1022/1410
[DEBUG] PCI: 00:13.0 cmd <- 02
[DEBUG] PCI: 00:13.2 subsystem <- 1022/1410
[DEBUG] PCI: 00:13.2 cmd <- 02
[DEBUG] PCI: 00:14.0 subsystem <- 1022/1410
[DEBUG] PCI: 00:14.0 cmd <- 403
[DEBUG] PCI: 00:14.2 subsystem <- 1022/1410
[DEBUG] PCI: 00:14.2 cmd <- 02
[DEBUG] PCI: 00:14.3 subsystem <- 1022/1410
[DEBUG] PCI: 00:14.3 cmd <- 0f
[DEBUG] hudson lpc decode:PNP: 002e.2, base=0x000003f8, end=0x000003ff
[DEBUG] hudson lpc decode:PNP: 002e.5, base=0x00000060, end=0x00000060
[DEBUG] hudson lpc decode:PNP: 002e.5, base=0x00000064, end=0x00000064
[DEBUG] hudson lpc decode:PNP: 002e.b, base=0x00000290, end=0x00000291
[DEBUG] PCI: 00:14.4 bridge ctrl <- 0013
[DEBUG] PCI: 00:14.4 cmd <- 00
[DEBUG] PCI: 00:14.5 cmd <- 02
[DEBUG] PCI: 00:15.0 bridge ctrl <- 0013
[DEBUG] PCI: 00:15.0 cmd <- 00
[DEBUG] PCI: 00:15.1 bridge ctrl <- 0013
[DEBUG] PCI: 00:15.1 cmd <- 06
[DEBUG] PCI: 00:16.0 cmd <- 02
[DEBUG] PCI: 00:16.2 cmd <- 02
[DEBUG] PCI: 00:18.0 cmd <- 00
[DEBUG] PCI: 00:18.1 subsystem <- 1022/1410
[DEBUG] PCI: 00:18.1 cmd <- 00
[DEBUG] PCI: 00:18.2 subsystem <- 1022/1410
[DEBUG] PCI: 00:18.2 cmd <- 00
[DEBUG] PCI: 00:18.3 subsystem <- 1022/1410
[DEBUG] PCI: 00:18.3 cmd <- 00
[DEBUG] PCI: 00:18.4 subsystem <- 1022/1410
[DEBUG] PCI: 00:18.4 cmd <- 00
[DEBUG] PCI: 00:18.5 subsystem <- 1022/1410
[DEBUG] PCI: 00:18.5 cmd <- 00
[DEBUG] PCI: 03:00.0 cmd <- 02
[INFO ] done.
[INFO ] Initializing devices...
[DEBUG] CPU_CLUSTER: 0 init
[DEBUG] MTRR check
[DEBUG] Fixed MTRRs : Disabled
[DEBUG] Variable MTRRs: Enabled
[INFO ] CPU: AMD A6-6400K APU with Radeon(tm) HD Graphics .
[INFO ] LAPIC 0x10 in XAPIC mode.
[DEBUG] Loading module at 0x00030000 with entry 0x00030000. filesize: 0x1b8 memsize: 0x1b8
[DEBUG] Processing 18 relocs. Offset value of 0x00030000
[DEBUG] Attempting to start 1 APs
[DEBUG] Waiting for 10ms after sending INIT.
[DEBUG] Waiting for SIPI to complete...
[INFO ] LAPIC 0x11 in XAPIC mode.
[DEBUG] done.
[INFO ] AP: slot 1 apic_id 11
[DEBUG] Waiting for SIPI to complete...
[DEBUG] done.
[INFO ] Initializing CPU #0
[DEBUG] CPU: vendor AMD device 610f31
[DEBUG] CPU: family 15, model 13, stepping 01
[DEBUG] Model 15 Init.
[DEBUG] siblings = 01, CPU #0 initialized
[INFO ] Initializing CPU #1
[DEBUG] CPU: vendor AMD device 610f31
[DEBUG] CPU: family 15, model 13, stepping 01
[DEBUG] Model 15 Init.
[DEBUG] siblings = 01, CPU #1 initialized
[INFO ] bsp_do_flight_plan done after 0 msecs.
[DEBUG] CPU_CLUSTER: 0 init finished in 10 msecs
[DEBUG] PCI: 00:00.0 init
[DEBUG] PCI: 00:00.0 init finished in 0 msecs
[DEBUG] PCI: 00:01.0 init
[DEBUG] PCI: 00:01.0 init finished in 0 msecs
[DEBUG] PCI: 00:01.1 init
[DEBUG] PCI: 00:01.1 init finished in 0 msecs
[DEBUG] PCI: 00:11.0 init
[DEBUG] PCI: 00:11.0 init finished in 0 msecs
[DEBUG] PCI: 00:12.0 init
[DEBUG] PCI: 00:12.0 init finished in 0 msecs
[DEBUG] PCI: 00:12.2 init
[DEBUG] PCI: 00:12.2 init finished in 0 msecs
[DEBUG] PCI: 00:13.0 init
[DEBUG] PCI: 00:13.0 init finished in 0 msecs
[DEBUG] PCI: 00:13.2 init
[DEBUG] PCI: 00:13.2 init finished in 0 msecs
[DEBUG] PCI: 00:14.0 init
[DEBUG] IOAPIC: Initializing IOAPIC at 0xfec00000
[DEBUG] IOAPIC: ID = 0x04
[DEBUG] IOAPIC: 24 interrupts
[DEBUG] IOAPIC: Clearing IOAPIC at 0xfec00000
[DEBUG] IOAPIC: Bootstrap Processor Local APIC = 0x10
[DEBUG] PCI: 00:14.0 init finished in 0 msecs
[DEBUG] PCI: 00:14.2 init
[DEBUG] PCI: 00:14.2 init finished in 0 msecs
[DEBUG] PCI: 00:14.3 init
[DEBUG] RTC Init
[DEBUG] PCI: 00:14.3 init finished in 0 msecs
[DEBUG] PCI: 00:14.5 init
[DEBUG] PCI: 00:14.5 init finished in 0 msecs
[DEBUG] PCI: 00:15.0 init
[DEBUG] PCI: 00:15.0 init finished in 0 msecs
[DEBUG] PCI: 00:15.1 init
[DEBUG] PCI: 00:15.1 init finished in 0 msecs
[DEBUG] PCI: 00:16.0 init
[DEBUG] PCI: 00:16.0 init finished in 0 msecs
[DEBUG] PCI: 00:16.2 init
[DEBUG] PCI: 00:16.2 init finished in 0 msecs
[DEBUG] PCI: 00:18.1 init
[DEBUG] PCI: 00:18.1 init finished in 0 msecs
[DEBUG] PCI: 00:18.2 init
[DEBUG] PCI: 00:18.2 init finished in 0 msecs
[DEBUG] PCI: 00:18.3 init
[DEBUG] PCI: 00:18.3 init finished in 0 msecs
[DEBUG] PCI: 00:18.4 init
[DEBUG] PCI: 00:18.4 init finished in 0 msecs
[DEBUG] PCI: 00:18.5 init
[DEBUG] PCI: 00:18.5 init finished in 0 msecs
[DEBUG] PNP: 002e.2 init
[DEBUG] PNP: 002e.2 init finished in 0 msecs
[DEBUG] PNP: 002e.5 init
[DEBUG] PNP: 002e.5 init finished in 0 msecs
[DEBUG] PNP: 002e.108 init
[DEBUG] PNP: 002e.108 init finished in 0 msecs
[DEBUG] PNP: 002e.109 init
[DEBUG] PNP: 002e.109 init finished in 0 msecs
[DEBUG] PNP: 002e.209 init
[DEBUG] PNP: 002e.209 init finished in 0 msecs
[DEBUG] PNP: 002e.309 init
[DEBUG] PNP: 002e.309 init finished in 0 msecs
[DEBUG] PNP: 002e.409 init
[DEBUG] PNP: 002e.409 init finished in 0 msecs
[DEBUG] PNP: 002e.509 init
[DEBUG] PNP: 002e.509 init finished in 0 msecs
[DEBUG] PNP: 002e.609 init
[DEBUG] PNP: 002e.609 init finished in 0 msecs
[DEBUG] PNP: 002e.709 init
[DEBUG] PNP: 002e.709 init finished in 0 msecs
[DEBUG] PNP: 002e.a init
[DEBUG] PNP: 002e.a init finished in 0 msecs
[DEBUG] PNP: 002e.b init
[DEBUG] PNP: 002e.b init finished in 0 msecs
[DEBUG] PNP: 002e.f init
[DEBUG] PNP: 002e.f init finished in 0 msecs
[DEBUG] PNP: 002e.14 init
[DEBUG] PNP: 002e.14 init finished in 0 msecs
[DEBUG] PCI: 03:00.0 init
[DEBUG] PCI: 03:00.0 init finished in 0 msecs
[INFO ] Devices initialized
[DEBUG] BS: BS_DEV_INIT run times (exec / console): 10 / 0 ms
[INFO ] Finalize devices...
[DEBUG] PCI: 00:14.3 final
[INFO ] Devices finalized
[DEBUG] APIC 00: ** Enter AmdInitLate [00020004]
[EMERG] ASSERTION ERROR: file 'src/vendorcode/amd/agesa/f15tn/Proc/Common/CommonReturns.c', line 187
[EMERG] ASSERTION ERROR: file 'src/vendorcode/amd/agesa/f15tn/Proc/CPU/cpuGeneralServices.c', line 776
[EMERG] ASSERTION ERROR: file 'src/vendorcode/amd/agesa/f15tn/Proc/Common/CommonReturns.c', line 187
[EMERG] ASSERTION ERROR: file 'src/vendorcode/amd/agesa/f15tn/Proc/CPU/cpuGeneralServices.c', line 776
[EMERG] ASSERTION ERROR: file 'src/vendorcode/amd/agesa/f15tn/Proc/Common/CommonReturns.c', line 187
[EMERG] ASSERTION ERROR: file 'src/vendorcode/amd/agesa/f15tn/Proc/CPU/cpuGeneralServices.c', line 776
[EMERG] ASSERTION ERROR: file 'src/vendorcode/amd/agesa/f15tn/Proc/Common/CommonReturns.c', line 187
[EMERG] ASSERTION ERROR: file 'src/vendorcode/amd/agesa/f15tn/Proc/CPU/cpuGeneralServices.c', line 776
[EMERG] ASSERTION ERROR: file 'src/vendorcode/amd/agesa/f15tn/Proc/Common/CommonReturns.c', line 187
[EMERG] ASSERTION ERROR: file 'src/vendorcode/amd/agesa/f15tn/Proc/CPU/cpuGeneralServices.c', line 776
[DEBUG] AmdInitLate() returned AGESA_SUCCESS
[DEBUG] APIC 00: Heap in SystemMem (4) at 0x10000014
[DEBUG] APIC 00: ** Exit AmdInitLate [00020004]
[DEBUG] APIC 00: ** Enter AmdS3Save [0002000b]
[DEBUG] FMAP: area COREBOOT found @ 10200 (4128256 bytes)
[INFO ] CBFS: Found 'pci1002,9996.rom' @0x8f6c0 size 0xf200 in mcache @0x5ffdd294
[DEBUG] FMAP: area RW_MRC_CACHE found @ 0 (65536 bytes)
[DEBUG] MRC: Checking cached data update for 'RW_MRC_CACHE'.
[INFO ] Manufacturer: ef
[INFO ] SF: Detected ef 4016 with sector size 0x1000, total 0x400000
[DEBUG] MRC: cache data 'RW_MRC_CACHE' needs update.
[DEBUG] MRC: updated 'RW_MRC_CACHE'.
[DEBUG] AmdS3Save() returned AGESA_SUCCESS
[EMERG] ASSERTION ERROR: file 'src/drivers/amd/agesa/state_machine.c', line 276
[DEBUG] APIC 00: Heap in SystemMem (4) at 0x10000014
[DEBUG] APIC 00: ** Exit AmdS3Save [0002000b]
[DEBUG] BS: BS_POST_DEVICE exit times (exec / console): 14 / 0 ms
[INFO ] Writing IRQ routing tables to 0xf0000...write_pirq_routing_table done.
[INFO ] Writing IRQ routing tables to 0x5fe6e000...write_pirq_routing_table done.
[DEBUG] PIRQ table: 48 bytes.
[INFO ] CBFS: Found 'fallback/dsdt.aml' @0x700 size 0x193a in mcache @0x5ffdd094
[WARN ] CBFS: 'fallback/slic' not found.
[INFO ] ACPI: Writing ACPI tables at 5fe4a000.
[DEBUG] ACPI: * FACS
[DEBUG] ACPI: * DSDT
[DEBUG] ACPI: * FADT
[DEBUG] pm_base: 0x0800
[DEBUG] ACPI: added table 1/32, length now 40
[DEBUG] ACPI: * SSDT
[INFO ] CBFS: Found 'pci1002,9996.rom' @0x8f6c0 size 0xf200 in mcache @0x5ffdd294
[DEBUG] In CBFS, ROM address for PCI: 00:01.0 = 0xffc9f8ec
[ERROR] PCI: 00:01.0: Missing ACPI scope
[DEBUG] ACPI: added table 2/32, length now 44
[DEBUG] ACPI: * MCFG
[DEBUG] ACPI: added table 3/32, length now 48
[DEBUG] ACPI: * MADT
[DEBUG] ACPI: added table 4/32, length now 52
[DEBUG] current = 5fe4be20
[DEBUG] ACPI: * HPET
[DEBUG] ACPI: added table 5/32, length now 56
[DEBUG] ACPI: added table 6/32, length now 60
[DEBUG] ACPI: * IVRS at 5fe4c030
[DEBUG] ACPI: added table 7/32, length now 64
[DEBUG] ACPI: * SRAT at 5fe4c0a0
[DEBUG] AGESA SRAT table NULL. Skipping.
[DEBUG] ACPI: * SLIT at 5fe4c0a0
[DEBUG] AGESA SLIT table NULL. Skipping.
[DEBUG] ACPI: * AGESA ALIB SSDT at 5fe4c0a0
[DEBUG] ACPI: added table 8/32, length now 68
[DEBUG] ACPI: * SSDT at 5fe4c5c0
[DEBUG] ACPI: added table 9/32, length now 72
[DEBUG] ACPI: * SSDT for PState at 5fe4cc72
[INFO ] CBFS: Found 'pci1002,9996.rom' @0x8f6c0 size 0xf200 in mcache @0x5ffdd294
[DEBUG] In CBFS, ROM address for PCI: 00:01.0 = 0xffc9f8ec
[DEBUG] Copying VBIOS image from 0xffc9f8ec
[DEBUG] ACPI: * VFCT at 5fe4cc80
[DEBUG] ACPI: added table 10/32, length now 76
[INFO ] ACPI: done.
[DEBUG] ACPI tables: 73456 bytes.
[DEBUG] smbios_write_tables: 5fe42000
[DEBUG] SMBIOS firmware version is set to coreboot_version: '4.18-15-gc782ef4345'
[DEBUG] SMBIOS tables: 531 bytes.
[DEBUG] Writing table forward entry at 0x00000500
[DEBUG] Wrote coreboot table at: 0x00000500, 0x10 bytes, checksum aff7
[DEBUG] Writing coreboot table at 0x5fe6f000
[DEBUG] 0. 0000000000000000-0000000000000fff: CONFIGURATION TABLES
[DEBUG] 1. 0000000000001000-000000000009ffff: RAM
[DEBUG] 2. 00000000000c0000-000000005fe41fff: RAM
[DEBUG] 3. 000000005fe42000-000000005feb7fff: CONFIGURATION TABLES
[DEBUG] 4. 000000005feb8000-000000005ffcefff: RAMSTAGE
[DEBUG] 5. 000000005ffcf000-000000005fffffff: CONFIGURATION TABLES
[DEBUG] 6. 0000000060000000-000000007fffffff: RESERVED
[DEBUG] 7. 00000000f8000000-00000000fbffffff: RESERVED
[DEBUG] 8. 00000000fec10000-00000000fec10fff: RESERVED
[DEBUG] 9. 0000000100000000-000000017effffff: RAM
[DEBUG] Wrote coreboot table at: 0x5fe6f000, 0x354 bytes, checksum d6c8
[DEBUG] coreboot table: 876 bytes.
[DEBUG] IMD ROOT 0. 0x5ffff000 0x00001000
[DEBUG] IMD SMALL 1. 0x5fffe000 0x00001000
[DEBUG] CONSOLE 2. 0x5ffde000 0x00020000
[DEBUG] RO MCACHE 3. 0x5ffdd000 0x00000330
[DEBUG] TIME STAMP 4. 0x5ffdc000 0x00000910
[DEBUG] AFTER CAR 5. 0x5ffcf000 0x0000d000
[DEBUG] RAMSTAGE 6. 0x5feb7000 0x00118000
[DEBUG] SMM BACKUP 7. 0x5fea7000 0x00010000
[DEBUG] ACPISCRATCH 8. 0x5fe77000 0x00030000
[DEBUG] COREBOOT 9. 0x5fe6f000 0x00008000
[DEBUG] IRQ TABLE 10. 0x5fe6e000 0x00001000
[DEBUG] ACPI 11. 0x5fe4a000 0x00024000
[DEBUG] SMBIOS 12. 0x5fe42000 0x00008000
[DEBUG] IMD small region:
[DEBUG] IMD ROOT 0. 0x5fffec00 0x00000400
[DEBUG] FMAP 1. 0x5fffeb20 0x000000e0
[DEBUG] ROMSTAGE 2. 0x5fffeb00 0x00000004
[DEBUG] ROMSTG STCK 3. 0x5fffea60 0x00000088
[DEBUG] AGESA MTRR 4. 0x5fffe960 0x000000f0
[DEBUG] BS: BS_WRITE_TABLES run times (exec / console): 9 / 0 ms
[INFO ] CBFS: Found 'fallback/payload' @0x9e900 size 0x118e9 in mcache @0x5ffdd2c0
[DEBUG] Checking segment from ROM address 0xffcaeb2c
[DEBUG] Checking segment from ROM address 0xffcaeb48
[DEBUG] Loading segment from ROM address 0xffcaeb2c
[DEBUG] code (compression=1)
[DEBUG] New segment dstaddr 0x000dee80 memsize 0x21180 srcaddr 0xffcaeb64 filesize 0x118b1
[DEBUG] Loading Segment: addr: 0x000dee80 memsz: 0x0000000000021180 filesz: 0x00000000000118b1
[DEBUG] using LZMA
[DEBUG] Loading segment from ROM address 0xffcaeb48
[DEBUG] Entry Point 0x000fd266
[DEBUG] BS: BS_PAYLOAD_LOAD run times (exec / console): 13 / 0 ms
[DEBUG] Jumping to boot code at 0x000fd266(0x5fe6f000)
SeaBIOS (version rel-1.16.2-0-gea1b7a0)
BUILD: gcc: (Debian 12.2.0-14) 12.2.0 binutils: (GNU Binutils for Debian) 2.40
Found coreboot cbmem console @ 5ffde000
Found mainboard ASUS F2A85-M_PRO
Relocating init from 0x000e05c0 to 0x5ee34b20 (size 54336)
Found CBFS header at 0xffc1022c
multiboot: eax=5fefb3d8, ebx=5fefb3a4
Found 26 PCI devices (max PCI bus is 03)
Copying SMBIOS from 0x5fe42000 to 0x000f6880
Copying SMBIOS 3.0 from 0x5fe42020 to 0x000f6860
Copying ACPI RSDP from 0x5fe4a000 to 0x000f6830
Copying PIR from 0x5fe6e000 to 0x000f6800
table(50434146)=0x5fe4bbc0 (via xsdt)
Using pmtimer, ioport 0x818
Scan for VGA option rom
Running option rom at c000:0003
Turning on vga text mode console
SeaBIOS (version rel-1.16.2-0-gea1b7a0)
PCI: XHCI at 03:00.0 (mmio 0xf0000000)
XHCI init: regs @ 0xf0000000, 4 ports, 32 slots, 32 byte contexts
XHCI extcap 0x1 @ 0xf0000800
XHCI protocol USB 3.00, 2 ports (offset 1), def 0
XHCI protocol USB 2.00, 2 ports (offset 3), def 1
EHCI init on dev 00:12.2 (regs=0xf01cd020)
EHCI init on dev 00:13.2 (regs=0xf01ce020)
EHCI init on dev 00:16.2 (regs=0xf01cf020)
OHCI init on dev 00:12.0 (regs=0xf01c8000)
OHCI init on dev 00:13.0 (regs=0xf01c9000)
OHCI init on dev 00:14.5 (regs=0xf01ca000)
OHCI init on dev 00:16.0 (regs=0xf01cb000)
AHCI controller at 00:11.0, iobase 0xf01cc000, irq 0
Searching bootorder for: HALT
Found 0 lpt ports
Found 1 serial ports
Searching bootorder for: /pci@i0cf8/*@11/drive@6/disk@0
AHCI/6: Set transfer mode to UDMA-6
Searching bios-geometry for: /pci@i0cf8/*@11/drive@6/disk@0
AHCI/6: registering: "AHCI/6: SanDisk SDSSDP064G ATA-9 Hard-Disk (61057 MiBytes)"
USB keyboard initialized
USB mouse initialized
XHCI no devices found
PS2 keyboard initialized
All threads complete.
Scan for option roms
Press ESC for boot menu.
Searching bootorder for: HALT
drive 0x000f6790: PCHS=16383/16/63 translation=lba LCHS=1024/255/63 s=125045424
Space available for UMB: cf800-ec000, f60a0-f6790
Returned 16756736 bytes of ZoneHigh
e820 map has 8 items:
0: 0000000000000000 - 000000000009fc00 = 1 RAM
1: 000000000009fc00 - 00000000000a0000 = 2 RESERVED
2: 00000000000f0000 - 0000000000100000 = 2 RESERVED
3: 0000000000100000 - 000000005fe3d000 = 1 RAM
4: 000000005fe3d000 - 0000000080000000 = 2 RESERVED
5: 00000000f8000000 - 00000000fc000000 = 2 RESERVED
6: 00000000fec10000 - 00000000fec11000 = 2 RESERVED
7: 0000000100000000 - 000000017f000000 = 1 RAM
enter handle_19:
NULL
Booting from Hard Disk...
Booting from 0000:7c00
[-- Attachment #4: Type: text/plain, Size: 161 bytes --]
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
2023-04-19 16:21 ` Andrew Cooper
(?)
@ 2023-04-20 8:32 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-20 8:32 UTC (permalink / raw)
To: Andrew Cooper, Paul Menzel
Cc: linux-kernel, x86, David Woodhouse, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Wed, Apr 19 2023 at 17:21, Andrew Cooper wrote:
> On 19/04/2023 2:50 pm, Andrew Cooper wrote:
>> What I'm confused by is why this system boots in the first place. I can
>> only think that's is a system which only has 4-bit APIC IDs, and happens
>> to function when bit 4 gets truncated off the top of the SIPI destination...
>
> https://www.amd.com/system/files/TechDocs/42300_15h_Mod_10h-1Fh_BKDG.pdf
>
> This system does still require the IO-APICs to be at 0, and the LAPICs
> to start at some offset, which is clearly 16 in this case. Also, this
> system has configurable 4-bit or 8-bit wide APIC IDs, and I can't tell
> which mode is active just from the manual.
That document contradicts itself:
"The ApicId of core j must be enumerated/assigned as:
ApicId[core=j] = (OFFSET_IDX) * MNC + j
Where OFFSET_IDX is an integer offset (0 to N) used to shift up the
core ApicId values to allow room for IOAPIC devices.
It is recommended that BIOS use the following APIC ID assignments for
the broadest operating system sup- port. Given N = MNC and M =
Number_Of_IOAPICs:
• Assign the core ApicId’s first from 0 to N-1, and the IOAPIC IDs
from N to N+(M-1)."
Oh well. If the rest of these docs is of the same quality then it's not
a surprise that BIOSes are trainwrecks.
> But, it does mean that the BIOS has genuinely modified the APIC IDs of
> the logic processors. This does highlight an error in reasoning with
> the parallel bringup code.
Yes.
> For xAPIC, the APIC_ID register is writeable (at least, model
> specifically), and CPUID is only the value it would have had at reset.
> So the AP bringup logic can't actually use CPUID reliably.
>
> This was changed in x2APIC, which made the x2APIC_ID immutable.
>
> I don't see an option other than the AP bringup code query for xAPIC vs
> x2APIC mode, and either looking at the real APIC_ID register, or falling
> back to CPUID.
I'm pondering to simply deny parallel mode if x2APIC is not there.
Thanks,
tglx
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-20 8:32 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-20 8:32 UTC (permalink / raw)
To: Andrew Cooper, Paul Menzel
Cc: linux-kernel, x86, David Woodhouse, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Wed, Apr 19 2023 at 17:21, Andrew Cooper wrote:
> On 19/04/2023 2:50 pm, Andrew Cooper wrote:
>> What I'm confused by is why this system boots in the first place. I can
>> only think that's is a system which only has 4-bit APIC IDs, and happens
>> to function when bit 4 gets truncated off the top of the SIPI destination...
>
> https://www.amd.com/system/files/TechDocs/42300_15h_Mod_10h-1Fh_BKDG.pdf
>
> This system does still require the IO-APICs to be at 0, and the LAPICs
> to start at some offset, which is clearly 16 in this case. Also, this
> system has configurable 4-bit or 8-bit wide APIC IDs, and I can't tell
> which mode is active just from the manual.
That document contradicts itself:
"The ApicId of core j must be enumerated/assigned as:
ApicId[core=j] = (OFFSET_IDX) * MNC + j
Where OFFSET_IDX is an integer offset (0 to N) used to shift up the
core ApicId values to allow room for IOAPIC devices.
It is recommended that BIOS use the following APIC ID assignments for
the broadest operating system sup- port. Given N = MNC and M =
Number_Of_IOAPICs:
• Assign the core ApicId’s first from 0 to N-1, and the IOAPIC IDs
from N to N+(M-1)."
Oh well. If the rest of these docs is of the same quality then it's not
a surprise that BIOSes are trainwrecks.
> But, it does mean that the BIOS has genuinely modified the APIC IDs of
> the logic processors. This does highlight an error in reasoning with
> the parallel bringup code.
Yes.
> For xAPIC, the APIC_ID register is writeable (at least, model
> specifically), and CPUID is only the value it would have had at reset.
> So the AP bringup logic can't actually use CPUID reliably.
>
> This was changed in x2APIC, which made the x2APIC_ID immutable.
>
> I don't see an option other than the AP bringup code query for xAPIC vs
> x2APIC mode, and either looking at the real APIC_ID register, or falling
> back to CPUID.
I'm pondering to simply deny parallel mode if x2APIC is not there.
Thanks,
tglx
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-20 8:32 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-20 8:32 UTC (permalink / raw)
To: Andrew Cooper, Paul Menzel
Cc: linux-kernel, x86, David Woodhouse, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Wed, Apr 19 2023 at 17:21, Andrew Cooper wrote:
> On 19/04/2023 2:50 pm, Andrew Cooper wrote:
>> What I'm confused by is why this system boots in the first place. I can
>> only think that's is a system which only has 4-bit APIC IDs, and happens
>> to function when bit 4 gets truncated off the top of the SIPI destination...
>
> https://www.amd.com/system/files/TechDocs/42300_15h_Mod_10h-1Fh_BKDG.pdf
>
> This system does still require the IO-APICs to be at 0, and the LAPICs
> to start at some offset, which is clearly 16 in this case. Also, this
> system has configurable 4-bit or 8-bit wide APIC IDs, and I can't tell
> which mode is active just from the manual.
That document contradicts itself:
"The ApicId of core j must be enumerated/assigned as:
ApicId[core=j] = (OFFSET_IDX) * MNC + j
Where OFFSET_IDX is an integer offset (0 to N) used to shift up the
core ApicId values to allow room for IOAPIC devices.
It is recommended that BIOS use the following APIC ID assignments for
the broadest operating system sup- port. Given N = MNC and M =
Number_Of_IOAPICs:
• Assign the core ApicId’s first from 0 to N-1, and the IOAPIC IDs
from N to N+(M-1)."
Oh well. If the rest of these docs is of the same quality then it's not
a surprise that BIOSes are trainwrecks.
> But, it does mean that the BIOS has genuinely modified the APIC IDs of
> the logic processors. This does highlight an error in reasoning with
> the parallel bringup code.
Yes.
> For xAPIC, the APIC_ID register is writeable (at least, model
> specifically), and CPUID is only the value it would have had at reset.
> So the AP bringup logic can't actually use CPUID reliably.
>
> This was changed in x2APIC, which made the x2APIC_ID immutable.
>
> I don't see an option other than the AP bringup code query for xAPIC vs
> x2APIC mode, and either looking at the real APIC_ID register, or falling
> back to CPUID.
I'm pondering to simply deny parallel mode if x2APIC is not there.
Thanks,
tglx
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
2023-04-20 8:32 ` Thomas Gleixner
(?)
@ 2023-04-20 9:23 ` Andrew Cooper
-1 siblings, 0 replies; 236+ messages in thread
From: Andrew Cooper @ 2023-04-20 9:23 UTC (permalink / raw)
To: Thomas Gleixner, Paul Menzel
Cc: linux-kernel, x86, David Woodhouse, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On 20/04/2023 9:32 am, Thomas Gleixner wrote:
> On Wed, Apr 19 2023 at 17:21, Andrew Cooper wrote:
>> On 19/04/2023 2:50 pm, Andrew Cooper wrote:
>> For xAPIC, the APIC_ID register is writeable (at least, model
>> specifically), and CPUID is only the value it would have had at reset.
>> So the AP bringup logic can't actually use CPUID reliably.
>>
>> This was changed in x2APIC, which made the x2APIC_ID immutable.
>>
>> I don't see an option other than the AP bringup code query for xAPIC vs
>> x2APIC mode, and either looking at the real APIC_ID register, or falling
>> back to CPUID.
> I'm pondering to simply deny parallel mode if x2APIC is not there.
I'm not sure if that will help much.
Just because x2APIC is there doesn't mean it's in use. There are
several generations of Intel system which have x2APIC but also use the
opt-out bit in ACPI tables. There are some machines which have
mismatched APIC-ness settings in the BIOS->OS handover.
There's very little you can do on the BSP alone to know for certain that
the APs come out of wait-for-SIPI already in x2APIC mode.
One way is the ÆPIC Leak "locked into x2APIC mode" giant security
bodge. If the system really does have a CPU with an APIC ID above 0xfe,
then chances are good that the APs come out consistently...
~Andrew
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-20 9:23 ` Andrew Cooper
0 siblings, 0 replies; 236+ messages in thread
From: Andrew Cooper @ 2023-04-20 9:23 UTC (permalink / raw)
To: Thomas Gleixner, Paul Menzel
Cc: linux-kernel, x86, David Woodhouse, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On 20/04/2023 9:32 am, Thomas Gleixner wrote:
> On Wed, Apr 19 2023 at 17:21, Andrew Cooper wrote:
>> On 19/04/2023 2:50 pm, Andrew Cooper wrote:
>> For xAPIC, the APIC_ID register is writeable (at least, model
>> specifically), and CPUID is only the value it would have had at reset.
>> So the AP bringup logic can't actually use CPUID reliably.
>>
>> This was changed in x2APIC, which made the x2APIC_ID immutable.
>>
>> I don't see an option other than the AP bringup code query for xAPIC vs
>> x2APIC mode, and either looking at the real APIC_ID register, or falling
>> back to CPUID.
> I'm pondering to simply deny parallel mode if x2APIC is not there.
I'm not sure if that will help much.
Just because x2APIC is there doesn't mean it's in use. There are
several generations of Intel system which have x2APIC but also use the
opt-out bit in ACPI tables. There are some machines which have
mismatched APIC-ness settings in the BIOS->OS handover.
There's very little you can do on the BSP alone to know for certain that
the APs come out of wait-for-SIPI already in x2APIC mode.
One way is the ÆPIC Leak "locked into x2APIC mode" giant security
bodge. If the system really does have a CPU with an APIC ID above 0xfe,
then chances are good that the APs come out consistently...
~Andrew
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-20 9:23 ` Andrew Cooper
0 siblings, 0 replies; 236+ messages in thread
From: Andrew Cooper @ 2023-04-20 9:23 UTC (permalink / raw)
To: Thomas Gleixner, Paul Menzel
Cc: linux-kernel, x86, David Woodhouse, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On 20/04/2023 9:32 am, Thomas Gleixner wrote:
> On Wed, Apr 19 2023 at 17:21, Andrew Cooper wrote:
>> On 19/04/2023 2:50 pm, Andrew Cooper wrote:
>> For xAPIC, the APIC_ID register is writeable (at least, model
>> specifically), and CPUID is only the value it would have had at reset.
>> So the AP bringup logic can't actually use CPUID reliably.
>>
>> This was changed in x2APIC, which made the x2APIC_ID immutable.
>>
>> I don't see an option other than the AP bringup code query for xAPIC vs
>> x2APIC mode, and either looking at the real APIC_ID register, or falling
>> back to CPUID.
> I'm pondering to simply deny parallel mode if x2APIC is not there.
I'm not sure if that will help much.
Just because x2APIC is there doesn't mean it's in use. There are
several generations of Intel system which have x2APIC but also use the
opt-out bit in ACPI tables. There are some machines which have
mismatched APIC-ness settings in the BIOS->OS handover.
There's very little you can do on the BSP alone to know for certain that
the APs come out of wait-for-SIPI already in x2APIC mode.
One way is the ÆPIC Leak "locked into x2APIC mode" giant security
bodge. If the system really does have a CPU with an APIC ID above 0xfe,
then chances are good that the APs come out consistently...
~Andrew
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
2023-04-20 9:23 ` Andrew Cooper
(?)
@ 2023-04-20 11:17 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-20 11:17 UTC (permalink / raw)
To: Andrew Cooper, Paul Menzel
Cc: linux-kernel, x86, David Woodhouse, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Thu, Apr 20 2023 at 10:23, Andrew Cooper wrote:
> On 20/04/2023 9:32 am, Thomas Gleixner wrote:
>> I'm pondering to simply deny parallel mode if x2APIC is not there.
>
> I'm not sure if that will help much.
Spoilsport.
> Just because x2APIC is there doesn't mean it's in use. There are
> several generations of Intel system which have x2APIC but also use the
> opt-out bit in ACPI tables. There are some machines which have
> mismatched APIC-ness settings in the BIOS->OS handover.
>
> There's very little you can do on the BSP alone to know for certain that
> the APs come out of wait-for-SIPI already in x2APIC mode.
Yeah. Reading the APIC that early is going to be entertaining too :)
> One way is the ÆPIC Leak "locked into x2APIC mode" giant security
> bodge.
Bah.
> If the system really does have a CPU with an APIC ID above 0xfe, then
> chances are good that the APs come out consistently...
Anything else would be really magic :)
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-20 11:17 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-20 11:17 UTC (permalink / raw)
To: Andrew Cooper, Paul Menzel
Cc: linux-kernel, x86, David Woodhouse, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Thu, Apr 20 2023 at 10:23, Andrew Cooper wrote:
> On 20/04/2023 9:32 am, Thomas Gleixner wrote:
>> I'm pondering to simply deny parallel mode if x2APIC is not there.
>
> I'm not sure if that will help much.
Spoilsport.
> Just because x2APIC is there doesn't mean it's in use. There are
> several generations of Intel system which have x2APIC but also use the
> opt-out bit in ACPI tables. There are some machines which have
> mismatched APIC-ness settings in the BIOS->OS handover.
>
> There's very little you can do on the BSP alone to know for certain that
> the APs come out of wait-for-SIPI already in x2APIC mode.
Yeah. Reading the APIC that early is going to be entertaining too :)
> One way is the ÆPIC Leak "locked into x2APIC mode" giant security
> bodge.
Bah.
> If the system really does have a CPU with an APIC ID above 0xfe, then
> chances are good that the APs come out consistently...
Anything else would be really magic :)
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-20 11:17 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-20 11:17 UTC (permalink / raw)
To: Andrew Cooper, Paul Menzel
Cc: linux-kernel, x86, David Woodhouse, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Thu, Apr 20 2023 at 10:23, Andrew Cooper wrote:
> On 20/04/2023 9:32 am, Thomas Gleixner wrote:
>> I'm pondering to simply deny parallel mode if x2APIC is not there.
>
> I'm not sure if that will help much.
Spoilsport.
> Just because x2APIC is there doesn't mean it's in use. There are
> several generations of Intel system which have x2APIC but also use the
> opt-out bit in ACPI tables. There are some machines which have
> mismatched APIC-ness settings in the BIOS->OS handover.
>
> There's very little you can do on the BSP alone to know for certain that
> the APs come out of wait-for-SIPI already in x2APIC mode.
Yeah. Reading the APIC that early is going to be entertaining too :)
> One way is the ÆPIC Leak "locked into x2APIC mode" giant security
> bodge.
Bah.
> If the system really does have a CPU with an APIC ID above 0xfe, then
> chances are good that the APs come out consistently...
Anything else would be really magic :)
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
2023-04-20 11:17 ` Thomas Gleixner
(?)
@ 2023-04-20 14:51 ` Sean Christopherson
-1 siblings, 0 replies; 236+ messages in thread
From: Sean Christopherson @ 2023-04-20 14:51 UTC (permalink / raw)
To: Thomas Gleixner
Cc: Andrew Cooper, Paul Menzel, linux-kernel, x86, David Woodhouse,
Brian Gerst, Arjan van de Veen, Paolo Bonzini, Paul McKenney,
Tom Lendacky, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Thu, Apr 20, 2023, Thomas Gleixner wrote:
> On Thu, Apr 20 2023 at 10:23, Andrew Cooper wrote:
> > On 20/04/2023 9:32 am, Thomas Gleixner wrote:
> > > On Wed, Apr 19, 2023, Andrew Cooper wrote:
> > > > This was changed in x2APIC, which made the x2APIC_ID immutable.
>
> >> I'm pondering to simply deny parallel mode if x2APIC is not there.
> >
> > I'm not sure if that will help much.
>
> Spoilsport.
LOL, well let me pile on then. x2APIC IDs aren't immutable on AMD hardware. The
ID is read-only when the CPU is in x2APIC mode, but any changes made to the ID
while the CPU is in xAPIC mode survive the transition to x2APIC. From the APM:
A value previously written by software to the 8-bit APIC_ID register (MMIO offset
30h) is converted by hardware into the appropriate format and reflected into the
32-bit x2APIC_ID register (MSR 802h).
FWIW, my observations from testing on bare metal are that the xAPIC ID is effectively
read-only (writes are dropped) on Intel CPUs as far back as Haswell, while the above
behavior described in the APM holds true on at least Rome and Milan.
My guess is that Intel's uArch specific behavior of the xAPIC ID being read-only
was introduced when x2APIC came along, but I didn't test farther back than Haswell.
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-20 14:51 ` Sean Christopherson
0 siblings, 0 replies; 236+ messages in thread
From: Sean Christopherson @ 2023-04-20 14:51 UTC (permalink / raw)
To: Thomas Gleixner
Cc: Andrew Cooper, Paul Menzel, linux-kernel, x86, David Woodhouse,
Brian Gerst, Arjan van de Veen, Paolo Bonzini, Paul McKenney,
Tom Lendacky, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Thu, Apr 20, 2023, Thomas Gleixner wrote:
> On Thu, Apr 20 2023 at 10:23, Andrew Cooper wrote:
> > On 20/04/2023 9:32 am, Thomas Gleixner wrote:
> > > On Wed, Apr 19, 2023, Andrew Cooper wrote:
> > > > This was changed in x2APIC, which made the x2APIC_ID immutable.
>
> >> I'm pondering to simply deny parallel mode if x2APIC is not there.
> >
> > I'm not sure if that will help much.
>
> Spoilsport.
LOL, well let me pile on then. x2APIC IDs aren't immutable on AMD hardware. The
ID is read-only when the CPU is in x2APIC mode, but any changes made to the ID
while the CPU is in xAPIC mode survive the transition to x2APIC. From the APM:
A value previously written by software to the 8-bit APIC_ID register (MMIO offset
30h) is converted by hardware into the appropriate format and reflected into the
32-bit x2APIC_ID register (MSR 802h).
FWIW, my observations from testing on bare metal are that the xAPIC ID is effectively
read-only (writes are dropped) on Intel CPUs as far back as Haswell, while the above
behavior described in the APM holds true on at least Rome and Milan.
My guess is that Intel's uArch specific behavior of the xAPIC ID being read-only
was introduced when x2APIC came along, but I didn't test farther back than Haswell.
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-20 14:51 ` Sean Christopherson
0 siblings, 0 replies; 236+ messages in thread
From: Sean Christopherson @ 2023-04-20 14:51 UTC (permalink / raw)
To: Thomas Gleixner
Cc: Andrew Cooper, Paul Menzel, linux-kernel, x86, David Woodhouse,
Brian Gerst, Arjan van de Veen, Paolo Bonzini, Paul McKenney,
Tom Lendacky, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Thu, Apr 20, 2023, Thomas Gleixner wrote:
> On Thu, Apr 20 2023 at 10:23, Andrew Cooper wrote:
> > On 20/04/2023 9:32 am, Thomas Gleixner wrote:
> > > On Wed, Apr 19, 2023, Andrew Cooper wrote:
> > > > This was changed in x2APIC, which made the x2APIC_ID immutable.
>
> >> I'm pondering to simply deny parallel mode if x2APIC is not there.
> >
> > I'm not sure if that will help much.
>
> Spoilsport.
LOL, well let me pile on then. x2APIC IDs aren't immutable on AMD hardware. The
ID is read-only when the CPU is in x2APIC mode, but any changes made to the ID
while the CPU is in xAPIC mode survive the transition to x2APIC. From the APM:
A value previously written by software to the 8-bit APIC_ID register (MMIO offset
30h) is converted by hardware into the appropriate format and reflected into the
32-bit x2APIC_ID register (MSR 802h).
FWIW, my observations from testing on bare metal are that the xAPIC ID is effectively
read-only (writes are dropped) on Intel CPUs as far back as Haswell, while the above
behavior described in the APM holds true on at least Rome and Milan.
My guess is that Intel's uArch specific behavior of the xAPIC ID being read-only
was introduced when x2APIC came along, but I didn't test farther back than Haswell.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
2023-04-20 14:51 ` Sean Christopherson
(?)
@ 2023-04-20 15:57 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-20 15:57 UTC (permalink / raw)
To: Sean Christopherson
Cc: Andrew Cooper, Paul Menzel, linux-kernel, x86, David Woodhouse,
Brian Gerst, Arjan van de Veen, Paolo Bonzini, Paul McKenney,
Tom Lendacky, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Thu, Apr 20 2023 at 07:51, Sean Christopherson wrote:
> On Thu, Apr 20, 2023, Thomas Gleixner wrote:
>> On Thu, Apr 20 2023 at 10:23, Andrew Cooper wrote:
>> > On 20/04/2023 9:32 am, Thomas Gleixner wrote:
>> > > On Wed, Apr 19, 2023, Andrew Cooper wrote:
>> > > > This was changed in x2APIC, which made the x2APIC_ID immutable.
>>
>> >> I'm pondering to simply deny parallel mode if x2APIC is not there.
>> >
>> > I'm not sure if that will help much.
>>
>> Spoilsport.
>
> LOL, well let me pile on then. x2APIC IDs aren't immutable on AMD hardware. The
> ID is read-only when the CPU is in x2APIC mode, but any changes made to the ID
> while the CPU is in xAPIC mode survive the transition to x2APIC. From the APM:
>
> A value previously written by software to the 8-bit APIC_ID register (MMIO offset
> 30h) is converted by hardware into the appropriate format and reflected into the
> 32-bit x2APIC_ID register (MSR 802h).
>
> FWIW, my observations from testing on bare metal are that the xAPIC ID is effectively
> read-only (writes are dropped) on Intel CPUs as far back as Haswell, while the above
> behavior described in the APM holds true on at least Rome and Milan.
>
> My guess is that Intel's uArch specific behavior of the xAPIC ID being read-only
> was introduced when x2APIC came along, but I didn't test farther back than Haswell.
I'm not so worried about modern hardware. The horrorshow is the old muck
as demonstrated and of course there is virt :)
Something like the completely untested below should just work whatever
APIC ID the BIOS decided to dice.
That might just work on SEV too without that GHCB muck, but what do I
know.
Thanks,
tglx
---
--- a/arch/x86/include/asm/apicdef.h
+++ b/arch/x86/include/asm/apicdef.h
@@ -138,7 +138,8 @@
#define APIC_EILVT_MASKED (1 << 16)
#define APIC_BASE (fix_to_virt(FIX_APIC_BASE))
-#define APIC_BASE_MSR 0x800
+#define APIC_BASE_MSR 0x800
+#define APIC_X2APIC_ID_MSR 0x802
#define XAPIC_ENABLE (1UL << 11)
#define X2APIC_ENABLE (1UL << 10)
@@ -162,6 +163,7 @@
#define APIC_CPUID(apicid) ((apicid) & XAPIC_DEST_CPUS_MASK)
#define NUM_APIC_CLUSTERS ((BAD_APICID + 1) >> XAPIC_DEST_CPUS_SHIFT)
+#ifndef __ASSEMBLY__
/*
* the local APIC register structure, memory mapped. Not terribly well
* tested, but we might eventually use this one in the future - the
@@ -435,4 +437,5 @@ enum apic_delivery_modes {
APIC_DELIVERY_MODE_EXTINT = 7,
};
+#endif /* !__ASSEMBLY__ */
#endif /* _ASM_X86_APICDEF_H */
--- a/arch/x86/include/asm/smp.h
+++ b/arch/x86/include/asm/smp.h
@@ -195,14 +195,13 @@ extern void nmi_selftest(void);
#endif
extern unsigned int smpboot_control;
+extern unsigned long apic_mmio_base;
#endif /* !__ASSEMBLY__ */
/* Control bits for startup_64 */
-#define STARTUP_APICID_CPUID_1F 0x80000000
-#define STARTUP_APICID_CPUID_0B 0x40000000
-#define STARTUP_APICID_CPUID_01 0x20000000
-#define STARTUP_APICID_SEV_ES 0x10000000
+#define STARTUP_READ_APICID 0x80000000
+#define STARTUP_APICID_SEV_ES 0x40000000
/* Top 8 bits are reserved for control */
#define STARTUP_PARALLEL_MASK 0xFF000000
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -101,6 +101,8 @@ static int apic_extnmi __ro_after_init =
*/
static bool virt_ext_dest_id __ro_after_init;
+unsigned long apic_mmio_base __ro_after_init;
+
/*
* Map cpu index to physical APIC ID
*/
@@ -2164,6 +2166,7 @@ void __init register_lapic_address(unsig
if (!x2apic_mode) {
set_fixmap_nocache(FIX_APIC_BASE, address);
+ apic_mmio_base = APIC_BASE;
apic_printk(APIC_VERBOSE, "mapped APIC to %16lx (%16lx)\n",
APIC_BASE, address);
}
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -24,8 +24,10 @@
#include "../entry/calling.h"
#include <asm/export.h>
#include <asm/nospec-branch.h>
+#include <asm/apicdef.h>
#include <asm/fixmap.h>
#include <asm/smp.h>
+
#include <asm/sev-common.h>
/*
@@ -237,37 +239,24 @@ SYM_INNER_LABEL(secondary_startup_64_no_
#ifdef CONFIG_SMP
/*
- * For parallel boot, the APIC ID is retrieved from CPUID, and then
- * used to look up the CPU number. For booting a single CPU, the
- * CPU number is encoded in smpboot_control.
+ * For parallel boot, the APIC ID is either retrieved the APIC or
+ * from CPUID, and then used to look up the CPU number.
+ * For booting a single CPU, the CPU number is encoded in
+ * smpboot_control.
*
- * Bit 31 STARTUP_APICID_CPUID_1F flag (use CPUID 0x1f)
- * Bit 30 STARTUP_APICID_CPUID_0B flag (use CPUID 0x0b)
- * Bit 29 STARTUP_APICID_CPUID_01 flag (use CPUID 0x01)
- * Bit 28 STARTUP_APICID_SEV_ES flag (CPUID 0x0b via GHCB MSR)
+ * Bit 31 STARTUP_APICID_READ (Read APICID from APIC)
+ * Bit 30 STARTUP_APICID_SEV_ES flag (CPUID 0x0b via GHCB MSR)
* Bit 0-23 CPU# if STARTUP_APICID_CPUID_xx flags are not set
*/
movl smpboot_control(%rip), %ecx
+ testl $STARTUP_READ_APICID, %ecx
#ifdef CONFIG_AMD_MEM_ENCRYPT
testl $STARTUP_APICID_SEV_ES, %ecx
jnz .Luse_sev_cpuid_0b
#endif
- testl $STARTUP_APICID_CPUID_1F, %ecx
- jnz .Luse_cpuid_1f
- testl $STARTUP_APICID_CPUID_0B, %ecx
- jnz .Luse_cpuid_0b
- testl $STARTUP_APICID_CPUID_01, %ecx
- jnz .Luse_cpuid_01
andl $(~STARTUP_PARALLEL_MASK), %ecx
jmp .Lsetup_cpu
-.Luse_cpuid_01:
- mov $0x01, %eax
- cpuid
- mov %ebx, %edx
- shr $24, %edx
- jmp .Lsetup_AP
-
#ifdef CONFIG_AMD_MEM_ENCRYPT
.Luse_sev_cpuid_0b:
/* Set the GHCB MSR to request CPUID 0x0B_EDX */
@@ -292,24 +281,30 @@ SYM_INNER_LABEL(secondary_startup_64_no_
jmp .Lsetup_AP
#endif
-.Luse_cpuid_0b:
- mov $0x0B, %eax
- xorl %ecx, %ecx
- cpuid
- jmp .Lsetup_AP
+.Lread_apicid:
+ mov $MSR_IA32_APICBASE, %ecx
+ rdmsr
+ testl $X2APIC_ENABLE, %eax
+ jnz read_apicid_msr
+
+ /* Read the APIC ID from the fix-mapped MMIO space. */
+ movq apic_mmio_base(%rip), %rcx
+ addq $APIC_ID, %rcx
+ movl (%rcx), %eax
+ shr $24, %eax
+ jnz .Lread_apicid
-.Luse_cpuid_1f:
- mov $0x1f, %eax
- xorl %ecx, %ecx
- cpuid
+.Lread_apicid_msr:
+ mov $APIC_X2APIC_ID_MSR, %ecx
+ rdmsr
.Lsetup_AP:
- /* EDX contains the APIC ID of the current CPU */
+ /* EAX contains the APIC ID of the current CPU */
xorq %rcx, %rcx
leaq cpuid_to_apicid(%rip), %rbx
.Lfind_cpunr:
- cmpl (%rbx,%rcx,4), %edx
+ cmpl (%rbx,%rcx,4), %eax
jz .Lsetup_cpu
inc %ecx
#ifdef CONFIG_FORCE_NR_CPUS
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -1253,41 +1253,22 @@ bool __init arch_cpuhp_init_parallel_bri
return false;
}
- /* Encrypted guests require special CPUID handling. */
+ /* Encrypted guests require special handling. */
if (cc_platform_has(CC_ATTR_GUEST_STATE_ENCRYPT)) {
switch (cc_get_vendor()) {
case CC_VENDOR_AMD:
ctrl = STARTUP_APICID_SEV_ES;
if (topology_extended_leaf == 0x0b)
- goto setup;
+ break;
fallthrough;
default:
pr_info("Parallel CPU startup disabled due to guest state encryption\n");
return false;
}
+ } else {
+ ctrl = STARTUP_READ_APICID;
}
- switch (topology_extended_leaf) {
- case 0x0b:
- ctrl = STARTUP_APICID_CPUID_0B;
- break;
- case 0x1f:
- ctrl = STARTUP_APICID_CPUID_1F;
- break;
- case 0x00:
- /* For !x2APIC mode 8 bits from leaf 0x01 are sufficient. */
- if (!x2apic_mode) {
- ctrl = STARTUP_APICID_CPUID_01;
- break;
- }
- fallthrough;
- default:
- pr_info("Parallel CPU startup disabled. Unsupported topology leaf %u\n",
- topology_extended_leaf);
- return false;
- }
-
-setup:
pr_debug("Parallel CPU startup enabled: 0x%08x\n", ctrl);
smpboot_control = ctrl;
return true;
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-20 15:57 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-20 15:57 UTC (permalink / raw)
To: Sean Christopherson
Cc: Andrew Cooper, Paul Menzel, linux-kernel, x86, David Woodhouse,
Brian Gerst, Arjan van de Veen, Paolo Bonzini, Paul McKenney,
Tom Lendacky, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Thu, Apr 20 2023 at 07:51, Sean Christopherson wrote:
> On Thu, Apr 20, 2023, Thomas Gleixner wrote:
>> On Thu, Apr 20 2023 at 10:23, Andrew Cooper wrote:
>> > On 20/04/2023 9:32 am, Thomas Gleixner wrote:
>> > > On Wed, Apr 19, 2023, Andrew Cooper wrote:
>> > > > This was changed in x2APIC, which made the x2APIC_ID immutable.
>>
>> >> I'm pondering to simply deny parallel mode if x2APIC is not there.
>> >
>> > I'm not sure if that will help much.
>>
>> Spoilsport.
>
> LOL, well let me pile on then. x2APIC IDs aren't immutable on AMD hardware. The
> ID is read-only when the CPU is in x2APIC mode, but any changes made to the ID
> while the CPU is in xAPIC mode survive the transition to x2APIC. From the APM:
>
> A value previously written by software to the 8-bit APIC_ID register (MMIO offset
> 30h) is converted by hardware into the appropriate format and reflected into the
> 32-bit x2APIC_ID register (MSR 802h).
>
> FWIW, my observations from testing on bare metal are that the xAPIC ID is effectively
> read-only (writes are dropped) on Intel CPUs as far back as Haswell, while the above
> behavior described in the APM holds true on at least Rome and Milan.
>
> My guess is that Intel's uArch specific behavior of the xAPIC ID being read-only
> was introduced when x2APIC came along, but I didn't test farther back than Haswell.
I'm not so worried about modern hardware. The horrorshow is the old muck
as demonstrated and of course there is virt :)
Something like the completely untested below should just work whatever
APIC ID the BIOS decided to dice.
That might just work on SEV too without that GHCB muck, but what do I
know.
Thanks,
tglx
---
--- a/arch/x86/include/asm/apicdef.h
+++ b/arch/x86/include/asm/apicdef.h
@@ -138,7 +138,8 @@
#define APIC_EILVT_MASKED (1 << 16)
#define APIC_BASE (fix_to_virt(FIX_APIC_BASE))
-#define APIC_BASE_MSR 0x800
+#define APIC_BASE_MSR 0x800
+#define APIC_X2APIC_ID_MSR 0x802
#define XAPIC_ENABLE (1UL << 11)
#define X2APIC_ENABLE (1UL << 10)
@@ -162,6 +163,7 @@
#define APIC_CPUID(apicid) ((apicid) & XAPIC_DEST_CPUS_MASK)
#define NUM_APIC_CLUSTERS ((BAD_APICID + 1) >> XAPIC_DEST_CPUS_SHIFT)
+#ifndef __ASSEMBLY__
/*
* the local APIC register structure, memory mapped. Not terribly well
* tested, but we might eventually use this one in the future - the
@@ -435,4 +437,5 @@ enum apic_delivery_modes {
APIC_DELIVERY_MODE_EXTINT = 7,
};
+#endif /* !__ASSEMBLY__ */
#endif /* _ASM_X86_APICDEF_H */
--- a/arch/x86/include/asm/smp.h
+++ b/arch/x86/include/asm/smp.h
@@ -195,14 +195,13 @@ extern void nmi_selftest(void);
#endif
extern unsigned int smpboot_control;
+extern unsigned long apic_mmio_base;
#endif /* !__ASSEMBLY__ */
/* Control bits for startup_64 */
-#define STARTUP_APICID_CPUID_1F 0x80000000
-#define STARTUP_APICID_CPUID_0B 0x40000000
-#define STARTUP_APICID_CPUID_01 0x20000000
-#define STARTUP_APICID_SEV_ES 0x10000000
+#define STARTUP_READ_APICID 0x80000000
+#define STARTUP_APICID_SEV_ES 0x40000000
/* Top 8 bits are reserved for control */
#define STARTUP_PARALLEL_MASK 0xFF000000
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -101,6 +101,8 @@ static int apic_extnmi __ro_after_init =
*/
static bool virt_ext_dest_id __ro_after_init;
+unsigned long apic_mmio_base __ro_after_init;
+
/*
* Map cpu index to physical APIC ID
*/
@@ -2164,6 +2166,7 @@ void __init register_lapic_address(unsig
if (!x2apic_mode) {
set_fixmap_nocache(FIX_APIC_BASE, address);
+ apic_mmio_base = APIC_BASE;
apic_printk(APIC_VERBOSE, "mapped APIC to %16lx (%16lx)\n",
APIC_BASE, address);
}
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -24,8 +24,10 @@
#include "../entry/calling.h"
#include <asm/export.h>
#include <asm/nospec-branch.h>
+#include <asm/apicdef.h>
#include <asm/fixmap.h>
#include <asm/smp.h>
+
#include <asm/sev-common.h>
/*
@@ -237,37 +239,24 @@ SYM_INNER_LABEL(secondary_startup_64_no_
#ifdef CONFIG_SMP
/*
- * For parallel boot, the APIC ID is retrieved from CPUID, and then
- * used to look up the CPU number. For booting a single CPU, the
- * CPU number is encoded in smpboot_control.
+ * For parallel boot, the APIC ID is either retrieved the APIC or
+ * from CPUID, and then used to look up the CPU number.
+ * For booting a single CPU, the CPU number is encoded in
+ * smpboot_control.
*
- * Bit 31 STARTUP_APICID_CPUID_1F flag (use CPUID 0x1f)
- * Bit 30 STARTUP_APICID_CPUID_0B flag (use CPUID 0x0b)
- * Bit 29 STARTUP_APICID_CPUID_01 flag (use CPUID 0x01)
- * Bit 28 STARTUP_APICID_SEV_ES flag (CPUID 0x0b via GHCB MSR)
+ * Bit 31 STARTUP_APICID_READ (Read APICID from APIC)
+ * Bit 30 STARTUP_APICID_SEV_ES flag (CPUID 0x0b via GHCB MSR)
* Bit 0-23 CPU# if STARTUP_APICID_CPUID_xx flags are not set
*/
movl smpboot_control(%rip), %ecx
+ testl $STARTUP_READ_APICID, %ecx
#ifdef CONFIG_AMD_MEM_ENCRYPT
testl $STARTUP_APICID_SEV_ES, %ecx
jnz .Luse_sev_cpuid_0b
#endif
- testl $STARTUP_APICID_CPUID_1F, %ecx
- jnz .Luse_cpuid_1f
- testl $STARTUP_APICID_CPUID_0B, %ecx
- jnz .Luse_cpuid_0b
- testl $STARTUP_APICID_CPUID_01, %ecx
- jnz .Luse_cpuid_01
andl $(~STARTUP_PARALLEL_MASK), %ecx
jmp .Lsetup_cpu
-.Luse_cpuid_01:
- mov $0x01, %eax
- cpuid
- mov %ebx, %edx
- shr $24, %edx
- jmp .Lsetup_AP
-
#ifdef CONFIG_AMD_MEM_ENCRYPT
.Luse_sev_cpuid_0b:
/* Set the GHCB MSR to request CPUID 0x0B_EDX */
@@ -292,24 +281,30 @@ SYM_INNER_LABEL(secondary_startup_64_no_
jmp .Lsetup_AP
#endif
-.Luse_cpuid_0b:
- mov $0x0B, %eax
- xorl %ecx, %ecx
- cpuid
- jmp .Lsetup_AP
+.Lread_apicid:
+ mov $MSR_IA32_APICBASE, %ecx
+ rdmsr
+ testl $X2APIC_ENABLE, %eax
+ jnz read_apicid_msr
+
+ /* Read the APIC ID from the fix-mapped MMIO space. */
+ movq apic_mmio_base(%rip), %rcx
+ addq $APIC_ID, %rcx
+ movl (%rcx), %eax
+ shr $24, %eax
+ jnz .Lread_apicid
-.Luse_cpuid_1f:
- mov $0x1f, %eax
- xorl %ecx, %ecx
- cpuid
+.Lread_apicid_msr:
+ mov $APIC_X2APIC_ID_MSR, %ecx
+ rdmsr
.Lsetup_AP:
- /* EDX contains the APIC ID of the current CPU */
+ /* EAX contains the APIC ID of the current CPU */
xorq %rcx, %rcx
leaq cpuid_to_apicid(%rip), %rbx
.Lfind_cpunr:
- cmpl (%rbx,%rcx,4), %edx
+ cmpl (%rbx,%rcx,4), %eax
jz .Lsetup_cpu
inc %ecx
#ifdef CONFIG_FORCE_NR_CPUS
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -1253,41 +1253,22 @@ bool __init arch_cpuhp_init_parallel_bri
return false;
}
- /* Encrypted guests require special CPUID handling. */
+ /* Encrypted guests require special handling. */
if (cc_platform_has(CC_ATTR_GUEST_STATE_ENCRYPT)) {
switch (cc_get_vendor()) {
case CC_VENDOR_AMD:
ctrl = STARTUP_APICID_SEV_ES;
if (topology_extended_leaf == 0x0b)
- goto setup;
+ break;
fallthrough;
default:
pr_info("Parallel CPU startup disabled due to guest state encryption\n");
return false;
}
+ } else {
+ ctrl = STARTUP_READ_APICID;
}
- switch (topology_extended_leaf) {
- case 0x0b:
- ctrl = STARTUP_APICID_CPUID_0B;
- break;
- case 0x1f:
- ctrl = STARTUP_APICID_CPUID_1F;
- break;
- case 0x00:
- /* For !x2APIC mode 8 bits from leaf 0x01 are sufficient. */
- if (!x2apic_mode) {
- ctrl = STARTUP_APICID_CPUID_01;
- break;
- }
- fallthrough;
- default:
- pr_info("Parallel CPU startup disabled. Unsupported topology leaf %u\n",
- topology_extended_leaf);
- return false;
- }
-
-setup:
pr_debug("Parallel CPU startup enabled: 0x%08x\n", ctrl);
smpboot_control = ctrl;
return true;
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-20 15:57 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-20 15:57 UTC (permalink / raw)
To: Sean Christopherson
Cc: Andrew Cooper, Paul Menzel, linux-kernel, x86, David Woodhouse,
Brian Gerst, Arjan van de Veen, Paolo Bonzini, Paul McKenney,
Tom Lendacky, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Thu, Apr 20 2023 at 07:51, Sean Christopherson wrote:
> On Thu, Apr 20, 2023, Thomas Gleixner wrote:
>> On Thu, Apr 20 2023 at 10:23, Andrew Cooper wrote:
>> > On 20/04/2023 9:32 am, Thomas Gleixner wrote:
>> > > On Wed, Apr 19, 2023, Andrew Cooper wrote:
>> > > > This was changed in x2APIC, which made the x2APIC_ID immutable.
>>
>> >> I'm pondering to simply deny parallel mode if x2APIC is not there.
>> >
>> > I'm not sure if that will help much.
>>
>> Spoilsport.
>
> LOL, well let me pile on then. x2APIC IDs aren't immutable on AMD hardware. The
> ID is read-only when the CPU is in x2APIC mode, but any changes made to the ID
> while the CPU is in xAPIC mode survive the transition to x2APIC. From the APM:
>
> A value previously written by software to the 8-bit APIC_ID register (MMIO offset
> 30h) is converted by hardware into the appropriate format and reflected into the
> 32-bit x2APIC_ID register (MSR 802h).
>
> FWIW, my observations from testing on bare metal are that the xAPIC ID is effectively
> read-only (writes are dropped) on Intel CPUs as far back as Haswell, while the above
> behavior described in the APM holds true on at least Rome and Milan.
>
> My guess is that Intel's uArch specific behavior of the xAPIC ID being read-only
> was introduced when x2APIC came along, but I didn't test farther back than Haswell.
I'm not so worried about modern hardware. The horrorshow is the old muck
as demonstrated and of course there is virt :)
Something like the completely untested below should just work whatever
APIC ID the BIOS decided to dice.
That might just work on SEV too without that GHCB muck, but what do I
know.
Thanks,
tglx
---
--- a/arch/x86/include/asm/apicdef.h
+++ b/arch/x86/include/asm/apicdef.h
@@ -138,7 +138,8 @@
#define APIC_EILVT_MASKED (1 << 16)
#define APIC_BASE (fix_to_virt(FIX_APIC_BASE))
-#define APIC_BASE_MSR 0x800
+#define APIC_BASE_MSR 0x800
+#define APIC_X2APIC_ID_MSR 0x802
#define XAPIC_ENABLE (1UL << 11)
#define X2APIC_ENABLE (1UL << 10)
@@ -162,6 +163,7 @@
#define APIC_CPUID(apicid) ((apicid) & XAPIC_DEST_CPUS_MASK)
#define NUM_APIC_CLUSTERS ((BAD_APICID + 1) >> XAPIC_DEST_CPUS_SHIFT)
+#ifndef __ASSEMBLY__
/*
* the local APIC register structure, memory mapped. Not terribly well
* tested, but we might eventually use this one in the future - the
@@ -435,4 +437,5 @@ enum apic_delivery_modes {
APIC_DELIVERY_MODE_EXTINT = 7,
};
+#endif /* !__ASSEMBLY__ */
#endif /* _ASM_X86_APICDEF_H */
--- a/arch/x86/include/asm/smp.h
+++ b/arch/x86/include/asm/smp.h
@@ -195,14 +195,13 @@ extern void nmi_selftest(void);
#endif
extern unsigned int smpboot_control;
+extern unsigned long apic_mmio_base;
#endif /* !__ASSEMBLY__ */
/* Control bits for startup_64 */
-#define STARTUP_APICID_CPUID_1F 0x80000000
-#define STARTUP_APICID_CPUID_0B 0x40000000
-#define STARTUP_APICID_CPUID_01 0x20000000
-#define STARTUP_APICID_SEV_ES 0x10000000
+#define STARTUP_READ_APICID 0x80000000
+#define STARTUP_APICID_SEV_ES 0x40000000
/* Top 8 bits are reserved for control */
#define STARTUP_PARALLEL_MASK 0xFF000000
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -101,6 +101,8 @@ static int apic_extnmi __ro_after_init =
*/
static bool virt_ext_dest_id __ro_after_init;
+unsigned long apic_mmio_base __ro_after_init;
+
/*
* Map cpu index to physical APIC ID
*/
@@ -2164,6 +2166,7 @@ void __init register_lapic_address(unsig
if (!x2apic_mode) {
set_fixmap_nocache(FIX_APIC_BASE, address);
+ apic_mmio_base = APIC_BASE;
apic_printk(APIC_VERBOSE, "mapped APIC to %16lx (%16lx)\n",
APIC_BASE, address);
}
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -24,8 +24,10 @@
#include "../entry/calling.h"
#include <asm/export.h>
#include <asm/nospec-branch.h>
+#include <asm/apicdef.h>
#include <asm/fixmap.h>
#include <asm/smp.h>
+
#include <asm/sev-common.h>
/*
@@ -237,37 +239,24 @@ SYM_INNER_LABEL(secondary_startup_64_no_
#ifdef CONFIG_SMP
/*
- * For parallel boot, the APIC ID is retrieved from CPUID, and then
- * used to look up the CPU number. For booting a single CPU, the
- * CPU number is encoded in smpboot_control.
+ * For parallel boot, the APIC ID is either retrieved the APIC or
+ * from CPUID, and then used to look up the CPU number.
+ * For booting a single CPU, the CPU number is encoded in
+ * smpboot_control.
*
- * Bit 31 STARTUP_APICID_CPUID_1F flag (use CPUID 0x1f)
- * Bit 30 STARTUP_APICID_CPUID_0B flag (use CPUID 0x0b)
- * Bit 29 STARTUP_APICID_CPUID_01 flag (use CPUID 0x01)
- * Bit 28 STARTUP_APICID_SEV_ES flag (CPUID 0x0b via GHCB MSR)
+ * Bit 31 STARTUP_APICID_READ (Read APICID from APIC)
+ * Bit 30 STARTUP_APICID_SEV_ES flag (CPUID 0x0b via GHCB MSR)
* Bit 0-23 CPU# if STARTUP_APICID_CPUID_xx flags are not set
*/
movl smpboot_control(%rip), %ecx
+ testl $STARTUP_READ_APICID, %ecx
#ifdef CONFIG_AMD_MEM_ENCRYPT
testl $STARTUP_APICID_SEV_ES, %ecx
jnz .Luse_sev_cpuid_0b
#endif
- testl $STARTUP_APICID_CPUID_1F, %ecx
- jnz .Luse_cpuid_1f
- testl $STARTUP_APICID_CPUID_0B, %ecx
- jnz .Luse_cpuid_0b
- testl $STARTUP_APICID_CPUID_01, %ecx
- jnz .Luse_cpuid_01
andl $(~STARTUP_PARALLEL_MASK), %ecx
jmp .Lsetup_cpu
-.Luse_cpuid_01:
- mov $0x01, %eax
- cpuid
- mov %ebx, %edx
- shr $24, %edx
- jmp .Lsetup_AP
-
#ifdef CONFIG_AMD_MEM_ENCRYPT
.Luse_sev_cpuid_0b:
/* Set the GHCB MSR to request CPUID 0x0B_EDX */
@@ -292,24 +281,30 @@ SYM_INNER_LABEL(secondary_startup_64_no_
jmp .Lsetup_AP
#endif
-.Luse_cpuid_0b:
- mov $0x0B, %eax
- xorl %ecx, %ecx
- cpuid
- jmp .Lsetup_AP
+.Lread_apicid:
+ mov $MSR_IA32_APICBASE, %ecx
+ rdmsr
+ testl $X2APIC_ENABLE, %eax
+ jnz read_apicid_msr
+
+ /* Read the APIC ID from the fix-mapped MMIO space. */
+ movq apic_mmio_base(%rip), %rcx
+ addq $APIC_ID, %rcx
+ movl (%rcx), %eax
+ shr $24, %eax
+ jnz .Lread_apicid
-.Luse_cpuid_1f:
- mov $0x1f, %eax
- xorl %ecx, %ecx
- cpuid
+.Lread_apicid_msr:
+ mov $APIC_X2APIC_ID_MSR, %ecx
+ rdmsr
.Lsetup_AP:
- /* EDX contains the APIC ID of the current CPU */
+ /* EAX contains the APIC ID of the current CPU */
xorq %rcx, %rcx
leaq cpuid_to_apicid(%rip), %rbx
.Lfind_cpunr:
- cmpl (%rbx,%rcx,4), %edx
+ cmpl (%rbx,%rcx,4), %eax
jz .Lsetup_cpu
inc %ecx
#ifdef CONFIG_FORCE_NR_CPUS
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -1253,41 +1253,22 @@ bool __init arch_cpuhp_init_parallel_bri
return false;
}
- /* Encrypted guests require special CPUID handling. */
+ /* Encrypted guests require special handling. */
if (cc_platform_has(CC_ATTR_GUEST_STATE_ENCRYPT)) {
switch (cc_get_vendor()) {
case CC_VENDOR_AMD:
ctrl = STARTUP_APICID_SEV_ES;
if (topology_extended_leaf == 0x0b)
- goto setup;
+ break;
fallthrough;
default:
pr_info("Parallel CPU startup disabled due to guest state encryption\n");
return false;
}
+ } else {
+ ctrl = STARTUP_READ_APICID;
}
- switch (topology_extended_leaf) {
- case 0x0b:
- ctrl = STARTUP_APICID_CPUID_0B;
- break;
- case 0x1f:
- ctrl = STARTUP_APICID_CPUID_1F;
- break;
- case 0x00:
- /* For !x2APIC mode 8 bits from leaf 0x01 are sufficient. */
- if (!x2apic_mode) {
- ctrl = STARTUP_APICID_CPUID_01;
- break;
- }
- fallthrough;
- default:
- pr_info("Parallel CPU startup disabled. Unsupported topology leaf %u\n",
- topology_extended_leaf);
- return false;
- }
-
-setup:
pr_debug("Parallel CPU startup enabled: 0x%08x\n", ctrl);
smpboot_control = ctrl;
return true;
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
2023-04-20 15:57 ` Thomas Gleixner
(?)
@ 2023-04-20 16:47 ` Paul Menzel
-1 siblings, 0 replies; 236+ messages in thread
From: Paul Menzel @ 2023-04-20 16:47 UTC (permalink / raw)
To: Thomas Gleixner
Cc: Sean Christopherson, Andrew Cooper, linux-kernel, x86,
David Woodhouse, Brian Gerst, Arjan van de Veen, Paolo Bonzini,
Paul McKenney, Tom Lendacky, Oleksandr Natalenko,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Dear Thomas,
Am 20.04.23 um 17:57 schrieb Thomas Gleixner:
> On Thu, Apr 20 2023 at 07:51, Sean Christopherson wrote:
>> On Thu, Apr 20, 2023, Thomas Gleixner wrote:
>>> On Thu, Apr 20 2023 at 10:23, Andrew Cooper wrote:
>>>> On 20/04/2023 9:32 am, Thomas Gleixner wrote:
>>>>> On Wed, Apr 19, 2023, Andrew Cooper wrote:
>>>>>> This was changed in x2APIC, which made the x2APIC_ID immutable.
>>>
>>>>> I'm pondering to simply deny parallel mode if x2APIC is not there.
>>>>
>>>> I'm not sure if that will help much.
>>>
>>> Spoilsport.
>>
>> LOL, well let me pile on then. x2APIC IDs aren't immutable on AMD hardware. The
>> ID is read-only when the CPU is in x2APIC mode, but any changes made to the ID
>> while the CPU is in xAPIC mode survive the transition to x2APIC. From the APM:
>>
>> A value previously written by software to the 8-bit APIC_ID register (MMIO offset
>> 30h) is converted by hardware into the appropriate format and reflected into the
>> 32-bit x2APIC_ID register (MSR 802h).
>>
>> FWIW, my observations from testing on bare metal are that the xAPIC ID is effectively
>> read-only (writes are dropped) on Intel CPUs as far back as Haswell, while the above
>> behavior described in the APM holds true on at least Rome and Milan.
>>
>> My guess is that Intel's uArch specific behavior of the xAPIC ID being read-only
>> was introduced when x2APIC came along, but I didn't test farther back than Haswell.
>
> I'm not so worried about modern hardware. The horrorshow is the old muck
> as demonstrated and of course there is virt :)
>
> Something like the completely untested below should just work whatever
> APIC ID the BIOS decided to dice.
>
> That might just work on SEV too without that GHCB muck, but what do I
> know.
>
> Thanks,
>
> tglx
> ---
> --- a/arch/x86/include/asm/apicdef.h
> +++ b/arch/x86/include/asm/apicdef.h
> @@ -138,7 +138,8 @@
> #define APIC_EILVT_MASKED (1 << 16)
>
> #define APIC_BASE (fix_to_virt(FIX_APIC_BASE))
> -#define APIC_BASE_MSR 0x800
> +#define APIC_BASE_MSR 0x800
> +#define APIC_X2APIC_ID_MSR 0x802
> #define XAPIC_ENABLE (1UL << 11)
> #define X2APIC_ENABLE (1UL << 10)
>
> @@ -162,6 +163,7 @@
> #define APIC_CPUID(apicid) ((apicid) & XAPIC_DEST_CPUS_MASK)
> #define NUM_APIC_CLUSTERS ((BAD_APICID + 1) >> XAPIC_DEST_CPUS_SHIFT)
>
> +#ifndef __ASSEMBLY__
> /*
> * the local APIC register structure, memory mapped. Not terribly well
> * tested, but we might eventually use this one in the future - the
> @@ -435,4 +437,5 @@ enum apic_delivery_modes {
> APIC_DELIVERY_MODE_EXTINT = 7,
> };
>
> +#endif /* !__ASSEMBLY__ */
> #endif /* _ASM_X86_APICDEF_H */
> --- a/arch/x86/include/asm/smp.h
> +++ b/arch/x86/include/asm/smp.h
> @@ -195,14 +195,13 @@ extern void nmi_selftest(void);
> #endif
>
> extern unsigned int smpboot_control;
> +extern unsigned long apic_mmio_base;
>
> #endif /* !__ASSEMBLY__ */
>
> /* Control bits for startup_64 */
> -#define STARTUP_APICID_CPUID_1F 0x80000000
> -#define STARTUP_APICID_CPUID_0B 0x40000000
> -#define STARTUP_APICID_CPUID_01 0x20000000
> -#define STARTUP_APICID_SEV_ES 0x10000000
> +#define STARTUP_READ_APICID 0x80000000
> +#define STARTUP_APICID_SEV_ES 0x40000000
>
> /* Top 8 bits are reserved for control */
> #define STARTUP_PARALLEL_MASK 0xFF000000
> --- a/arch/x86/kernel/apic/apic.c
> +++ b/arch/x86/kernel/apic/apic.c
> @@ -101,6 +101,8 @@ static int apic_extnmi __ro_after_init =
> */
> static bool virt_ext_dest_id __ro_after_init;
>
> +unsigned long apic_mmio_base __ro_after_init;
> +
> /*
> * Map cpu index to physical APIC ID
> */
> @@ -2164,6 +2166,7 @@ void __init register_lapic_address(unsig
>
> if (!x2apic_mode) {
> set_fixmap_nocache(FIX_APIC_BASE, address);
> + apic_mmio_base = APIC_BASE;
> apic_printk(APIC_VERBOSE, "mapped APIC to %16lx (%16lx)\n",
> APIC_BASE, address);
> }
> --- a/arch/x86/kernel/head_64.S
> +++ b/arch/x86/kernel/head_64.S
> @@ -24,8 +24,10 @@
> #include "../entry/calling.h"
> #include <asm/export.h>
> #include <asm/nospec-branch.h>
> +#include <asm/apicdef.h>
> #include <asm/fixmap.h>
> #include <asm/smp.h>
> +
> #include <asm/sev-common.h>
>
> /*
> @@ -237,37 +239,24 @@ SYM_INNER_LABEL(secondary_startup_64_no_
>
> #ifdef CONFIG_SMP
> /*
> - * For parallel boot, the APIC ID is retrieved from CPUID, and then
> - * used to look up the CPU number. For booting a single CPU, the
> - * CPU number is encoded in smpboot_control.
> + * For parallel boot, the APIC ID is either retrieved the APIC or
> + * from CPUID, and then used to look up the CPU number.
> + * For booting a single CPU, the CPU number is encoded in
> + * smpboot_control.
> *
> - * Bit 31 STARTUP_APICID_CPUID_1F flag (use CPUID 0x1f)
> - * Bit 30 STARTUP_APICID_CPUID_0B flag (use CPUID 0x0b)
> - * Bit 29 STARTUP_APICID_CPUID_01 flag (use CPUID 0x01)
> - * Bit 28 STARTUP_APICID_SEV_ES flag (CPUID 0x0b via GHCB MSR)
> + * Bit 31 STARTUP_APICID_READ (Read APICID from APIC)
> + * Bit 30 STARTUP_APICID_SEV_ES flag (CPUID 0x0b via GHCB MSR)
> * Bit 0-23 CPU# if STARTUP_APICID_CPUID_xx flags are not set
> */
> movl smpboot_control(%rip), %ecx
> + testl $STARTUP_READ_APICID, %ecx
> #ifdef CONFIG_AMD_MEM_ENCRYPT
> testl $STARTUP_APICID_SEV_ES, %ecx
> jnz .Luse_sev_cpuid_0b
> #endif
> - testl $STARTUP_APICID_CPUID_1F, %ecx
> - jnz .Luse_cpuid_1f
> - testl $STARTUP_APICID_CPUID_0B, %ecx
> - jnz .Luse_cpuid_0b
> - testl $STARTUP_APICID_CPUID_01, %ecx
> - jnz .Luse_cpuid_01
> andl $(~STARTUP_PARALLEL_MASK), %ecx
> jmp .Lsetup_cpu
>
> -.Luse_cpuid_01:
> - mov $0x01, %eax
> - cpuid
> - mov %ebx, %edx
> - shr $24, %edx
> - jmp .Lsetup_AP
> -
> #ifdef CONFIG_AMD_MEM_ENCRYPT
> .Luse_sev_cpuid_0b:
> /* Set the GHCB MSR to request CPUID 0x0B_EDX */
> @@ -292,24 +281,30 @@ SYM_INNER_LABEL(secondary_startup_64_no_
> jmp .Lsetup_AP
> #endif
>
> -.Luse_cpuid_0b:
> - mov $0x0B, %eax
> - xorl %ecx, %ecx
> - cpuid
> - jmp .Lsetup_AP
> +.Lread_apicid:
> + mov $MSR_IA32_APICBASE, %ecx
> + rdmsr
> + testl $X2APIC_ENABLE, %eax
> + jnz read_apicid_msr
> +
> + /* Read the APIC ID from the fix-mapped MMIO space. */
> + movq apic_mmio_base(%rip), %rcx
> + addq $APIC_ID, %rcx
> + movl (%rcx), %eax
> + shr $24, %eax
> + jnz .Lread_apicid
>
> -.Luse_cpuid_1f:
> - mov $0x1f, %eax
> - xorl %ecx, %ecx
> - cpuid
> +.Lread_apicid_msr:
> + mov $APIC_X2APIC_ID_MSR, %ecx
> + rdmsr
>
> .Lsetup_AP:
> - /* EDX contains the APIC ID of the current CPU */
> + /* EAX contains the APIC ID of the current CPU */
> xorq %rcx, %rcx
> leaq cpuid_to_apicid(%rip), %rbx
>
> .Lfind_cpunr:
> - cmpl (%rbx,%rcx,4), %edx
> + cmpl (%rbx,%rcx,4), %eax
> jz .Lsetup_cpu
> inc %ecx
> #ifdef CONFIG_FORCE_NR_CPUS
> --- a/arch/x86/kernel/smpboot.c
> +++ b/arch/x86/kernel/smpboot.c
> @@ -1253,41 +1253,22 @@ bool __init arch_cpuhp_init_parallel_bri
> return false;
> }
>
> - /* Encrypted guests require special CPUID handling. */
> + /* Encrypted guests require special handling. */
> if (cc_platform_has(CC_ATTR_GUEST_STATE_ENCRYPT)) {
> switch (cc_get_vendor()) {
> case CC_VENDOR_AMD:
> ctrl = STARTUP_APICID_SEV_ES;
> if (topology_extended_leaf == 0x0b)
> - goto setup;
> + break;
> fallthrough;
> default:
> pr_info("Parallel CPU startup disabled due to guest state encryption\n");
> return false;
> }
> + } else {
> + ctrl = STARTUP_READ_APICID;
> }
>
> - switch (topology_extended_leaf) {
> - case 0x0b:
> - ctrl = STARTUP_APICID_CPUID_0B;
> - break;
> - case 0x1f:
> - ctrl = STARTUP_APICID_CPUID_1F;
> - break;
> - case 0x00:
> - /* For !x2APIC mode 8 bits from leaf 0x01 are sufficient. */
> - if (!x2apic_mode) {
> - ctrl = STARTUP_APICID_CPUID_01;
> - break;
> - }
> - fallthrough;
> - default:
> - pr_info("Parallel CPU startup disabled. Unsupported topology leaf %u\n",
> - topology_extended_leaf);
> - return false;
> - }
> -
> -setup:
> pr_debug("Parallel CPU startup enabled: 0x%08x\n", ctrl);
> smpboot_control = ctrl;
> return true;
I quickly applied it on top of your branch, but I am getting:
```
$ wget https://lore.kernel.org/lkml/87v8hq35sk.ffs@tglx/raw
$ patch -p1 < raw
$ make
[…]
LD .tmp_vmlinux.kallsyms1
ld: arch/x86/kernel/head_64.o: in function `secondary_startup_64_no_verify':
(.head.text+0xbf): undefined reference to `read_apicid_msr'
make[1]: *** [scripts/Makefile.vmlinux:35: vmlinux] Error 1
make: *** [Makefile:1249: vmlinux] Error 2
```
Kind regards,
Paul
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-20 16:47 ` Paul Menzel
0 siblings, 0 replies; 236+ messages in thread
From: Paul Menzel @ 2023-04-20 16:47 UTC (permalink / raw)
To: Thomas Gleixner
Cc: Sean Christopherson, Andrew Cooper, linux-kernel, x86,
David Woodhouse, Brian Gerst, Arjan van de Veen, Paolo Bonzini,
Paul McKenney, Tom Lendacky, Oleksandr Natalenko,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Dear Thomas,
Am 20.04.23 um 17:57 schrieb Thomas Gleixner:
> On Thu, Apr 20 2023 at 07:51, Sean Christopherson wrote:
>> On Thu, Apr 20, 2023, Thomas Gleixner wrote:
>>> On Thu, Apr 20 2023 at 10:23, Andrew Cooper wrote:
>>>> On 20/04/2023 9:32 am, Thomas Gleixner wrote:
>>>>> On Wed, Apr 19, 2023, Andrew Cooper wrote:
>>>>>> This was changed in x2APIC, which made the x2APIC_ID immutable.
>>>
>>>>> I'm pondering to simply deny parallel mode if x2APIC is not there.
>>>>
>>>> I'm not sure if that will help much.
>>>
>>> Spoilsport.
>>
>> LOL, well let me pile on then. x2APIC IDs aren't immutable on AMD hardware. The
>> ID is read-only when the CPU is in x2APIC mode, but any changes made to the ID
>> while the CPU is in xAPIC mode survive the transition to x2APIC. From the APM:
>>
>> A value previously written by software to the 8-bit APIC_ID register (MMIO offset
>> 30h) is converted by hardware into the appropriate format and reflected into the
>> 32-bit x2APIC_ID register (MSR 802h).
>>
>> FWIW, my observations from testing on bare metal are that the xAPIC ID is effectively
>> read-only (writes are dropped) on Intel CPUs as far back as Haswell, while the above
>> behavior described in the APM holds true on at least Rome and Milan.
>>
>> My guess is that Intel's uArch specific behavior of the xAPIC ID being read-only
>> was introduced when x2APIC came along, but I didn't test farther back than Haswell.
>
> I'm not so worried about modern hardware. The horrorshow is the old muck
> as demonstrated and of course there is virt :)
>
> Something like the completely untested below should just work whatever
> APIC ID the BIOS decided to dice.
>
> That might just work on SEV too without that GHCB muck, but what do I
> know.
>
> Thanks,
>
> tglx
> ---
> --- a/arch/x86/include/asm/apicdef.h
> +++ b/arch/x86/include/asm/apicdef.h
> @@ -138,7 +138,8 @@
> #define APIC_EILVT_MASKED (1 << 16)
>
> #define APIC_BASE (fix_to_virt(FIX_APIC_BASE))
> -#define APIC_BASE_MSR 0x800
> +#define APIC_BASE_MSR 0x800
> +#define APIC_X2APIC_ID_MSR 0x802
> #define XAPIC_ENABLE (1UL << 11)
> #define X2APIC_ENABLE (1UL << 10)
>
> @@ -162,6 +163,7 @@
> #define APIC_CPUID(apicid) ((apicid) & XAPIC_DEST_CPUS_MASK)
> #define NUM_APIC_CLUSTERS ((BAD_APICID + 1) >> XAPIC_DEST_CPUS_SHIFT)
>
> +#ifndef __ASSEMBLY__
> /*
> * the local APIC register structure, memory mapped. Not terribly well
> * tested, but we might eventually use this one in the future - the
> @@ -435,4 +437,5 @@ enum apic_delivery_modes {
> APIC_DELIVERY_MODE_EXTINT = 7,
> };
>
> +#endif /* !__ASSEMBLY__ */
> #endif /* _ASM_X86_APICDEF_H */
> --- a/arch/x86/include/asm/smp.h
> +++ b/arch/x86/include/asm/smp.h
> @@ -195,14 +195,13 @@ extern void nmi_selftest(void);
> #endif
>
> extern unsigned int smpboot_control;
> +extern unsigned long apic_mmio_base;
>
> #endif /* !__ASSEMBLY__ */
>
> /* Control bits for startup_64 */
> -#define STARTUP_APICID_CPUID_1F 0x80000000
> -#define STARTUP_APICID_CPUID_0B 0x40000000
> -#define STARTUP_APICID_CPUID_01 0x20000000
> -#define STARTUP_APICID_SEV_ES 0x10000000
> +#define STARTUP_READ_APICID 0x80000000
> +#define STARTUP_APICID_SEV_ES 0x40000000
>
> /* Top 8 bits are reserved for control */
> #define STARTUP_PARALLEL_MASK 0xFF000000
> --- a/arch/x86/kernel/apic/apic.c
> +++ b/arch/x86/kernel/apic/apic.c
> @@ -101,6 +101,8 @@ static int apic_extnmi __ro_after_init =
> */
> static bool virt_ext_dest_id __ro_after_init;
>
> +unsigned long apic_mmio_base __ro_after_init;
> +
> /*
> * Map cpu index to physical APIC ID
> */
> @@ -2164,6 +2166,7 @@ void __init register_lapic_address(unsig
>
> if (!x2apic_mode) {
> set_fixmap_nocache(FIX_APIC_BASE, address);
> + apic_mmio_base = APIC_BASE;
> apic_printk(APIC_VERBOSE, "mapped APIC to %16lx (%16lx)\n",
> APIC_BASE, address);
> }
> --- a/arch/x86/kernel/head_64.S
> +++ b/arch/x86/kernel/head_64.S
> @@ -24,8 +24,10 @@
> #include "../entry/calling.h"
> #include <asm/export.h>
> #include <asm/nospec-branch.h>
> +#include <asm/apicdef.h>
> #include <asm/fixmap.h>
> #include <asm/smp.h>
> +
> #include <asm/sev-common.h>
>
> /*
> @@ -237,37 +239,24 @@ SYM_INNER_LABEL(secondary_startup_64_no_
>
> #ifdef CONFIG_SMP
> /*
> - * For parallel boot, the APIC ID is retrieved from CPUID, and then
> - * used to look up the CPU number. For booting a single CPU, the
> - * CPU number is encoded in smpboot_control.
> + * For parallel boot, the APIC ID is either retrieved the APIC or
> + * from CPUID, and then used to look up the CPU number.
> + * For booting a single CPU, the CPU number is encoded in
> + * smpboot_control.
> *
> - * Bit 31 STARTUP_APICID_CPUID_1F flag (use CPUID 0x1f)
> - * Bit 30 STARTUP_APICID_CPUID_0B flag (use CPUID 0x0b)
> - * Bit 29 STARTUP_APICID_CPUID_01 flag (use CPUID 0x01)
> - * Bit 28 STARTUP_APICID_SEV_ES flag (CPUID 0x0b via GHCB MSR)
> + * Bit 31 STARTUP_APICID_READ (Read APICID from APIC)
> + * Bit 30 STARTUP_APICID_SEV_ES flag (CPUID 0x0b via GHCB MSR)
> * Bit 0-23 CPU# if STARTUP_APICID_CPUID_xx flags are not set
> */
> movl smpboot_control(%rip), %ecx
> + testl $STARTUP_READ_APICID, %ecx
> #ifdef CONFIG_AMD_MEM_ENCRYPT
> testl $STARTUP_APICID_SEV_ES, %ecx
> jnz .Luse_sev_cpuid_0b
> #endif
> - testl $STARTUP_APICID_CPUID_1F, %ecx
> - jnz .Luse_cpuid_1f
> - testl $STARTUP_APICID_CPUID_0B, %ecx
> - jnz .Luse_cpuid_0b
> - testl $STARTUP_APICID_CPUID_01, %ecx
> - jnz .Luse_cpuid_01
> andl $(~STARTUP_PARALLEL_MASK), %ecx
> jmp .Lsetup_cpu
>
> -.Luse_cpuid_01:
> - mov $0x01, %eax
> - cpuid
> - mov %ebx, %edx
> - shr $24, %edx
> - jmp .Lsetup_AP
> -
> #ifdef CONFIG_AMD_MEM_ENCRYPT
> .Luse_sev_cpuid_0b:
> /* Set the GHCB MSR to request CPUID 0x0B_EDX */
> @@ -292,24 +281,30 @@ SYM_INNER_LABEL(secondary_startup_64_no_
> jmp .Lsetup_AP
> #endif
>
> -.Luse_cpuid_0b:
> - mov $0x0B, %eax
> - xorl %ecx, %ecx
> - cpuid
> - jmp .Lsetup_AP
> +.Lread_apicid:
> + mov $MSR_IA32_APICBASE, %ecx
> + rdmsr
> + testl $X2APIC_ENABLE, %eax
> + jnz read_apicid_msr
> +
> + /* Read the APIC ID from the fix-mapped MMIO space. */
> + movq apic_mmio_base(%rip), %rcx
> + addq $APIC_ID, %rcx
> + movl (%rcx), %eax
> + shr $24, %eax
> + jnz .Lread_apicid
>
> -.Luse_cpuid_1f:
> - mov $0x1f, %eax
> - xorl %ecx, %ecx
> - cpuid
> +.Lread_apicid_msr:
> + mov $APIC_X2APIC_ID_MSR, %ecx
> + rdmsr
>
> .Lsetup_AP:
> - /* EDX contains the APIC ID of the current CPU */
> + /* EAX contains the APIC ID of the current CPU */
> xorq %rcx, %rcx
> leaq cpuid_to_apicid(%rip), %rbx
>
> .Lfind_cpunr:
> - cmpl (%rbx,%rcx,4), %edx
> + cmpl (%rbx,%rcx,4), %eax
> jz .Lsetup_cpu
> inc %ecx
> #ifdef CONFIG_FORCE_NR_CPUS
> --- a/arch/x86/kernel/smpboot.c
> +++ b/arch/x86/kernel/smpboot.c
> @@ -1253,41 +1253,22 @@ bool __init arch_cpuhp_init_parallel_bri
> return false;
> }
>
> - /* Encrypted guests require special CPUID handling. */
> + /* Encrypted guests require special handling. */
> if (cc_platform_has(CC_ATTR_GUEST_STATE_ENCRYPT)) {
> switch (cc_get_vendor()) {
> case CC_VENDOR_AMD:
> ctrl = STARTUP_APICID_SEV_ES;
> if (topology_extended_leaf == 0x0b)
> - goto setup;
> + break;
> fallthrough;
> default:
> pr_info("Parallel CPU startup disabled due to guest state encryption\n");
> return false;
> }
> + } else {
> + ctrl = STARTUP_READ_APICID;
> }
>
> - switch (topology_extended_leaf) {
> - case 0x0b:
> - ctrl = STARTUP_APICID_CPUID_0B;
> - break;
> - case 0x1f:
> - ctrl = STARTUP_APICID_CPUID_1F;
> - break;
> - case 0x00:
> - /* For !x2APIC mode 8 bits from leaf 0x01 are sufficient. */
> - if (!x2apic_mode) {
> - ctrl = STARTUP_APICID_CPUID_01;
> - break;
> - }
> - fallthrough;
> - default:
> - pr_info("Parallel CPU startup disabled. Unsupported topology leaf %u\n",
> - topology_extended_leaf);
> - return false;
> - }
> -
> -setup:
> pr_debug("Parallel CPU startup enabled: 0x%08x\n", ctrl);
> smpboot_control = ctrl;
> return true;
I quickly applied it on top of your branch, but I am getting:
```
$ wget https://lore.kernel.org/lkml/87v8hq35sk.ffs@tglx/raw
$ patch -p1 < raw
$ make
[…]
LD .tmp_vmlinux.kallsyms1
ld: arch/x86/kernel/head_64.o: in function `secondary_startup_64_no_verify':
(.head.text+0xbf): undefined reference to `read_apicid_msr'
make[1]: *** [scripts/Makefile.vmlinux:35: vmlinux] Error 1
make: *** [Makefile:1249: vmlinux] Error 2
```
Kind regards,
Paul
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-20 16:47 ` Paul Menzel
0 siblings, 0 replies; 236+ messages in thread
From: Paul Menzel @ 2023-04-20 16:47 UTC (permalink / raw)
To: Thomas Gleixner
Cc: Sean Christopherson, Andrew Cooper, linux-kernel, x86,
David Woodhouse, Brian Gerst, Arjan van de Veen, Paolo Bonzini,
Paul McKenney, Tom Lendacky, Oleksandr Natalenko,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Dear Thomas,
Am 20.04.23 um 17:57 schrieb Thomas Gleixner:
> On Thu, Apr 20 2023 at 07:51, Sean Christopherson wrote:
>> On Thu, Apr 20, 2023, Thomas Gleixner wrote:
>>> On Thu, Apr 20 2023 at 10:23, Andrew Cooper wrote:
>>>> On 20/04/2023 9:32 am, Thomas Gleixner wrote:
>>>>> On Wed, Apr 19, 2023, Andrew Cooper wrote:
>>>>>> This was changed in x2APIC, which made the x2APIC_ID immutable.
>>>
>>>>> I'm pondering to simply deny parallel mode if x2APIC is not there.
>>>>
>>>> I'm not sure if that will help much.
>>>
>>> Spoilsport.
>>
>> LOL, well let me pile on then. x2APIC IDs aren't immutable on AMD hardware. The
>> ID is read-only when the CPU is in x2APIC mode, but any changes made to the ID
>> while the CPU is in xAPIC mode survive the transition to x2APIC. From the APM:
>>
>> A value previously written by software to the 8-bit APIC_ID register (MMIO offset
>> 30h) is converted by hardware into the appropriate format and reflected into the
>> 32-bit x2APIC_ID register (MSR 802h).
>>
>> FWIW, my observations from testing on bare metal are that the xAPIC ID is effectively
>> read-only (writes are dropped) on Intel CPUs as far back as Haswell, while the above
>> behavior described in the APM holds true on at least Rome and Milan.
>>
>> My guess is that Intel's uArch specific behavior of the xAPIC ID being read-only
>> was introduced when x2APIC came along, but I didn't test farther back than Haswell.
>
> I'm not so worried about modern hardware. The horrorshow is the old muck
> as demonstrated and of course there is virt :)
>
> Something like the completely untested below should just work whatever
> APIC ID the BIOS decided to dice.
>
> That might just work on SEV too without that GHCB muck, but what do I
> know.
>
> Thanks,
>
> tglx
> ---
> --- a/arch/x86/include/asm/apicdef.h
> +++ b/arch/x86/include/asm/apicdef.h
> @@ -138,7 +138,8 @@
> #define APIC_EILVT_MASKED (1 << 16)
>
> #define APIC_BASE (fix_to_virt(FIX_APIC_BASE))
> -#define APIC_BASE_MSR 0x800
> +#define APIC_BASE_MSR 0x800
> +#define APIC_X2APIC_ID_MSR 0x802
> #define XAPIC_ENABLE (1UL << 11)
> #define X2APIC_ENABLE (1UL << 10)
>
> @@ -162,6 +163,7 @@
> #define APIC_CPUID(apicid) ((apicid) & XAPIC_DEST_CPUS_MASK)
> #define NUM_APIC_CLUSTERS ((BAD_APICID + 1) >> XAPIC_DEST_CPUS_SHIFT)
>
> +#ifndef __ASSEMBLY__
> /*
> * the local APIC register structure, memory mapped. Not terribly well
> * tested, but we might eventually use this one in the future - the
> @@ -435,4 +437,5 @@ enum apic_delivery_modes {
> APIC_DELIVERY_MODE_EXTINT = 7,
> };
>
> +#endif /* !__ASSEMBLY__ */
> #endif /* _ASM_X86_APICDEF_H */
> --- a/arch/x86/include/asm/smp.h
> +++ b/arch/x86/include/asm/smp.h
> @@ -195,14 +195,13 @@ extern void nmi_selftest(void);
> #endif
>
> extern unsigned int smpboot_control;
> +extern unsigned long apic_mmio_base;
>
> #endif /* !__ASSEMBLY__ */
>
> /* Control bits for startup_64 */
> -#define STARTUP_APICID_CPUID_1F 0x80000000
> -#define STARTUP_APICID_CPUID_0B 0x40000000
> -#define STARTUP_APICID_CPUID_01 0x20000000
> -#define STARTUP_APICID_SEV_ES 0x10000000
> +#define STARTUP_READ_APICID 0x80000000
> +#define STARTUP_APICID_SEV_ES 0x40000000
>
> /* Top 8 bits are reserved for control */
> #define STARTUP_PARALLEL_MASK 0xFF000000
> --- a/arch/x86/kernel/apic/apic.c
> +++ b/arch/x86/kernel/apic/apic.c
> @@ -101,6 +101,8 @@ static int apic_extnmi __ro_after_init =
> */
> static bool virt_ext_dest_id __ro_after_init;
>
> +unsigned long apic_mmio_base __ro_after_init;
> +
> /*
> * Map cpu index to physical APIC ID
> */
> @@ -2164,6 +2166,7 @@ void __init register_lapic_address(unsig
>
> if (!x2apic_mode) {
> set_fixmap_nocache(FIX_APIC_BASE, address);
> + apic_mmio_base = APIC_BASE;
> apic_printk(APIC_VERBOSE, "mapped APIC to %16lx (%16lx)\n",
> APIC_BASE, address);
> }
> --- a/arch/x86/kernel/head_64.S
> +++ b/arch/x86/kernel/head_64.S
> @@ -24,8 +24,10 @@
> #include "../entry/calling.h"
> #include <asm/export.h>
> #include <asm/nospec-branch.h>
> +#include <asm/apicdef.h>
> #include <asm/fixmap.h>
> #include <asm/smp.h>
> +
> #include <asm/sev-common.h>
>
> /*
> @@ -237,37 +239,24 @@ SYM_INNER_LABEL(secondary_startup_64_no_
>
> #ifdef CONFIG_SMP
> /*
> - * For parallel boot, the APIC ID is retrieved from CPUID, and then
> - * used to look up the CPU number. For booting a single CPU, the
> - * CPU number is encoded in smpboot_control.
> + * For parallel boot, the APIC ID is either retrieved the APIC or
> + * from CPUID, and then used to look up the CPU number.
> + * For booting a single CPU, the CPU number is encoded in
> + * smpboot_control.
> *
> - * Bit 31 STARTUP_APICID_CPUID_1F flag (use CPUID 0x1f)
> - * Bit 30 STARTUP_APICID_CPUID_0B flag (use CPUID 0x0b)
> - * Bit 29 STARTUP_APICID_CPUID_01 flag (use CPUID 0x01)
> - * Bit 28 STARTUP_APICID_SEV_ES flag (CPUID 0x0b via GHCB MSR)
> + * Bit 31 STARTUP_APICID_READ (Read APICID from APIC)
> + * Bit 30 STARTUP_APICID_SEV_ES flag (CPUID 0x0b via GHCB MSR)
> * Bit 0-23 CPU# if STARTUP_APICID_CPUID_xx flags are not set
> */
> movl smpboot_control(%rip), %ecx
> + testl $STARTUP_READ_APICID, %ecx
> #ifdef CONFIG_AMD_MEM_ENCRYPT
> testl $STARTUP_APICID_SEV_ES, %ecx
> jnz .Luse_sev_cpuid_0b
> #endif
> - testl $STARTUP_APICID_CPUID_1F, %ecx
> - jnz .Luse_cpuid_1f
> - testl $STARTUP_APICID_CPUID_0B, %ecx
> - jnz .Luse_cpuid_0b
> - testl $STARTUP_APICID_CPUID_01, %ecx
> - jnz .Luse_cpuid_01
> andl $(~STARTUP_PARALLEL_MASK), %ecx
> jmp .Lsetup_cpu
>
> -.Luse_cpuid_01:
> - mov $0x01, %eax
> - cpuid
> - mov %ebx, %edx
> - shr $24, %edx
> - jmp .Lsetup_AP
> -
> #ifdef CONFIG_AMD_MEM_ENCRYPT
> .Luse_sev_cpuid_0b:
> /* Set the GHCB MSR to request CPUID 0x0B_EDX */
> @@ -292,24 +281,30 @@ SYM_INNER_LABEL(secondary_startup_64_no_
> jmp .Lsetup_AP
> #endif
>
> -.Luse_cpuid_0b:
> - mov $0x0B, %eax
> - xorl %ecx, %ecx
> - cpuid
> - jmp .Lsetup_AP
> +.Lread_apicid:
> + mov $MSR_IA32_APICBASE, %ecx
> + rdmsr
> + testl $X2APIC_ENABLE, %eax
> + jnz read_apicid_msr
> +
> + /* Read the APIC ID from the fix-mapped MMIO space. */
> + movq apic_mmio_base(%rip), %rcx
> + addq $APIC_ID, %rcx
> + movl (%rcx), %eax
> + shr $24, %eax
> + jnz .Lread_apicid
>
> -.Luse_cpuid_1f:
> - mov $0x1f, %eax
> - xorl %ecx, %ecx
> - cpuid
> +.Lread_apicid_msr:
> + mov $APIC_X2APIC_ID_MSR, %ecx
> + rdmsr
>
> .Lsetup_AP:
> - /* EDX contains the APIC ID of the current CPU */
> + /* EAX contains the APIC ID of the current CPU */
> xorq %rcx, %rcx
> leaq cpuid_to_apicid(%rip), %rbx
>
> .Lfind_cpunr:
> - cmpl (%rbx,%rcx,4), %edx
> + cmpl (%rbx,%rcx,4), %eax
> jz .Lsetup_cpu
> inc %ecx
> #ifdef CONFIG_FORCE_NR_CPUS
> --- a/arch/x86/kernel/smpboot.c
> +++ b/arch/x86/kernel/smpboot.c
> @@ -1253,41 +1253,22 @@ bool __init arch_cpuhp_init_parallel_bri
> return false;
> }
>
> - /* Encrypted guests require special CPUID handling. */
> + /* Encrypted guests require special handling. */
> if (cc_platform_has(CC_ATTR_GUEST_STATE_ENCRYPT)) {
> switch (cc_get_vendor()) {
> case CC_VENDOR_AMD:
> ctrl = STARTUP_APICID_SEV_ES;
> if (topology_extended_leaf == 0x0b)
> - goto setup;
> + break;
> fallthrough;
> default:
> pr_info("Parallel CPU startup disabled due to guest state encryption\n");
> return false;
> }
> + } else {
> + ctrl = STARTUP_READ_APICID;
> }
>
> - switch (topology_extended_leaf) {
> - case 0x0b:
> - ctrl = STARTUP_APICID_CPUID_0B;
> - break;
> - case 0x1f:
> - ctrl = STARTUP_APICID_CPUID_1F;
> - break;
> - case 0x00:
> - /* For !x2APIC mode 8 bits from leaf 0x01 are sufficient. */
> - if (!x2apic_mode) {
> - ctrl = STARTUP_APICID_CPUID_01;
> - break;
> - }
> - fallthrough;
> - default:
> - pr_info("Parallel CPU startup disabled. Unsupported topology leaf %u\n",
> - topology_extended_leaf);
> - return false;
> - }
> -
> -setup:
> pr_debug("Parallel CPU startup enabled: 0x%08x\n", ctrl);
> smpboot_control = ctrl;
> return true;
I quickly applied it on top of your branch, but I am getting:
```
$ wget https://lore.kernel.org/lkml/87v8hq35sk.ffs@tglx/raw
$ patch -p1 < raw
$ make
[…]
LD .tmp_vmlinux.kallsyms1
ld: arch/x86/kernel/head_64.o: in function `secondary_startup_64_no_verify':
(.head.text+0xbf): undefined reference to `read_apicid_msr'
make[1]: *** [scripts/Makefile.vmlinux:35: vmlinux] Error 1
make: *** [Makefile:1249: vmlinux] Error 2
```
Kind regards,
Paul
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
2023-04-20 16:47 ` Paul Menzel
(?)
@ 2023-04-20 19:10 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-20 19:10 UTC (permalink / raw)
To: Paul Menzel
Cc: Sean Christopherson, Andrew Cooper, linux-kernel, x86,
David Woodhouse, Brian Gerst, Arjan van de Veen, Paolo Bonzini,
Paul McKenney, Tom Lendacky, Oleksandr Natalenko,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Thu, Apr 20 2023 at 18:47, Paul Menzel wrote:
> Am 20.04.23 um 17:57 schrieb Thomas Gleixner:
> I quickly applied it on top of your branch, but I am getting:
As I said it was untested. I was traveling and did not have access to a
machine to even build it completely. Fixed up and tested version below.
Thanks,
tglx
---
--- a/arch/x86/include/asm/apicdef.h
+++ b/arch/x86/include/asm/apicdef.h
@@ -138,7 +138,8 @@
#define APIC_EILVT_MASKED (1 << 16)
#define APIC_BASE (fix_to_virt(FIX_APIC_BASE))
-#define APIC_BASE_MSR 0x800
+#define APIC_BASE_MSR 0x800
+#define APIC_X2APIC_ID_MSR 0x802
#define XAPIC_ENABLE (1UL << 11)
#define X2APIC_ENABLE (1UL << 10)
@@ -162,6 +163,7 @@
#define APIC_CPUID(apicid) ((apicid) & XAPIC_DEST_CPUS_MASK)
#define NUM_APIC_CLUSTERS ((BAD_APICID + 1) >> XAPIC_DEST_CPUS_SHIFT)
+#ifndef __ASSEMBLY__
/*
* the local APIC register structure, memory mapped. Not terribly well
* tested, but we might eventually use this one in the future - the
@@ -435,4 +437,5 @@ enum apic_delivery_modes {
APIC_DELIVERY_MODE_EXTINT = 7,
};
+#endif /* !__ASSEMBLY__ */
#endif /* _ASM_X86_APICDEF_H */
--- a/arch/x86/include/asm/smp.h
+++ b/arch/x86/include/asm/smp.h
@@ -195,14 +195,13 @@ extern void nmi_selftest(void);
#endif
extern unsigned int smpboot_control;
+extern unsigned long apic_mmio_base;
#endif /* !__ASSEMBLY__ */
/* Control bits for startup_64 */
-#define STARTUP_APICID_CPUID_1F 0x80000000
-#define STARTUP_APICID_CPUID_0B 0x40000000
-#define STARTUP_APICID_CPUID_01 0x20000000
-#define STARTUP_APICID_SEV_ES 0x10000000
+#define STARTUP_READ_APICID 0x80000000
+#define STARTUP_APICID_SEV_ES 0x40000000
/* Top 8 bits are reserved for control */
#define STARTUP_PARALLEL_MASK 0xFF000000
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -101,6 +101,8 @@ static int apic_extnmi __ro_after_init =
*/
static bool virt_ext_dest_id __ro_after_init;
+unsigned long apic_mmio_base __ro_after_init;
+
/*
* Map cpu index to physical APIC ID
*/
@@ -2164,6 +2166,7 @@ void __init register_lapic_address(unsig
if (!x2apic_mode) {
set_fixmap_nocache(FIX_APIC_BASE, address);
+ apic_mmio_base = APIC_BASE;
apic_printk(APIC_VERBOSE, "mapped APIC to %16lx (%16lx)\n",
APIC_BASE, address);
}
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -24,8 +24,10 @@
#include "../entry/calling.h"
#include <asm/export.h>
#include <asm/nospec-branch.h>
+#include <asm/apicdef.h>
#include <asm/fixmap.h>
#include <asm/smp.h>
+
#include <asm/sev-common.h>
/*
@@ -237,37 +239,25 @@ SYM_INNER_LABEL(secondary_startup_64_no_
#ifdef CONFIG_SMP
/*
- * For parallel boot, the APIC ID is retrieved from CPUID, and then
- * used to look up the CPU number. For booting a single CPU, the
- * CPU number is encoded in smpboot_control.
+ * For parallel boot, the APIC ID is either retrieved the APIC or
+ * from CPUID, and then used to look up the CPU number.
+ * For booting a single CPU, the CPU number is encoded in
+ * smpboot_control.
*
- * Bit 31 STARTUP_APICID_CPUID_1F flag (use CPUID 0x1f)
- * Bit 30 STARTUP_APICID_CPUID_0B flag (use CPUID 0x0b)
- * Bit 29 STARTUP_APICID_CPUID_01 flag (use CPUID 0x01)
- * Bit 28 STARTUP_APICID_SEV_ES flag (CPUID 0x0b via GHCB MSR)
+ * Bit 31 STARTUP_APICID_READ (Read APICID from APIC)
+ * Bit 30 STARTUP_APICID_SEV_ES flag (CPUID 0x0b via GHCB MSR)
* Bit 0-23 CPU# if STARTUP_APICID_CPUID_xx flags are not set
*/
movl smpboot_control(%rip), %ecx
+ testl $STARTUP_READ_APICID, %ecx
+ jnz .Lread_apicid
#ifdef CONFIG_AMD_MEM_ENCRYPT
testl $STARTUP_APICID_SEV_ES, %ecx
jnz .Luse_sev_cpuid_0b
#endif
- testl $STARTUP_APICID_CPUID_1F, %ecx
- jnz .Luse_cpuid_1f
- testl $STARTUP_APICID_CPUID_0B, %ecx
- jnz .Luse_cpuid_0b
- testl $STARTUP_APICID_CPUID_01, %ecx
- jnz .Luse_cpuid_01
andl $(~STARTUP_PARALLEL_MASK), %ecx
jmp .Lsetup_cpu
-.Luse_cpuid_01:
- mov $0x01, %eax
- cpuid
- mov %ebx, %edx
- shr $24, %edx
- jmp .Lsetup_AP
-
#ifdef CONFIG_AMD_MEM_ENCRYPT
.Luse_sev_cpuid_0b:
/* Set the GHCB MSR to request CPUID 0x0B_EDX */
@@ -292,24 +282,30 @@ SYM_INNER_LABEL(secondary_startup_64_no_
jmp .Lsetup_AP
#endif
-.Luse_cpuid_0b:
- mov $0x0B, %eax
- xorl %ecx, %ecx
- cpuid
+.Lread_apicid:
+ mov $MSR_IA32_APICBASE, %ecx
+ rdmsr
+ testl $X2APIC_ENABLE, %eax
+ jnz .Lread_apicid_msr
+
+ /* Read the APIC ID from the fix-mapped MMIO space. */
+ movq apic_mmio_base(%rip), %rcx
+ addq $APIC_ID, %rcx
+ movl (%rcx), %eax
+ shr $24, %eax
jmp .Lsetup_AP
-.Luse_cpuid_1f:
- mov $0x1f, %eax
- xorl %ecx, %ecx
- cpuid
+.Lread_apicid_msr:
+ mov $APIC_X2APIC_ID_MSR, %ecx
+ rdmsr
.Lsetup_AP:
- /* EDX contains the APIC ID of the current CPU */
+ /* EAX contains the APIC ID of the current CPU */
xorq %rcx, %rcx
leaq cpuid_to_apicid(%rip), %rbx
.Lfind_cpunr:
- cmpl (%rbx,%rcx,4), %edx
+ cmpl (%rbx,%rcx,4), %eax
jz .Lsetup_cpu
inc %ecx
#ifdef CONFIG_FORCE_NR_CPUS
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -1253,41 +1253,22 @@ bool __init arch_cpuhp_init_parallel_bri
return false;
}
- /* Encrypted guests require special CPUID handling. */
+ /* Encrypted guests require special handling. */
if (cc_platform_has(CC_ATTR_GUEST_STATE_ENCRYPT)) {
switch (cc_get_vendor()) {
case CC_VENDOR_AMD:
ctrl = STARTUP_APICID_SEV_ES;
if (topology_extended_leaf == 0x0b)
- goto setup;
+ break;
fallthrough;
default:
pr_info("Parallel CPU startup disabled due to guest state encryption\n");
return false;
}
+ } else {
+ ctrl = STARTUP_READ_APICID;
}
- switch (topology_extended_leaf) {
- case 0x0b:
- ctrl = STARTUP_APICID_CPUID_0B;
- break;
- case 0x1f:
- ctrl = STARTUP_APICID_CPUID_1F;
- break;
- case 0x00:
- /* For !x2APIC mode 8 bits from leaf 0x01 are sufficient. */
- if (!x2apic_mode) {
- ctrl = STARTUP_APICID_CPUID_01;
- break;
- }
- fallthrough;
- default:
- pr_info("Parallel CPU startup disabled. Unsupported topology leaf %u\n",
- topology_extended_leaf);
- return false;
- }
-
-setup:
pr_debug("Parallel CPU startup enabled: 0x%08x\n", ctrl);
smpboot_control = ctrl;
return true;
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-20 19:10 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-20 19:10 UTC (permalink / raw)
To: Paul Menzel
Cc: Sean Christopherson, Andrew Cooper, linux-kernel, x86,
David Woodhouse, Brian Gerst, Arjan van de Veen, Paolo Bonzini,
Paul McKenney, Tom Lendacky, Oleksandr Natalenko,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Thu, Apr 20 2023 at 18:47, Paul Menzel wrote:
> Am 20.04.23 um 17:57 schrieb Thomas Gleixner:
> I quickly applied it on top of your branch, but I am getting:
As I said it was untested. I was traveling and did not have access to a
machine to even build it completely. Fixed up and tested version below.
Thanks,
tglx
---
--- a/arch/x86/include/asm/apicdef.h
+++ b/arch/x86/include/asm/apicdef.h
@@ -138,7 +138,8 @@
#define APIC_EILVT_MASKED (1 << 16)
#define APIC_BASE (fix_to_virt(FIX_APIC_BASE))
-#define APIC_BASE_MSR 0x800
+#define APIC_BASE_MSR 0x800
+#define APIC_X2APIC_ID_MSR 0x802
#define XAPIC_ENABLE (1UL << 11)
#define X2APIC_ENABLE (1UL << 10)
@@ -162,6 +163,7 @@
#define APIC_CPUID(apicid) ((apicid) & XAPIC_DEST_CPUS_MASK)
#define NUM_APIC_CLUSTERS ((BAD_APICID + 1) >> XAPIC_DEST_CPUS_SHIFT)
+#ifndef __ASSEMBLY__
/*
* the local APIC register structure, memory mapped. Not terribly well
* tested, but we might eventually use this one in the future - the
@@ -435,4 +437,5 @@ enum apic_delivery_modes {
APIC_DELIVERY_MODE_EXTINT = 7,
};
+#endif /* !__ASSEMBLY__ */
#endif /* _ASM_X86_APICDEF_H */
--- a/arch/x86/include/asm/smp.h
+++ b/arch/x86/include/asm/smp.h
@@ -195,14 +195,13 @@ extern void nmi_selftest(void);
#endif
extern unsigned int smpboot_control;
+extern unsigned long apic_mmio_base;
#endif /* !__ASSEMBLY__ */
/* Control bits for startup_64 */
-#define STARTUP_APICID_CPUID_1F 0x80000000
-#define STARTUP_APICID_CPUID_0B 0x40000000
-#define STARTUP_APICID_CPUID_01 0x20000000
-#define STARTUP_APICID_SEV_ES 0x10000000
+#define STARTUP_READ_APICID 0x80000000
+#define STARTUP_APICID_SEV_ES 0x40000000
/* Top 8 bits are reserved for control */
#define STARTUP_PARALLEL_MASK 0xFF000000
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -101,6 +101,8 @@ static int apic_extnmi __ro_after_init =
*/
static bool virt_ext_dest_id __ro_after_init;
+unsigned long apic_mmio_base __ro_after_init;
+
/*
* Map cpu index to physical APIC ID
*/
@@ -2164,6 +2166,7 @@ void __init register_lapic_address(unsig
if (!x2apic_mode) {
set_fixmap_nocache(FIX_APIC_BASE, address);
+ apic_mmio_base = APIC_BASE;
apic_printk(APIC_VERBOSE, "mapped APIC to %16lx (%16lx)\n",
APIC_BASE, address);
}
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -24,8 +24,10 @@
#include "../entry/calling.h"
#include <asm/export.h>
#include <asm/nospec-branch.h>
+#include <asm/apicdef.h>
#include <asm/fixmap.h>
#include <asm/smp.h>
+
#include <asm/sev-common.h>
/*
@@ -237,37 +239,25 @@ SYM_INNER_LABEL(secondary_startup_64_no_
#ifdef CONFIG_SMP
/*
- * For parallel boot, the APIC ID is retrieved from CPUID, and then
- * used to look up the CPU number. For booting a single CPU, the
- * CPU number is encoded in smpboot_control.
+ * For parallel boot, the APIC ID is either retrieved the APIC or
+ * from CPUID, and then used to look up the CPU number.
+ * For booting a single CPU, the CPU number is encoded in
+ * smpboot_control.
*
- * Bit 31 STARTUP_APICID_CPUID_1F flag (use CPUID 0x1f)
- * Bit 30 STARTUP_APICID_CPUID_0B flag (use CPUID 0x0b)
- * Bit 29 STARTUP_APICID_CPUID_01 flag (use CPUID 0x01)
- * Bit 28 STARTUP_APICID_SEV_ES flag (CPUID 0x0b via GHCB MSR)
+ * Bit 31 STARTUP_APICID_READ (Read APICID from APIC)
+ * Bit 30 STARTUP_APICID_SEV_ES flag (CPUID 0x0b via GHCB MSR)
* Bit 0-23 CPU# if STARTUP_APICID_CPUID_xx flags are not set
*/
movl smpboot_control(%rip), %ecx
+ testl $STARTUP_READ_APICID, %ecx
+ jnz .Lread_apicid
#ifdef CONFIG_AMD_MEM_ENCRYPT
testl $STARTUP_APICID_SEV_ES, %ecx
jnz .Luse_sev_cpuid_0b
#endif
- testl $STARTUP_APICID_CPUID_1F, %ecx
- jnz .Luse_cpuid_1f
- testl $STARTUP_APICID_CPUID_0B, %ecx
- jnz .Luse_cpuid_0b
- testl $STARTUP_APICID_CPUID_01, %ecx
- jnz .Luse_cpuid_01
andl $(~STARTUP_PARALLEL_MASK), %ecx
jmp .Lsetup_cpu
-.Luse_cpuid_01:
- mov $0x01, %eax
- cpuid
- mov %ebx, %edx
- shr $24, %edx
- jmp .Lsetup_AP
-
#ifdef CONFIG_AMD_MEM_ENCRYPT
.Luse_sev_cpuid_0b:
/* Set the GHCB MSR to request CPUID 0x0B_EDX */
@@ -292,24 +282,30 @@ SYM_INNER_LABEL(secondary_startup_64_no_
jmp .Lsetup_AP
#endif
-.Luse_cpuid_0b:
- mov $0x0B, %eax
- xorl %ecx, %ecx
- cpuid
+.Lread_apicid:
+ mov $MSR_IA32_APICBASE, %ecx
+ rdmsr
+ testl $X2APIC_ENABLE, %eax
+ jnz .Lread_apicid_msr
+
+ /* Read the APIC ID from the fix-mapped MMIO space. */
+ movq apic_mmio_base(%rip), %rcx
+ addq $APIC_ID, %rcx
+ movl (%rcx), %eax
+ shr $24, %eax
jmp .Lsetup_AP
-.Luse_cpuid_1f:
- mov $0x1f, %eax
- xorl %ecx, %ecx
- cpuid
+.Lread_apicid_msr:
+ mov $APIC_X2APIC_ID_MSR, %ecx
+ rdmsr
.Lsetup_AP:
- /* EDX contains the APIC ID of the current CPU */
+ /* EAX contains the APIC ID of the current CPU */
xorq %rcx, %rcx
leaq cpuid_to_apicid(%rip), %rbx
.Lfind_cpunr:
- cmpl (%rbx,%rcx,4), %edx
+ cmpl (%rbx,%rcx,4), %eax
jz .Lsetup_cpu
inc %ecx
#ifdef CONFIG_FORCE_NR_CPUS
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -1253,41 +1253,22 @@ bool __init arch_cpuhp_init_parallel_bri
return false;
}
- /* Encrypted guests require special CPUID handling. */
+ /* Encrypted guests require special handling. */
if (cc_platform_has(CC_ATTR_GUEST_STATE_ENCRYPT)) {
switch (cc_get_vendor()) {
case CC_VENDOR_AMD:
ctrl = STARTUP_APICID_SEV_ES;
if (topology_extended_leaf == 0x0b)
- goto setup;
+ break;
fallthrough;
default:
pr_info("Parallel CPU startup disabled due to guest state encryption\n");
return false;
}
+ } else {
+ ctrl = STARTUP_READ_APICID;
}
- switch (topology_extended_leaf) {
- case 0x0b:
- ctrl = STARTUP_APICID_CPUID_0B;
- break;
- case 0x1f:
- ctrl = STARTUP_APICID_CPUID_1F;
- break;
- case 0x00:
- /* For !x2APIC mode 8 bits from leaf 0x01 are sufficient. */
- if (!x2apic_mode) {
- ctrl = STARTUP_APICID_CPUID_01;
- break;
- }
- fallthrough;
- default:
- pr_info("Parallel CPU startup disabled. Unsupported topology leaf %u\n",
- topology_extended_leaf);
- return false;
- }
-
-setup:
pr_debug("Parallel CPU startup enabled: 0x%08x\n", ctrl);
smpboot_control = ctrl;
return true;
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-20 19:10 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-20 19:10 UTC (permalink / raw)
To: Paul Menzel
Cc: Sean Christopherson, Andrew Cooper, linux-kernel, x86,
David Woodhouse, Brian Gerst, Arjan van de Veen, Paolo Bonzini,
Paul McKenney, Tom Lendacky, Oleksandr Natalenko,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Thu, Apr 20 2023 at 18:47, Paul Menzel wrote:
> Am 20.04.23 um 17:57 schrieb Thomas Gleixner:
> I quickly applied it on top of your branch, but I am getting:
As I said it was untested. I was traveling and did not have access to a
machine to even build it completely. Fixed up and tested version below.
Thanks,
tglx
---
--- a/arch/x86/include/asm/apicdef.h
+++ b/arch/x86/include/asm/apicdef.h
@@ -138,7 +138,8 @@
#define APIC_EILVT_MASKED (1 << 16)
#define APIC_BASE (fix_to_virt(FIX_APIC_BASE))
-#define APIC_BASE_MSR 0x800
+#define APIC_BASE_MSR 0x800
+#define APIC_X2APIC_ID_MSR 0x802
#define XAPIC_ENABLE (1UL << 11)
#define X2APIC_ENABLE (1UL << 10)
@@ -162,6 +163,7 @@
#define APIC_CPUID(apicid) ((apicid) & XAPIC_DEST_CPUS_MASK)
#define NUM_APIC_CLUSTERS ((BAD_APICID + 1) >> XAPIC_DEST_CPUS_SHIFT)
+#ifndef __ASSEMBLY__
/*
* the local APIC register structure, memory mapped. Not terribly well
* tested, but we might eventually use this one in the future - the
@@ -435,4 +437,5 @@ enum apic_delivery_modes {
APIC_DELIVERY_MODE_EXTINT = 7,
};
+#endif /* !__ASSEMBLY__ */
#endif /* _ASM_X86_APICDEF_H */
--- a/arch/x86/include/asm/smp.h
+++ b/arch/x86/include/asm/smp.h
@@ -195,14 +195,13 @@ extern void nmi_selftest(void);
#endif
extern unsigned int smpboot_control;
+extern unsigned long apic_mmio_base;
#endif /* !__ASSEMBLY__ */
/* Control bits for startup_64 */
-#define STARTUP_APICID_CPUID_1F 0x80000000
-#define STARTUP_APICID_CPUID_0B 0x40000000
-#define STARTUP_APICID_CPUID_01 0x20000000
-#define STARTUP_APICID_SEV_ES 0x10000000
+#define STARTUP_READ_APICID 0x80000000
+#define STARTUP_APICID_SEV_ES 0x40000000
/* Top 8 bits are reserved for control */
#define STARTUP_PARALLEL_MASK 0xFF000000
--- a/arch/x86/kernel/apic/apic.c
+++ b/arch/x86/kernel/apic/apic.c
@@ -101,6 +101,8 @@ static int apic_extnmi __ro_after_init =
*/
static bool virt_ext_dest_id __ro_after_init;
+unsigned long apic_mmio_base __ro_after_init;
+
/*
* Map cpu index to physical APIC ID
*/
@@ -2164,6 +2166,7 @@ void __init register_lapic_address(unsig
if (!x2apic_mode) {
set_fixmap_nocache(FIX_APIC_BASE, address);
+ apic_mmio_base = APIC_BASE;
apic_printk(APIC_VERBOSE, "mapped APIC to %16lx (%16lx)\n",
APIC_BASE, address);
}
--- a/arch/x86/kernel/head_64.S
+++ b/arch/x86/kernel/head_64.S
@@ -24,8 +24,10 @@
#include "../entry/calling.h"
#include <asm/export.h>
#include <asm/nospec-branch.h>
+#include <asm/apicdef.h>
#include <asm/fixmap.h>
#include <asm/smp.h>
+
#include <asm/sev-common.h>
/*
@@ -237,37 +239,25 @@ SYM_INNER_LABEL(secondary_startup_64_no_
#ifdef CONFIG_SMP
/*
- * For parallel boot, the APIC ID is retrieved from CPUID, and then
- * used to look up the CPU number. For booting a single CPU, the
- * CPU number is encoded in smpboot_control.
+ * For parallel boot, the APIC ID is either retrieved the APIC or
+ * from CPUID, and then used to look up the CPU number.
+ * For booting a single CPU, the CPU number is encoded in
+ * smpboot_control.
*
- * Bit 31 STARTUP_APICID_CPUID_1F flag (use CPUID 0x1f)
- * Bit 30 STARTUP_APICID_CPUID_0B flag (use CPUID 0x0b)
- * Bit 29 STARTUP_APICID_CPUID_01 flag (use CPUID 0x01)
- * Bit 28 STARTUP_APICID_SEV_ES flag (CPUID 0x0b via GHCB MSR)
+ * Bit 31 STARTUP_APICID_READ (Read APICID from APIC)
+ * Bit 30 STARTUP_APICID_SEV_ES flag (CPUID 0x0b via GHCB MSR)
* Bit 0-23 CPU# if STARTUP_APICID_CPUID_xx flags are not set
*/
movl smpboot_control(%rip), %ecx
+ testl $STARTUP_READ_APICID, %ecx
+ jnz .Lread_apicid
#ifdef CONFIG_AMD_MEM_ENCRYPT
testl $STARTUP_APICID_SEV_ES, %ecx
jnz .Luse_sev_cpuid_0b
#endif
- testl $STARTUP_APICID_CPUID_1F, %ecx
- jnz .Luse_cpuid_1f
- testl $STARTUP_APICID_CPUID_0B, %ecx
- jnz .Luse_cpuid_0b
- testl $STARTUP_APICID_CPUID_01, %ecx
- jnz .Luse_cpuid_01
andl $(~STARTUP_PARALLEL_MASK), %ecx
jmp .Lsetup_cpu
-.Luse_cpuid_01:
- mov $0x01, %eax
- cpuid
- mov %ebx, %edx
- shr $24, %edx
- jmp .Lsetup_AP
-
#ifdef CONFIG_AMD_MEM_ENCRYPT
.Luse_sev_cpuid_0b:
/* Set the GHCB MSR to request CPUID 0x0B_EDX */
@@ -292,24 +282,30 @@ SYM_INNER_LABEL(secondary_startup_64_no_
jmp .Lsetup_AP
#endif
-.Luse_cpuid_0b:
- mov $0x0B, %eax
- xorl %ecx, %ecx
- cpuid
+.Lread_apicid:
+ mov $MSR_IA32_APICBASE, %ecx
+ rdmsr
+ testl $X2APIC_ENABLE, %eax
+ jnz .Lread_apicid_msr
+
+ /* Read the APIC ID from the fix-mapped MMIO space. */
+ movq apic_mmio_base(%rip), %rcx
+ addq $APIC_ID, %rcx
+ movl (%rcx), %eax
+ shr $24, %eax
jmp .Lsetup_AP
-.Luse_cpuid_1f:
- mov $0x1f, %eax
- xorl %ecx, %ecx
- cpuid
+.Lread_apicid_msr:
+ mov $APIC_X2APIC_ID_MSR, %ecx
+ rdmsr
.Lsetup_AP:
- /* EDX contains the APIC ID of the current CPU */
+ /* EAX contains the APIC ID of the current CPU */
xorq %rcx, %rcx
leaq cpuid_to_apicid(%rip), %rbx
.Lfind_cpunr:
- cmpl (%rbx,%rcx,4), %edx
+ cmpl (%rbx,%rcx,4), %eax
jz .Lsetup_cpu
inc %ecx
#ifdef CONFIG_FORCE_NR_CPUS
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -1253,41 +1253,22 @@ bool __init arch_cpuhp_init_parallel_bri
return false;
}
- /* Encrypted guests require special CPUID handling. */
+ /* Encrypted guests require special handling. */
if (cc_platform_has(CC_ATTR_GUEST_STATE_ENCRYPT)) {
switch (cc_get_vendor()) {
case CC_VENDOR_AMD:
ctrl = STARTUP_APICID_SEV_ES;
if (topology_extended_leaf == 0x0b)
- goto setup;
+ break;
fallthrough;
default:
pr_info("Parallel CPU startup disabled due to guest state encryption\n");
return false;
}
+ } else {
+ ctrl = STARTUP_READ_APICID;
}
- switch (topology_extended_leaf) {
- case 0x0b:
- ctrl = STARTUP_APICID_CPUID_0B;
- break;
- case 0x1f:
- ctrl = STARTUP_APICID_CPUID_1F;
- break;
- case 0x00:
- /* For !x2APIC mode 8 bits from leaf 0x01 are sufficient. */
- if (!x2apic_mode) {
- ctrl = STARTUP_APICID_CPUID_01;
- break;
- }
- fallthrough;
- default:
- pr_info("Parallel CPU startup disabled. Unsupported topology leaf %u\n",
- topology_extended_leaf);
- return false;
- }
-
-setup:
pr_debug("Parallel CPU startup enabled: 0x%08x\n", ctrl);
smpboot_control = ctrl;
return true;
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
2023-04-20 19:10 ` Thomas Gleixner
(?)
@ 2023-04-21 16:36 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-21 16:36 UTC (permalink / raw)
To: Paul Menzel
Cc: Sean Christopherson, Andrew Cooper, linux-kernel, x86,
David Woodhouse, Brian Gerst, Arjan van de Veen, Paolo Bonzini,
Paul McKenney, Tom Lendacky, Oleksandr Natalenko,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Thu, Apr 20 2023 at 21:10, Thomas Gleixner wrote:
> On Thu, Apr 20 2023 at 18:47, Paul Menzel wrote:
>> Am 20.04.23 um 17:57 schrieb Thomas Gleixner:
>> I quickly applied it on top of your branch, but I am getting:
>
> As I said it was untested. I was traveling and did not have access to a
> machine to even build it completely. Fixed up and tested version below.
I've updated
git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git hotplug
for your conveniance.
Thanks,
tglx
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-21 16:36 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-21 16:36 UTC (permalink / raw)
To: Paul Menzel
Cc: Sean Christopherson, Andrew Cooper, linux-kernel, x86,
David Woodhouse, Brian Gerst, Arjan van de Veen, Paolo Bonzini,
Paul McKenney, Tom Lendacky, Oleksandr Natalenko,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Thu, Apr 20 2023 at 21:10, Thomas Gleixner wrote:
> On Thu, Apr 20 2023 at 18:47, Paul Menzel wrote:
>> Am 20.04.23 um 17:57 schrieb Thomas Gleixner:
>> I quickly applied it on top of your branch, but I am getting:
>
> As I said it was untested. I was traveling and did not have access to a
> machine to even build it completely. Fixed up and tested version below.
I've updated
git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git hotplug
for your conveniance.
Thanks,
tglx
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-21 16:36 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-21 16:36 UTC (permalink / raw)
To: Paul Menzel
Cc: Sean Christopherson, Andrew Cooper, linux-kernel, x86,
David Woodhouse, Brian Gerst, Arjan van de Veen, Paolo Bonzini,
Paul McKenney, Tom Lendacky, Oleksandr Natalenko,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Thu, Apr 20 2023 at 21:10, Thomas Gleixner wrote:
> On Thu, Apr 20 2023 at 18:47, Paul Menzel wrote:
>> Am 20.04.23 um 17:57 schrieb Thomas Gleixner:
>> I quickly applied it on top of your branch, but I am getting:
>
> As I said it was untested. I was traveling and did not have access to a
> machine to even build it completely. Fixed up and tested version below.
I've updated
git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git hotplug
for your conveniance.
Thanks,
tglx
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 35/37] x86/smpboot: Support parallel startup of secondary CPUs
2023-04-15 21:06 ` Thomas Gleixner
(?)
@ 2023-04-24 17:58 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-24 17:58 UTC (permalink / raw)
To: Brian Gerst
Cc: LKML, x86, David Woodhouse, Andrew Cooper, Arjan van de Veen,
Paolo Bonzini, Paul McKenney, Tom Lendacky, Sean Christopherson,
Oleksandr Natalenko, Paul Menzel, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Juergen Gross,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Sat, Apr 15 2023 at 23:06, Thomas Gleixner wrote:
> On Sat, Apr 15 2023 at 09:22, Brian Gerst wrote:
>> On Fri, Apr 14, 2023 at 7:45 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>>> @@ -248,10 +311,20 @@ SYM_INNER_LABEL(secondary_startup_64_no_
>>> *
>>> * RDX contains the per-cpu offset
>>> */
>>> - movq pcpu_hot + X86_current_task(%rdx), %rax
>>> - movq TASK_threadsp(%rax), %rsp
>>> + movq pcpu_hot + X86_top_of_stack(%rdx), %rsp
>>
>> Switching to using pcpu_hot.top_of_stack is ok, but it's not
>> completely equivalent. top_of_stack points to the end of the pt_regs
>> structure, while the kernel stack starts below pt_regs even for kernel
>> threads. So you need to subtract PTREGS_SIZE from the stack pointer
>> after this.
>>
>> This change should also be a separate patch.
>
> You're right on both counts.
Actually no. We can't do that as this breaks suspend/resume (again).
/me drops it.
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 35/37] x86/smpboot: Support parallel startup of secondary CPUs
@ 2023-04-24 17:58 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-24 17:58 UTC (permalink / raw)
To: Brian Gerst
Cc: LKML, x86, David Woodhouse, Andrew Cooper, Arjan van de Veen,
Paolo Bonzini, Paul McKenney, Tom Lendacky, Sean Christopherson,
Oleksandr Natalenko, Paul Menzel, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Juergen Gross,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Sat, Apr 15 2023 at 23:06, Thomas Gleixner wrote:
> On Sat, Apr 15 2023 at 09:22, Brian Gerst wrote:
>> On Fri, Apr 14, 2023 at 7:45 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>>> @@ -248,10 +311,20 @@ SYM_INNER_LABEL(secondary_startup_64_no_
>>> *
>>> * RDX contains the per-cpu offset
>>> */
>>> - movq pcpu_hot + X86_current_task(%rdx), %rax
>>> - movq TASK_threadsp(%rax), %rsp
>>> + movq pcpu_hot + X86_top_of_stack(%rdx), %rsp
>>
>> Switching to using pcpu_hot.top_of_stack is ok, but it's not
>> completely equivalent. top_of_stack points to the end of the pt_regs
>> structure, while the kernel stack starts below pt_regs even for kernel
>> threads. So you need to subtract PTREGS_SIZE from the stack pointer
>> after this.
>>
>> This change should also be a separate patch.
>
> You're right on both counts.
Actually no. We can't do that as this breaks suspend/resume (again).
/me drops it.
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 35/37] x86/smpboot: Support parallel startup of secondary CPUs
@ 2023-04-24 17:58 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-24 17:58 UTC (permalink / raw)
To: Brian Gerst
Cc: LKML, x86, David Woodhouse, Andrew Cooper, Arjan van de Veen,
Paolo Bonzini, Paul McKenney, Tom Lendacky, Sean Christopherson,
Oleksandr Natalenko, Paul Menzel, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Juergen Gross,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Sat, Apr 15 2023 at 23:06, Thomas Gleixner wrote:
> On Sat, Apr 15 2023 at 09:22, Brian Gerst wrote:
>> On Fri, Apr 14, 2023 at 7:45 PM Thomas Gleixner <tglx@linutronix.de> wrote:
>>> @@ -248,10 +311,20 @@ SYM_INNER_LABEL(secondary_startup_64_no_
>>> *
>>> * RDX contains the per-cpu offset
>>> */
>>> - movq pcpu_hot + X86_current_task(%rdx), %rax
>>> - movq TASK_threadsp(%rax), %rsp
>>> + movq pcpu_hot + X86_top_of_stack(%rdx), %rsp
>>
>> Switching to using pcpu_hot.top_of_stack is ok, but it's not
>> completely equivalent. top_of_stack points to the end of the pt_regs
>> structure, while the kernel stack starts below pt_regs even for kernel
>> threads. So you need to subtract PTREGS_SIZE from the stack pointer
>> after this.
>>
>> This change should also be a separate patch.
>
> You're right on both counts.
Actually no. We can't do that as this breaks suspend/resume (again).
/me drops it.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
2023-04-20 19:10 ` Thomas Gleixner
(?)
@ 2023-04-24 18:46 ` Paul Menzel
-1 siblings, 0 replies; 236+ messages in thread
From: Paul Menzel @ 2023-04-24 18:46 UTC (permalink / raw)
To: Thomas Gleixner
Cc: Sean Christopherson, Andrew Cooper, linux-kernel, x86,
David Woodhouse, Brian Gerst, Arjan van de Veen, Paolo Bonzini,
Paul McKenney, Tom Lendacky, Oleksandr Natalenko,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
[-- Attachment #1: Type: text/plain, Size: 722 bytes --]
Dear Thomas,
Am 20.04.23 um 21:10 schrieb Thomas Gleixner:
> On Thu, Apr 20 2023 at 18:47, Paul Menzel wrote:
>> Am 20.04.23 um 17:57 schrieb Thomas Gleixner:
>> I quickly applied it on top of your branch, but I am getting:
>
> As I said it was untested. I was traveling and did not have access to a
> machine to even build it completely. Fixed up and tested version below.
Sorry, if it sounded like a complaint. I just wanted to give a quick
feedback.
[…]
I tested your new version even on Friday, and it worked fine – no ten
seconds delay. Please find the messages attached.
Thank you all for your great work.
Kind regards,
Paul
PS: I am going to try to test your updated branch at the end of the week.
[-- Attachment #2: kodi-linux-6.3-rc3-smp-tglx-with-apic-fix.txt --]
[-- Type: text/plain, Size: 57511 bytes --]
[ 0.000000] Linux version 6.3.0-rc3-00046-g8ba643d7e1c7 (root@bf16f3646a84) (gcc (Debian 11.2.0-12) 11.2.0, GNU ld (GNU Binutils for Debian) 2.40) #452 SMP PREEMPT_DYNAMIC Thu Apr 20 20:15:01 UTC 2023
[ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc3-00046-g8ba643d7e1c7 root=/dev/sda3 rw quiet noisapnp cryptomgr.notests ipv6.disable_ipv6=1 selinux=0
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256
[ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format.
[ 0.000000] signal: max sigframe size: 1776
[ 0.000000] BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
[ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005fe4cfff] usable
[ 0.000000] BIOS-e820: [mem 0x000000005fe4d000-0x000000007fffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000f8000000-0x00000000fbffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000fec10000-0x00000000fec10fff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000017effffff] usable
[ 0.000000] NX (Execute Disable) protection: active
[ 0.000000] SMBIOS 3.0.0 present.
[ 0.000000] DMI: ASUS F2A85-M_PRO/F2A85-M_PRO, BIOS 4.18-9-gb640ed51b2 04/17/2023
[ 0.000000] tsc: Fast TSC calibration using PIT
[ 0.000000] tsc: Detected 3900.440 MHz processor
[ 0.000756] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
[ 0.000759] e820: remove [mem 0x000a0000-0x000fffff] usable
[ 0.000763] last_pfn = 0x17f000 max_arch_pfn = 0x400000000
[ 0.000768] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT
[ 0.000938] last_pfn = 0x5fe4d max_arch_pfn = 0x400000000
[ 0.004000] Using GB pages for direct mapping
[ 0.004000] ACPI: Early table checksum verification disabled
[ 0.004000] ACPI: RSDP 0x00000000000F6830 000024 (v02 COREv4)
[ 0.004000] ACPI: XSDT 0x000000005FE5A0E0 000074 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: FACP 0x000000005FE5BBC0 000114 (v06 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: DSDT 0x000000005FE5A280 00193A (v02 COREv4 COREBOOT 00010001 INTL 20200925)
[ 0.004000] ACPI: FACS 0x000000005FE5A240 000040
[ 0.004000] ACPI: FACS 0x000000005FE5A240 000040
[ 0.004000] ACPI: SSDT 0x000000005FE5BCE0 00008A (v02 COREv4 COREBOOT 0000002A CORE 20200925)
[ 0.004000] ACPI: MCFG 0x000000005FE5BD70 00003C (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: APIC 0x000000005FE5BDB0 000062 (v03 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: HPET 0x000000005FE5BE20 000038 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: HEST 0x000000005FE5BE60 0001D0 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: IVRS 0x000000005FE5C030 000070 (v02 AMD AMDIOMMU 00000001 AMD 00000000)
[ 0.004000] ACPI: SSDT 0x000000005FE5C0A0 00051F (v02 AMD ALIB 00000001 MSFT 04000000)
[ 0.004000] ACPI: SSDT 0x000000005FE5C5C0 0006B2 (v01 AMD POWERNOW 00000001 AMD 00000001)
[ 0.004000] ACPI: VFCT 0x000000005FE5CC80 00F269 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: Reserving FACP table memory at [mem 0x5fe5bbc0-0x5fe5bcd3]
[ 0.004000] ACPI: Reserving DSDT table memory at [mem 0x5fe5a280-0x5fe5bbb9]
[ 0.004000] ACPI: Reserving FACS table memory at [mem 0x5fe5a240-0x5fe5a27f]
[ 0.004000] ACPI: Reserving FACS table memory at [mem 0x5fe5a240-0x5fe5a27f]
[ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe5bce0-0x5fe5bd69]
[ 0.004000] ACPI: Reserving MCFG table memory at [mem 0x5fe5bd70-0x5fe5bdab]
[ 0.004000] ACPI: Reserving APIC table memory at [mem 0x5fe5bdb0-0x5fe5be11]
[ 0.004000] ACPI: Reserving HPET table memory at [mem 0x5fe5be20-0x5fe5be57]
[ 0.004000] ACPI: Reserving HEST table memory at [mem 0x5fe5be60-0x5fe5c02f]
[ 0.004000] ACPI: Reserving IVRS table memory at [mem 0x5fe5c030-0x5fe5c09f]
[ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe5c0a0-0x5fe5c5be]
[ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe5c5c0-0x5fe5cc71]
[ 0.004000] ACPI: Reserving VFCT table memory at [mem 0x5fe5cc80-0x5fe6bee8]
[ 0.004000] No NUMA configuration found
[ 0.004000] Faking a node at [mem 0x0000000000000000-0x000000017effffff]
[ 0.004000] NODE_DATA(0) allocated [mem 0x17efe9000-0x17effffff]
[ 0.004000] Zone ranges:
[ 0.004000] DMA [mem 0x0000000000001000-0x0000000000ffffff]
[ 0.004000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]
[ 0.004000] Normal [mem 0x0000000100000000-0x000000017effffff]
[ 0.004000] Device empty
[ 0.004000] Movable zone start for each node
[ 0.004000] Early memory node ranges
[ 0.004000] node 0: [mem 0x0000000000001000-0x000000000009efff]
[ 0.004000] node 0: [mem 0x0000000000100000-0x000000005fe4cfff]
[ 0.004000] node 0: [mem 0x0000000100000000-0x000000017effffff]
[ 0.004000] Initmem setup node 0 [mem 0x0000000000001000-0x000000017effffff]
[ 0.004000] On node 0, zone DMA: 1 pages in unavailable ranges
[ 0.004000] On node 0, zone DMA: 97 pages in unavailable ranges
[ 0.004000] On node 0, zone Normal: 435 pages in unavailable ranges
[ 0.004000] On node 0, zone Normal: 4096 pages in unavailable ranges
[ 0.004000] ACPI: PM-Timer IO Port: 0x818
[ 0.004000] ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1])
[ 0.004000] IOAPIC[0]: apic_id 4, version 33, address 0xfec00000, GSI 0-23
[ 0.004000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[ 0.004000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 low level)
[ 0.004000] ACPI: Using ACPI (MADT) for SMP configuration information
[ 0.004000] ACPI: HPET id: 0x10228210 base: 0xfed00000
[ 0.004000] smpboot: Allowing 2 CPUs, 0 hotplug CPUs
[ 0.004000] [mem 0x80000000-0xf7ffffff] available for PCI devices
[ 0.004000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns
[ 0.004000] setup_percpu: NR_CPUS:64 nr_cpumask_bits:2 nr_cpu_ids:2 nr_node_ids:1
[ 0.004000] percpu: Embedded 55 pages/cpu s188392 r8192 d28696 u1048576
[ 0.004000] pcpu-alloc: s188392 r8192 d28696 u1048576 alloc=1*2097152
[ 0.004000] pcpu-alloc: [0] 0 1
[ 0.004000] Fallback order for Node 0: 0
[ 0.004000] Built 1 zonelists, mobility grouping on. Total pages: 898451
[ 0.004000] Policy zone: Normal
[ 0.004000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc3-00046-g8ba643d7e1c7 root=/dev/sda3 rw quiet noisapnp cryptomgr.notests ipv6.disable_ipv6=1 selinux=0
[ 0.004000] Unknown kernel command line parameters "noisapnp BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc3-00046-g8ba643d7e1c7", will be passed to user space.
[ 0.004000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[ 0.004000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[ 0.004000] mem auto-init: stack:off, heap alloc:off, heap free:off
[ 0.004000] stackdepot: allocating hash table via alloc_large_system_hash
[ 0.004000] stackdepot hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[ 0.004000] software IO TLB: area num 2.
[ 0.004000] Memory: 3477168K/3651500K available (14336K kernel code, 2340K rwdata, 5308K rodata, 2908K init, 11064K bss, 174072K reserved, 0K cma-reserved)
[ 0.004000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
[ 0.004000] ftrace: allocating 38652 entries in 151 pages
[ 0.004000] ftrace: allocated 151 pages with 5 groups
[ 0.004000] Dynamic Preempt: full
[ 0.004000] rcu: Preemptible hierarchical RCU implementation.
[ 0.004000] rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=2.
[ 0.004000] Trampoline variant of Tasks RCU enabled.
[ 0.004000] Rude variant of Tasks RCU enabled.
[ 0.004000] Tracing variant of Tasks RCU enabled.
[ 0.004000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[ 0.004000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2
[ 0.004000] NR_IRQS: 4352, nr_irqs: 440, preallocated irqs: 16
[ 0.004000] rcu: srcu_init: Setting srcu_struct sizes based on contention.
[ 0.004000] spurious 8259A interrupt: IRQ7.
[ 0.004000] Console: colour VGA+ 80x25
[ 0.004000] printk: console [tty0] enabled
[ 0.004000] ACPI: Core revision 20221020
[ 0.004000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 133484873504 ns
[ 0.004000] APIC: Switch to symmetric I/O mode setup
[ 0.004000] AMD-Vi: Using global IVHD EFR:0x0, EFR2:0x0
[ 0.004000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[ 0.004000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x7071f4ed18f, max_idle_ns: 881590420850 ns
[ 0.145332] Calibrating delay loop (skipped), value calculated using timer frequency.. 7800.88 BogoMIPS (lpj=15601760)
[ 0.145336] pid_max: default: 32768 minimum: 301
[ 0.145430] LSM: initializing lsm=capability
[ 0.145526] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[ 0.145542] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[ 0.145940] Last level iTLB entries: 4KB 512, 2MB 1024, 4MB 512
[ 0.145943] Last level dTLB entries: 4KB 1024, 2MB 1024, 4MB 512, 1GB 0
[ 0.145947] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization
[ 0.145950] Spectre V2 : Mitigation: Retpolines
[ 0.145951] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch
[ 0.145952] Spectre V2 : Spectre v2 / SpectreRSB : Filling RSB on VMEXIT
[ 0.145952] Spectre V2 : Enabling Speculation Barrier for firmware calls
[ 0.145953] RETBleed: Mitigation: untrained return thunk
[ 0.145955] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier
[ 0.145957] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl
[ 0.150469] Freeing SMP alternatives memory: 32K
[ 0.258608] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD Graphics (family: 0x15, model: 0x13, stepping: 0x1)
[ 0.258845] cblist_init_generic: Setting adjustable number of callback queues.
[ 0.258846] cblist_init_generic: Setting shift to 1 and lim to 1.
[ 0.258878] cblist_init_generic: Setting shift to 1 and lim to 1.
[ 0.258905] cblist_init_generic: Setting shift to 1 and lim to 1.
[ 0.258933] Performance Events: Fam15h core perfctr, AMD PMU driver.
[ 0.258955] ... version: 0
[ 0.258956] ... bit width: 48
[ 0.258957] ... generic registers: 6
[ 0.258958] ... value mask: 0000ffffffffffff
[ 0.258959] ... max period: 00007fffffffffff
[ 0.258960] ... fixed-purpose events: 0
[ 0.258961] ... event mask: 000000000000003f
[ 0.259083] rcu: Hierarchical SRCU implementation.
[ 0.259084] rcu: Max phase no-delay instances is 1000.
[ 0.259676] NMI watchdog: Enabled. Permanently consumes one hw-PMU counter.
[ 0.259750] smp: Bringing up secondary CPUs ...
[ 0.259952] x86: Booting SMP configuration:
[ 0.259953] .... node #0, CPUs: #1
[ 0.259958] smpboot: Kicking AP alive: 17
[ 0.260088] smp: Brought up 1 node, 2 CPUs
[ 0.260088] smpboot: Max logical packages: 1
[ 0.260088] smpboot: Total of 2 processors activated (15601.76 BogoMIPS)
[ 0.261513] devtmpfs: initialized
[ 0.261513] x86/mm: Memory block size: 128MB
[ 0.262387] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[ 0.262387] futex hash table entries: 512 (order: 3, 32768 bytes, linear)
[ 0.262387] pinctrl core: initialized pinctrl subsystem
[ 0.262387] PM: RTC time: 20:56:07, date: 2023-04-21
[ 0.262387] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[ 0.262600] audit: initializing netlink subsys (disabled)
[ 0.262617] audit: type=2000 audit(1682110567.140:1): state=initialized audit_enabled=0 res=1
[ 0.262617] thermal_sys: Registered thermal governor 'fair_share'
[ 0.262617] thermal_sys: Registered thermal governor 'bang_bang'
[ 0.262617] thermal_sys: Registered thermal governor 'step_wise'
[ 0.262617] thermal_sys: Registered thermal governor 'user_space'
[ 0.262617] cpuidle: using governor ladder
[ 0.262617] cpuidle: using governor menu
[ 0.262617] PCI: MMCONFIG for domain 0000 [bus 00-3f] at [mem 0xf8000000-0xfbffffff] (base 0xf8000000)
[ 0.262617] PCI: MMCONFIG at [mem 0xf8000000-0xfbffffff] reserved as E820 entry
[ 0.262617] PCI: Using configuration type 1 for base access
[ 0.262617] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.
[ 0.273356] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages
[ 0.273356] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page
[ 0.273356] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages
[ 0.273356] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page
[ 0.280509] cryptd: max_cpu_qlen set to 1000
[ 0.280509] ACPI: Added _OSI(Module Device)
[ 0.280509] ACPI: Added _OSI(Processor Device)
[ 0.280509] ACPI: Added _OSI(3.0 _SCP Extensions)
[ 0.280509] ACPI: Added _OSI(Processor Aggregator Device)
[ 0.287039] ACPI: 4 ACPI AML tables successfully acquired and loaded
[ 0.288072] ACPI: Interpreter enabled
[ 0.288072] ACPI: PM: (supports S0 S1 S3 S5)
[ 0.288072] ACPI: Using IOAPIC for interrupt routing
[ 0.288072] HEST: Table parsing has been initialized.
[ 0.288072] GHES: Failed to enable APEI firmware first mode.
[ 0.288072] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[ 0.288072] PCI: Ignoring E820 reservations for host bridge windows
[ 0.288072] ACPI: Enabled 8 GPEs in block 00 to 1F
[ 0.290922] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[ 0.290932] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]
[ 0.291018] acpi PNP0A03:00: _OSC: OS now controls [PME AER PCIeCapability LTR]
[ 0.291034] acpi PNP0A03:00: [Firmware Info]: MMCONFIG for domain 0000 [bus 00-3f] only partially covers this bridge
[ 0.291117] acpi PNP0A03:00: host bridge window expanded to [io 0x0000-0x0cf7 window]; [io 0x03b0-0x03df window] ignored
[ 0.291369] PCI host bridge to bus 0000:00
[ 0.291371] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]
[ 0.291374] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]
[ 0.291376] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000dffff]
[ 0.291379] pci_bus 0000:00: root bus resource [mem 0x80000000-0xffffffff]
[ 0.291381] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 0.291406] pci 0000:00:00.0: [1022:1410] type 00 class 0x060000
[ 0.291559] pci 0000:00:00.2: [1022:1419] type 00 class 0x080600
[ 0.291648] pci 0000:00:01.0: [1002:9996] type 00 class 0x030000
[ 0.291656] pci 0000:00:01.0: reg 0x10: [mem 0xe0000000-0xefffffff pref]
[ 0.291661] pci 0000:00:01.0: reg 0x14: [io 0x1000-0x10ff]
[ 0.291665] pci 0000:00:01.0: reg 0x18: [mem 0xf0180000-0xf01bffff]
[ 0.291681] pci 0000:00:01.0: enabling Extended Tags
[ 0.291694] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]
[ 0.291711] pci 0000:00:01.0: supports D1 D2
[ 0.291776] pci 0000:00:01.1: [1002:9902] type 00 class 0x040300
[ 0.291784] pci 0000:00:01.1: reg 0x10: [mem 0xf01c0000-0xf01c3fff]
[ 0.291805] pci 0000:00:01.1: enabling Extended Tags
[ 0.291828] pci 0000:00:01.1: supports D1 D2
[ 0.291918] pci 0000:00:11.0: [1022:7801] type 00 class 0x010601
[ 0.291931] pci 0000:00:11.0: reg 0x10: [io 0x1410-0x1417]
[ 0.291938] pci 0000:00:11.0: reg 0x14: [io 0x1420-0x1423]
[ 0.291946] pci 0000:00:11.0: reg 0x18: [io 0x1418-0x141f]
[ 0.291953] pci 0000:00:11.0: reg 0x1c: [io 0x1424-0x1427]
[ 0.291960] pci 0000:00:11.0: reg 0x20: [io 0x1400-0x140f]
[ 0.291968] pci 0000:00:11.0: reg 0x24: [mem 0xf01cc000-0xf01cc7ff]
[ 0.292125] pci 0000:00:12.0: [1022:7807] type 00 class 0x0c0310
[ 0.292139] pci 0000:00:12.0: reg 0x10: [mem 0xf01c8000-0xf01c8fff]
[ 0.292322] pci 0000:00:12.2: [1022:7808] type 00 class 0x0c0320
[ 0.292335] pci 0000:00:12.2: reg 0x10: [mem 0xf01cd000-0xf01cd0ff]
[ 0.292400] pci 0000:00:12.2: supports D1 D2
[ 0.292402] pci 0000:00:12.2: PME# supported from D0 D1 D2 D3hot
[ 0.292542] pci 0000:00:13.0: [1022:7807] type 00 class 0x0c0310
[ 0.292555] pci 0000:00:13.0: reg 0x10: [mem 0xf01c9000-0xf01c9fff]
[ 0.292737] pci 0000:00:13.2: [1022:7808] type 00 class 0x0c0320
[ 0.292750] pci 0000:00:13.2: reg 0x10: [mem 0xf01ce000-0xf01ce0ff]
[ 0.292815] pci 0000:00:13.2: supports D1 D2
[ 0.292816] pci 0000:00:13.2: PME# supported from D0 D1 D2 D3hot
[ 0.292955] pci 0000:00:14.0: [1022:780b] type 00 class 0x0c0500
[ 0.293139] pci 0000:00:14.2: [1022:780d] type 00 class 0x040300
[ 0.293156] pci 0000:00:14.2: reg 0x10: [mem 0xf01c4000-0xf01c7fff 64bit]
[ 0.293210] pci 0000:00:14.2: PME# supported from D0 D3hot D3cold
[ 0.293355] pci 0000:00:14.3: [1022:780e] type 00 class 0x060100
[ 0.293541] pci 0000:00:14.4: [1022:780f] type 01 class 0x060401
[ 0.293703] pci 0000:00:14.5: [1022:7809] type 00 class 0x0c0310
[ 0.293717] pci 0000:00:14.5: reg 0x10: [mem 0xf01ca000-0xf01cafff]
[ 0.293906] pci 0000:00:15.0: [1022:43a0] type 01 class 0x060400
[ 0.293935] pci 0000:00:15.0: enabling Extended Tags
[ 0.293975] pci 0000:00:15.0: supports D1 D2
[ 0.294151] pci 0000:00:15.1: [1022:43a1] type 01 class 0x060400
[ 0.294182] pci 0000:00:15.1: enabling Extended Tags
[ 0.294221] pci 0000:00:15.1: supports D1 D2
[ 0.294398] pci 0000:00:15.2: [1022:43a2] type 01 class 0x060400
[ 0.294426] pci 0000:00:15.2: enabling Extended Tags
[ 0.294468] pci 0000:00:15.2: supports D1 D2
[ 0.294547] pci 0000:00:16.0: [1022:7807] type 00 class 0x0c0310
[ 0.294560] pci 0000:00:16.0: reg 0x10: [mem 0xf01cb000-0xf01cbfff]
[ 0.294737] pci 0000:00:16.2: [1022:7808] type 00 class 0x0c0320
[ 0.294751] pci 0000:00:16.2: reg 0x10: [mem 0xf01cf000-0xf01cf0ff]
[ 0.294815] pci 0000:00:16.2: supports D1 D2
[ 0.294817] pci 0000:00:16.2: PME# supported from D0 D1 D2 D3hot
[ 0.294958] pci 0000:00:18.0: [1022:1400] type 00 class 0x060000
[ 0.295024] pci 0000:00:18.1: [1022:1401] type 00 class 0x060000
[ 0.295086] pci 0000:00:18.2: [1022:1402] type 00 class 0x060000
[ 0.295152] pci 0000:00:18.3: [1022:1403] type 00 class 0x060000
[ 0.295289] pci 0000:00:18.4: [1022:1404] type 00 class 0x060000
[ 0.295353] pci 0000:00:18.5: [1022:1405] type 00 class 0x060000
[ 0.295436] pci_bus 0000:01: extended config space not accessible
[ 0.295499] pci 0000:00:14.4: PCI bridge to [bus 01] (subtractive decode)
[ 0.295510] pci 0000:00:14.4: bridge window [io 0x0000-0x0cf7 window] (subtractive decode)
[ 0.295513] pci 0000:00:14.4: bridge window [io 0x0d00-0xffff window] (subtractive decode)
[ 0.295515] pci 0000:00:14.4: bridge window [mem 0x000a0000-0x000dffff] (subtractive decode)
[ 0.295517] pci 0000:00:14.4: bridge window [mem 0x80000000-0xffffffff] (subtractive decode)
[ 0.295571] pci 0000:00:15.0: PCI bridge to [bus 02]
[ 0.295653] pci 0000:03:00.0: [1b21:1042] type 00 class 0x0c0330
[ 0.295689] pci 0000:03:00.0: reg 0x10: [mem 0xf0000000-0xf0007fff 64bit]
[ 0.295863] pci 0000:03:00.0: PME# supported from D3hot D3cold
[ 0.295907] pci 0000:03:00.0: 2.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s PCIe x1 link at 0000:00:15.1 (capable of 4.000 Gb/s with 5.0 GT/s PCIe x1 link)
[ 0.309393] pci 0000:00:15.1: PCI bridge to [bus 03]
[ 0.309405] pci 0000:00:15.1: bridge window [mem 0xf0000000-0xf00fffff]
[ 0.309414] pci 0000:00:15.2: bridge configuration invalid ([bus 00-00]), reconfiguring
[ 0.309545] pci 0000:04:00.0: [10ec:8168] type 00 class 0x020000
[ 0.309563] pci 0000:04:00.0: reg 0x10: [io 0x0000-0x00ff]
[ 0.309585] pci 0000:04:00.0: reg 0x18: [mem 0x00000000-0x00000fff 64bit pref]
[ 0.309598] pci 0000:04:00.0: reg 0x20: [mem 0x00000000-0x00003fff 64bit pref]
[ 0.309705] pci 0000:04:00.0: supports D1 D2
[ 0.309707] pci 0000:04:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[ 0.325403] pci 0000:00:15.2: PCI bridge to [bus 04-ff]
[ 0.325414] pci 0000:00:15.2: bridge window [io 0x0000-0x0fff]
[ 0.325417] pci 0000:00:15.2: bridge window [mem 0x00000000-0x000fffff]
[ 0.325422] pci 0000:00:15.2: bridge window [mem 0x00000000-0x000fffff 64bit pref]
[ 0.325425] pci_bus 0000:04: busn_res: [bus 04-ff] end is updated to 04
[ 0.325919] ACPI: PCI: Interrupt link INTA configured for IRQ 0
[ 0.326011] ACPI: PCI: Interrupt link INTB configured for IRQ 0
[ 0.326102] ACPI: PCI: Interrupt link INTC configured for IRQ 0
[ 0.326193] ACPI: PCI: Interrupt link INTD configured for IRQ 0
[ 0.326284] ACPI: PCI: Interrupt link INTE configured for IRQ 0
[ 0.326375] ACPI: PCI: Interrupt link INTF configured for IRQ 0
[ 0.326465] ACPI: PCI: Interrupt link INTG configured for IRQ 0
[ 0.326556] ACPI: PCI: Interrupt link INTH configured for IRQ 0
[ 0.326777] iommu: Default domain type: Translated
[ 0.326778] iommu: DMA domain TLB invalidation policy: lazy mode
[ 0.326974] SCSI subsystem initialized
[ 0.329389] libata version 3.00 loaded.
[ 0.329395] ACPI: bus type USB registered
[ 0.329418] usbcore: registered new interface driver usbfs
[ 0.329428] usbcore: registered new interface driver hub
[ 0.329437] usbcore: registered new device driver usb
[ 0.329564] PCI: Using ACPI for IRQ routing
[ 0.331132] PCI: pci_cache_line_size set to 64 bytes
[ 0.331184] e820: reserve RAM buffer [mem 0x0009fc00-0x0009ffff]
[ 0.331187] e820: reserve RAM buffer [mem 0x5fe4d000-0x5fffffff]
[ 0.331189] e820: reserve RAM buffer [mem 0x17f000000-0x17fffffff]
[ 0.331235] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0
[ 0.331240] hpet0: 3 comparators, 32-bit 14.318180 MHz counter
[ 0.334410] clocksource: Switched to clocksource tsc-early
[ 0.334656] VFS: Disk quotas dquot_6.6.0
[ 0.334682] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[ 0.334803] pnp: PnP ACPI init
[ 0.335147] system 00:00: [mem 0xfec10002-0xfec11001] could not be reserved
[ 0.335525] pnp: PnP ACPI: found 2 devices
[ 0.342072] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
[ 0.342260] NET: Registered PF_INET protocol family
[ 0.342434] IP idents hash table entries: 65536 (order: 7, 524288 bytes, linear)
[ 0.344041] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear)
[ 0.344058] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
[ 0.344070] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear)
[ 0.344141] TCP bind hash table entries: 32768 (order: 8, 1048576 bytes, linear)
[ 0.344505] TCP: Hash tables configured (established 32768 bind 32768)
[ 0.344576] UDP hash table entries: 2048 (order: 4, 65536 bytes, linear)
[ 0.344601] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear)
[ 0.344722] NET: Registered PF_UNIX/PF_LOCAL protocol family
[ 0.344757] pci 0000:00:15.2: BAR 15: assigned [mem 0x80000000-0x800fffff 64bit pref]
[ 0.344763] pci 0000:00:15.2: BAR 13: assigned [io 0x2000-0x2fff]
[ 0.344769] pci 0000:00:14.4: PCI bridge to [bus 01]
[ 0.344781] pci 0000:00:15.0: PCI bridge to [bus 02]
[ 0.344790] pci 0000:00:15.1: PCI bridge to [bus 03]
[ 0.344793] pci 0000:00:15.1: bridge window [mem 0xf0000000-0xf00fffff]
[ 0.344807] pci 0000:04:00.0: BAR 4: assigned [mem 0x80000000-0x80003fff 64bit pref]
[ 0.344821] pci 0000:04:00.0: BAR 2: assigned [mem 0x80004000-0x80004fff 64bit pref]
[ 0.344833] pci 0000:04:00.0: BAR 0: assigned [io 0x2000-0x20ff]
[ 0.344840] pci 0000:00:15.2: PCI bridge to [bus 04]
[ 0.344842] pci 0000:00:15.2: bridge window [io 0x2000-0x2fff]
[ 0.344847] pci 0000:00:15.2: bridge window [mem 0x80000000-0x800fffff 64bit pref]
[ 0.344854] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]
[ 0.344857] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]
[ 0.344860] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000dffff]
[ 0.344862] pci_bus 0000:00: resource 7 [mem 0x80000000-0xffffffff]
[ 0.344864] pci_bus 0000:01: resource 4 [io 0x0000-0x0cf7 window]
[ 0.344867] pci_bus 0000:01: resource 5 [io 0x0d00-0xffff window]
[ 0.344869] pci_bus 0000:01: resource 6 [mem 0x000a0000-0x000dffff]
[ 0.344871] pci_bus 0000:01: resource 7 [mem 0x80000000-0xffffffff]
[ 0.344873] pci_bus 0000:03: resource 1 [mem 0xf0000000-0xf00fffff]
[ 0.344875] pci_bus 0000:04: resource 0 [io 0x2000-0x2fff]
[ 0.344877] pci_bus 0000:04: resource 2 [mem 0x80000000-0x800fffff 64bit pref]
[ 0.345003] pci 0000:00:01.1: D0 power state depends on 0000:00:01.0
[ 0.345887] pci 0000:00:12.2: PME# does not work under D3, disabling it
[ 0.346671] pci 0000:00:13.2: PME# does not work under D3, disabling it
[ 0.347980] pci 0000:00:16.2: PME# does not work under D3, disabling it
[ 0.348353] PCI: CLS 64 bytes, default 64
[ 0.348449] pci 0000:00:00.2: AMD-Vi: Applying erratum 746 workaround
[ 0.348560] pci 0000:00:01.0: Adding to iommu group 0
[ 0.348584] pci 0000:00:01.1: Adding to iommu group 0
[ 0.348616] pci 0000:00:11.0: Adding to iommu group 1
[ 0.348659] pci 0000:00:12.0: Adding to iommu group 2
[ 0.348681] pci 0000:00:12.2: Adding to iommu group 2
[ 0.348723] pci 0000:00:13.0: Adding to iommu group 3
[ 0.348743] pci 0000:00:13.2: Adding to iommu group 3
[ 0.348792] pci 0000:00:14.0: Adding to iommu group 4
[ 0.348817] pci 0000:00:14.2: Adding to iommu group 4
[ 0.348840] pci 0000:00:14.3: Adding to iommu group 4
[ 0.348879] pci 0000:00:14.4: Adding to iommu group 5
[ 0.348903] pci 0000:00:14.5: Adding to iommu group 6
[ 0.348946] pci 0000:00:15.0: Adding to iommu group 7
[ 0.348974] pci 0000:00:15.1: Adding to iommu group 7
[ 0.349000] pci 0000:00:15.2: Adding to iommu group 7
[ 0.349040] pci 0000:00:16.0: Adding to iommu group 8
[ 0.349063] pci 0000:00:16.2: Adding to iommu group 8
[ 0.349128] pci 0000:00:18.0: Adding to iommu group 9
[ 0.349150] pci 0000:00:18.1: Adding to iommu group 9
[ 0.349188] pci 0000:00:18.2: Adding to iommu group 9
[ 0.349210] pci 0000:00:18.3: Adding to iommu group 9
[ 0.349234] pci 0000:00:18.4: Adding to iommu group 9
[ 0.349255] pci 0000:00:18.5: Adding to iommu group 9
[ 0.349269] pci 0000:03:00.0: Adding to iommu group 7
[ 0.349279] pci 0000:04:00.0: Adding to iommu group 7
[ 0.352005] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[ 0.352011] AMD-Vi: Extended features (0x800000853, 0x0): PreF PPR GT IA
[ 0.352020] AMD-Vi: Interrupt remapping enabled
[ 0.352238] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[ 0.352240] software IO TLB: mapped [mem 0x000000005be4d000-0x000000005fe4d000] (64MB)
[ 0.352297] LVT offset 0 assigned for vector 0x400
[ 0.352343] perf: AMD IBS detected (0x000000ff)
[ 0.352351] amd_uncore: 4 amd_nb counters detected
[ 0.356598] workingset: timestamp_bits=37 max_order=20 bucket_order=0
[ 0.356626] zbud: loaded
[ 0.357112] NET: Registered PF_ALG protocol family
[ 0.357118] Key type asymmetric registered
[ 0.357119] Asymmetric key parser 'x509' registered
[ 0.357471] alg: self-tests disabled
[ 0.357567] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 251)
[ 0.357610] io scheduler mq-deadline registered
[ 0.357612] io scheduler kyber registered
[ 0.358782] pcieport 0000:00:15.0: PME: Signaling with IRQ 25
[ 0.358948] pcieport 0000:00:15.1: PME: Signaling with IRQ 26
[ 0.359013] pcieport 0000:00:15.2: enabling device (0000 -> 0003)
[ 0.359228] pcieport 0000:00:15.2: PME: Signaling with IRQ 27
[ 0.359503] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
[ 0.359567] ACPI: button: Power Button [PWRF]
[ 0.359621] ACPI: \_SB_.P000: Found 2 idle states
[ 0.359743] ACPI: \_SB_.P001: Found 2 idle states
[ 0.360670] thermal LNXTHERM:00: registered as thermal_zone0
[ 0.360673] ACPI: thermal: Thermal Zone [TZ00] (15 C)
[ 0.360984] Non-volatile memory driver v1.3
[ 0.361051] AMD-Vi: AMD IOMMUv2 loaded and initialized
[ 0.361079] ACPI: bus type drm_connector registered
[ 0.361292] ahci 0000:00:11.0: version 3.0
[ 0.361580] ahci 0000:00:11.0: AHCI 0001.0300 32 slots 8 ports 6 Gbps 0x40 impl SATA mode
[ 0.361584] ahci 0000:00:11.0: flags: 64bit ncq sntf ilck led clo pio
[ 0.362741] scsi host0: ahci
[ 0.362967] scsi host1: ahci
[ 0.363157] scsi host2: ahci
[ 0.363372] scsi host3: ahci
[ 0.363571] scsi host4: ahci
[ 0.363773] scsi host5: ahci
[ 0.363984] scsi host6: ahci
[ 0.364182] scsi host7: ahci
[ 0.364275] ata1: DUMMY
[ 0.364277] ata2: DUMMY
[ 0.364278] ata3: DUMMY
[ 0.364278] ata4: DUMMY
[ 0.364279] ata5: DUMMY
[ 0.364280] ata6: DUMMY
[ 0.364282] ata7: SATA max UDMA/133 abar m2048@0xf01cc000 port 0xf01cc400 irq 19
[ 0.364284] ata8: DUMMY
[ 0.364564] i8042: PNP: No PS/2 controller found.
[ 0.364565] i8042: Probing ports directly.
[ 0.366981] serio: i8042 KBD port at 0x60,0x64 irq 1
[ 0.366988] serio: i8042 AUX port at 0x60,0x64 irq 12
[ 0.367123] mousedev: PS/2 mouse device common for all mice
[ 0.367188] rtc_cmos 00:01: RTC can wake from S4
[ 0.367487] rtc_cmos 00:01: registered as rtc0
[ 0.367510] rtc_cmos 00:01: setting system clock to 2023-04-21T20:56:07 UTC (1682110567)
[ 0.367553] rtc_cmos 00:01: alarms up to one day, y3k, 114 bytes nvram, hpet irqs
[ 0.367590] device-mapper: uevent: version 1.0.3
[ 0.367665] device-mapper: ioctl: 4.47.0-ioctl (2022-07-28) initialised: dm-devel@redhat.com
[ 0.367827] hid: raw HID events driver (C) Jiri Kosina
[ 0.367913] usbcore: registered new interface driver usbhid
[ 0.367914] usbhid: USB HID core driver
[ 0.368010] Initializing XFRM netlink socket
[ 0.368020] NET: Registered PF_PACKET protocol family
[ 0.368021] x86/pm: family 0x15 cpu detected, MSR saving is needed during suspending.
[ 0.368340] microcode: CPU1: patch_level=0x0600111f
[ 0.368342] microcode: CPU0: patch_level=0x0600111f
[ 0.368353] microcode: Microcode Update Driver: v2.2.
[ 0.368358] IPI shorthand broadcast: enabled
[ 0.368370] AVX version of gcm_enc/dec engaged.
[ 0.368401] AES CTR mode by8 optimization enabled
[ 0.372495] sched_clock: Marking stable (254511749, 117328013)->(374590709, -2750947)
[ 0.372745] registered taskstats version 1
[ 0.372987] zswap: loaded using pool lzo/zbud
[ 0.377461] kmemleak: Kernel memory leak detector initialized (mem pool available: 15679)
[ 0.377464] kmemleak: Automatic memory scanning thread started
[ 0.377465] debug_vm_pgtable: [debug_vm_pgtable ]: Validating architecture page table helpers
[ 0.382250] Key type encrypted registered
[ 0.385395] PM: Magic number: 3:579:953
[ 0.385449] memory memory6: hash matches
[ 0.483426] ata7: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[ 0.483587] ata7.00: ATA-9: SanDisk SDSSDP064G, 2.0.0, max UDMA/133
[ 0.483590] ata7.00: 125045424 sectors, multi 1: LBA48 NCQ (depth 32)
[ 0.483792] ata7.00: configured for UDMA/133
[ 0.484004] scsi 6:0:0:0: Direct-Access ATA SanDisk SDSSDP06 0 PQ: 0 ANSI: 5
[ 0.485006] sd 6:0:0:0: [sda] 125045424 512-byte logical blocks: (64.0 GB/59.6 GiB)
[ 0.485028] sd 6:0:0:0: [sda] Write Protect is off
[ 0.485033] sd 6:0:0:0: [sda] Mode Sense: 00 3a 00 00
[ 0.485062] sd 6:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 0.485103] sd 6:0:0:0: [sda] Preferred minimum I/O size 512 bytes
[ 0.486554] sda: sda1 sda2 sda3
[ 0.487033] sd 6:0:0:0: [sda] Attached SCSI disk
[ 0.508543] EXT4-fs (sda3): mounted filesystem fe29e0dc-6303-4401-987c-8472bc1b9516 with ordered data mode. Quota mode: none.
[ 0.508591] VFS: Mounted root (ext4 filesystem) on device 8:3.
[ 0.510618] devtmpfs: mounted
[ 0.514506] Freeing unused kernel image (initmem) memory: 2908K
[ 0.532141] Write protecting the kernel read-only data: 20480k
[ 0.532425] Freeing unused kernel image (rodata/data gap) memory: 836K
[ 0.569736] x86/mm: Checked W+X mappings: passed, no W+X pages found.
[ 0.569742] rodata_test: all tests were successful
[ 0.569773] Run /sbin/init as init process
[ 0.569775] with arguments:
[ 0.569776] /sbin/init
[ 0.569777] noisapnp
[ 0.569778] with environment:
[ 0.569779] HOME=/
[ 0.569780] TERM=linux
[ 0.569781] BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc3-00046-g8ba643d7e1c7
[ 0.751269] systemd[1]: Inserted module 'autofs4'
[ 0.777767] NET: Registered PF_INET6 protocol family
[ 0.778674] Segment Routing with IPv6
[ 0.778702] In-situ OAM (IOAM) with IPv6
[ 0.804133] systemd[1]: systemd 252.6-1 running in system mode (+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
[ 0.804143] systemd[1]: Detected architecture x86-64.
[ 0.808903] systemd[1]: Hostname set to <kodi>.
[ 1.081619] systemd[1]: Queued start job for default target graphical.target.
[ 1.102404] systemd[1]: Created slice system-getty.slice - Slice /system/getty.
[ 1.103505] systemd[1]: Created slice system-modprobe.slice - Slice /system/modprobe.
[ 1.104345] systemd[1]: Created slice user.slice - User and Session Slice.
[ 1.104531] systemd[1]: Started systemd-ask-password-console.path - Dispatch Password Requests to Console Directory Watch.
[ 1.104655] systemd[1]: Started systemd-ask-password-wall.path - Forward Password Requests to Wall Directory Watch.
[ 1.105082] systemd[1]: Set up automount proc-sys-fs-binfmt_misc.automount - Arbitrary Executable File Formats File System Automount Point.
[ 1.105122] systemd[1]: Reached target cryptsetup.target - Local Encrypted Volumes.
[ 1.105171] systemd[1]: Reached target integritysetup.target - Local Integrity Protected Volumes.
[ 1.105213] systemd[1]: Reached target paths.target - Path Units.
[ 1.105248] systemd[1]: Reached target remote-fs.target - Remote File Systems.
[ 1.105276] systemd[1]: Reached target slices.target - Slice Units.
[ 1.105302] systemd[1]: Reached target swap.target - Swaps.
[ 1.105344] systemd[1]: Reached target veritysetup.target - Local Verity Protected Volumes.
[ 1.107911] systemd[1]: Listening on systemd-coredump.socket - Process Core Dump Socket.
[ 1.108154] systemd[1]: Listening on systemd-fsckd.socket - fsck to fsckd communication Socket.
[ 1.108318] systemd[1]: Listening on systemd-initctl.socket - initctl Compatibility Named Pipe.
[ 1.108624] systemd[1]: Listening on systemd-journald-audit.socket - Journal Audit Socket.
[ 1.108886] systemd[1]: Listening on systemd-journald-dev-log.socket - Journal Socket (/dev/log).
[ 1.109157] systemd[1]: Listening on systemd-journald.socket - Journal Socket.
[ 1.109414] systemd[1]: Listening on systemd-networkd.socket - Network Service Netlink Socket.
[ 1.110215] systemd[1]: Listening on systemd-udevd-control.socket - udev Control Socket.
[ 1.110475] systemd[1]: Listening on systemd-udevd-kernel.socket - udev Kernel Socket.
[ 1.113168] systemd[1]: Mounting dev-hugepages.mount - Huge Pages File System...
[ 1.116585] systemd[1]: Mounting dev-mqueue.mount - POSIX Message Queue File System...
[ 1.121085] systemd[1]: Mounting sys-kernel-debug.mount - Kernel Debug File System...
[ 1.124574] systemd[1]: Mounting sys-kernel-tracing.mount - Kernel Trace File System...
[ 1.128227] systemd[1]: Starting kmod-static-nodes.service - Create List of Static Device Nodes...
[ 1.132312] systemd[1]: Starting modprobe@configfs.service - Load Kernel Module configfs...
[ 1.135610] systemd[1]: Starting modprobe@dm_mod.service - Load Kernel Module dm_mod...
[ 1.138876] systemd[1]: Starting modprobe@drm.service - Load Kernel Module drm...
[ 1.147425] systemd[1]: Starting modprobe@efi_pstore.service - Load Kernel Module efi_pstore...
[ 1.150601] systemd[1]: Starting modprobe@fuse.service - Load Kernel Module fuse...
[ 1.153921] systemd[1]: Starting modprobe@loop.service - Load Kernel Module loop...
[ 1.154024] systemd[1]: systemd-firstboot.service - First Boot Wizard was skipped because of an unmet condition check (ConditionFirstBoot=yes).
[ 1.154112] systemd[1]: systemd-fsck-root.service - File System Check on Root Device was skipped because of an unmet condition check (ConditionPathIsReadWrite=!/).
[ 1.154158] systemd[1]: Reached target local-fs.target - Local File Systems.
[ 1.154259] systemd[1]: apparmor.service - Load AppArmor profiles was skipped because of an unmet condition check (ConditionSecurity=apparmor).
[ 1.167112] fuse: init (API version 7.38)
[ 1.167255] loop: module loaded
[ 1.168586] systemd[1]: Starting systemd-binfmt.service - Set Up Additional Binary Formats...
[ 1.174035] systemd[1]: Starting systemd-journald.service - Journal Service...
[ 1.177353] systemd[1]: Starting systemd-random-seed.service - Load/Save Random Seed...
[ 1.180848] systemd[1]: Starting systemd-sysctl.service - Apply Kernel Variables...
[ 1.195025] systemd[1]: Starting systemd-sysusers.service - Create System Users...
[ 1.201956] systemd[1]: Starting systemd-udev-trigger.service - Coldplug All udev Devices...
[ 1.217695] systemd[1]: Mounted dev-hugepages.mount - Huge Pages File System.
[ 1.218409] systemd[1]: Mounted dev-mqueue.mount - POSIX Message Queue File System.
[ 1.218683] systemd[1]: Mounted sys-kernel-debug.mount - Kernel Debug File System.
[ 1.218957] systemd[1]: Mounted sys-kernel-tracing.mount - Kernel Trace File System.
[ 1.225546] systemd[1]: Finished kmod-static-nodes.service - Create List of Static Device Nodes.
[ 1.226534] systemd[1]: modprobe@configfs.service: Deactivated successfully.
[ 1.226893] systemd[1]: Finished modprobe@configfs.service - Load Kernel Module configfs.
[ 1.227645] systemd[1]: modprobe@dm_mod.service: Deactivated successfully.
[ 1.228209] systemd[1]: Finished modprobe@dm_mod.service - Load Kernel Module dm_mod.
[ 1.230383] systemd[1]: modprobe@drm.service: Deactivated successfully.
[ 1.230725] systemd[1]: Finished modprobe@drm.service - Load Kernel Module drm.
[ 1.231459] systemd[1]: modprobe@efi_pstore.service: Deactivated successfully.
[ 1.231774] systemd[1]: Finished modprobe@efi_pstore.service - Load Kernel Module efi_pstore.
[ 1.232474] systemd[1]: modprobe@fuse.service: Deactivated successfully.
[ 1.232795] systemd[1]: Finished modprobe@fuse.service - Load Kernel Module fuse.
[ 1.234105] systemd[1]: modprobe@loop.service: Deactivated successfully.
[ 1.234462] systemd[1]: Finished modprobe@loop.service - Load Kernel Module loop.
[ 1.235099] systemd[1]: proc-sys-fs-binfmt_misc.automount: Got automount request for /proc/sys/fs/binfmt_misc, triggered by 140 (systemd-binfmt)
[ 1.261703] systemd[1]: Mounting proc-sys-fs-binfmt_misc.mount - Arbitrary Executable File Formats File System...
[ 1.270160] systemd[1]: Mounting sys-fs-fuse-connections.mount - FUSE Control File System...
[ 1.273274] systemd[1]: Mounting sys-kernel-config.mount - Kernel Configuration File System...
[ 1.273402] systemd[1]: systemd-pstore.service - Platform Persistent Storage Archival was skipped because of an unmet condition check (ConditionDirectoryNotEmpty=/sys/fs/pstore).
[ 1.273574] systemd[1]: systemd-repart.service - Repartition Root Disk was skipped because no trigger condition checks were met.
[ 1.290481] systemd[1]: Finished systemd-sysctl.service - Apply Kernel Variables.
[ 1.294159] systemd[1]: Finished systemd-sysusers.service - Create System Users.
[ 1.294555] systemd[1]: Mounted sys-fs-fuse-connections.mount - FUSE Control File System.
[ 1.303356] systemd[1]: Starting systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev...
[ 1.304817] systemd[1]: Mounted sys-kernel-config.mount - Kernel Configuration File System.
[ 1.320909] systemd[1]: Mounted proc-sys-fs-binfmt_misc.mount - Arbitrary Executable File Formats File System.
[ 1.323560] systemd[1]: Finished systemd-binfmt.service - Set Up Additional Binary Formats.
[ 1.369356] tsc: Refined TSC clocksource calibration: 3900.224 MHz
[ 1.369367] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x70705a6472c, max_idle_ns: 881590586812 ns
[ 1.370036] systemd[1]: Finished systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev.
[ 1.373207] clocksource: Switched to clocksource tsc
[ 1.389831] systemd[1]: Starting systemd-udevd.service - Rule-based Manager for Device Events and Files...
[ 1.427455] systemd[1]: Started systemd-journald.service - Journal Service.
[ 1.472737] systemd-journald[142]: Received client request to flush runtime journal.
[ 1.742223] sd 6:0:0:0: Attached scsi generic sg0 type 0
[ 1.866042] acpi_cpufreq: overriding BIOS provided _PSD data
[ 2.005347] random: crng init done
[ 2.137832] QUIRK: Enable AMD PLL fix
[ 2.137832] QUIRK: Enable AMD PLL fix
[ 2.137887] ehci-pci 0000:00:13.2: EHCI Host Controller
[ 2.137922] ehci-pci 0000:00:13.2: new USB bus registered, assigned bus number 1
[ 2.137936] ehci-pci 0000:00:13.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
[ 2.137945] ehci-pci 0000:00:13.2: debug port 1
[ 2.138141] ehci-pci 0000:00:13.2: irq 17, io mem 0xf01ce000
[ 2.139491] piix4_smbus 0000:00:14.0: SMBus Host Controller at 0xb00, revision 0
[ 2.139505] piix4_smbus 0000:00:14.0: Using register 0x2e for SMBus port selection
[ 2.140107] piix4_smbus 0000:00:14.0: Auxiliary SMBus Host Controller at 0xb20
[ 2.154632] ehci-pci 0000:00:13.2: USB 2.0 started, EHCI 1.00
[ 2.155284] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 2.155291] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.155294] usb usb1: Product: EHCI Host Controller
[ 2.155296] usb usb1: Manufacturer: Linux 6.3.0-rc3-00046-g8ba643d7e1c7 ehci_hcd
[ 2.155299] usb usb1: SerialNumber: 0000:00:13.2
[ 2.174521] hub 1-0:1.0: USB hub found
[ 2.174560] hub 1-0:1.0: 5 ports detected
[ 2.175582] ehci-pci 0000:00:12.2: EHCI Host Controller
[ 2.175610] ehci-pci 0000:00:12.2: new USB bus registered, assigned bus number 2
[ 2.175623] ehci-pci 0000:00:12.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
[ 2.175632] ehci-pci 0000:00:12.2: debug port 1
[ 2.175789] ehci-pci 0000:00:12.2: irq 17, io mem 0xf01cd000
[ 2.189373] ehci-pci 0000:00:12.2: USB 2.0 started, EHCI 1.00
[ 2.189720] usb usb2: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 2.189725] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.189727] usb usb2: Product: EHCI Host Controller
[ 2.189730] usb usb2: Manufacturer: Linux 6.3.0-rc3-00046-g8ba643d7e1c7 ehci_hcd
[ 2.189732] usb usb2: SerialNumber: 0000:00:12.2
[ 2.190283] hub 2-0:1.0: USB hub found
[ 2.190323] hub 2-0:1.0: 5 ports detected
[ 2.191801] ehci-pci 0000:00:16.2: EHCI Host Controller
[ 2.191831] ehci-pci 0000:00:16.2: new USB bus registered, assigned bus number 3
[ 2.191845] ehci-pci 0000:00:16.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
[ 2.191855] ehci-pci 0000:00:16.2: debug port 1
[ 2.192018] ehci-pci 0000:00:16.2: irq 17, io mem 0xf01cf000
[ 2.205373] ehci-pci 0000:00:16.2: USB 2.0 started, EHCI 1.00
[ 2.205651] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 2.205655] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.205658] usb usb3: Product: EHCI Host Controller
[ 2.205660] usb usb3: Manufacturer: Linux 6.3.0-rc3-00046-g8ba643d7e1c7 ehci_hcd
[ 2.205662] usb usb3: SerialNumber: 0000:00:16.2
[ 2.206200] hub 3-0:1.0: USB hub found
[ 2.206236] hub 3-0:1.0: 4 ports detected
[ 2.221053] ohci-pci 0000:00:12.0: OHCI PCI host controller
[ 2.221085] ohci-pci 0000:00:12.0: new USB bus registered, assigned bus number 4
[ 2.221264] ohci-pci 0000:00:12.0: irq 18, io mem 0xf01c8000
[ 2.237796] r8169 0000:04:00.0: enabling device (0000 -> 0003)
[ 2.261308] xhci_hcd 0000:03:00.0: xHCI Host Controller
[ 2.261394] xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 5
[ 2.272125] r8169 0000:04:00.0 eth0: RTL8168f/8111f, 08:60:6e:74:7a:51, XID 480, IRQ 28
[ 2.272143] r8169 0000:04:00.0 eth0: jumbo features [frames: 9194 bytes, tx checksumming: ko]
[ 2.282633] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 2.282644] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.282650] usb usb4: Product: OHCI PCI host controller
[ 2.282655] usb usb4: Manufacturer: Linux 6.3.0-rc3-00046-g8ba643d7e1c7 ohci_hcd
[ 2.282659] usb usb4: SerialNumber: 0000:00:12.0
[ 2.283372] hub 4-0:1.0: USB hub found
[ 2.283431] hub 4-0:1.0: 5 ports detected
[ 2.297576] ohci-pci 0000:00:13.0: OHCI PCI host controller
[ 2.297609] ohci-pci 0000:00:13.0: new USB bus registered, assigned bus number 6
[ 2.297727] ohci-pci 0000:00:13.0: irq 18, io mem 0xf01c9000
[ 2.320955] xhci_hcd 0000:03:00.0: hcc params 0x0200f180 hci version 0x96 quirks 0x0000000000080010
[ 2.322099] xhci_hcd 0000:03:00.0: xHCI Host Controller
[ 2.322121] xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 7
[ 2.322136] xhci_hcd 0000:03:00.0: Host supports USB 3.0 SuperSpeed
[ 2.324029] usb usb5: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 2.324036] usb usb5: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.324039] usb usb5: Product: xHCI Host Controller
[ 2.324041] usb usb5: Manufacturer: Linux 6.3.0-rc3-00046-g8ba643d7e1c7 xhci-hcd
[ 2.324043] usb usb5: SerialNumber: 0000:03:00.0
[ 2.324687] hub 5-0:1.0: USB hub found
[ 2.324727] hub 5-0:1.0: 2 ports detected
[ 2.325467] usb usb7: We don't know the algorithms for LPM for this host, disabling LPM.
[ 2.325624] usb usb7: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 6.03
[ 2.325628] usb usb7: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.325630] usb usb7: Product: xHCI Host Controller
[ 2.325632] usb usb7: Manufacturer: Linux 6.3.0-rc3-00046-g8ba643d7e1c7 xhci-hcd
[ 2.325634] usb usb7: SerialNumber: 0000:03:00.0
[ 2.326195] hub 7-0:1.0: USB hub found
[ 2.327034] hub 7-0:1.0: 2 ports detected
[ 2.349205] snd_hda_intel 0000:00:01.1: Force to non-snoop mode
[ 2.366399] usb usb6: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 2.366406] usb usb6: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.366409] usb usb6: Product: OHCI PCI host controller
[ 2.366411] usb usb6: Manufacturer: Linux 6.3.0-rc3-00046-g8ba643d7e1c7 ohci_hcd
[ 2.366413] usb usb6: SerialNumber: 0000:00:13.0
[ 2.367210] hub 6-0:1.0: USB hub found
[ 2.367303] hub 6-0:1.0: 5 ports detected
[ 2.391447] r8169 0000:04:00.0 enp4s0: renamed from eth0
[ 2.420037] ohci-pci 0000:00:14.5: OHCI PCI host controller
[ 2.420070] ohci-pci 0000:00:14.5: new USB bus registered, assigned bus number 8
[ 2.421556] ohci-pci 0000:00:14.5: irq 18, io mem 0xf01ca000
[ 2.445281] r8169 0000:04:00.0: Direct firmware load for rtl_nic/rtl8168f-1.fw failed with error -2
[ 2.445292] r8169 0000:04:00.0: Unable to load firmware rtl_nic/rtl8168f-1.fw (-2)
[ 2.445868] RTL8211E Gigabit Ethernet r8169-0-400:00: attached PHY driver (mii_bus:phy_addr=r8169-0-400:00, irq=MAC)
[ 2.481250] input: HDA ATI HDMI HDMI/DP,pcm=3 as /devices/pci0000:00/0000:00:01.1/sound/card0/input1
[ 2.481880] input: HDA ATI HDMI HDMI/DP,pcm=7 as /devices/pci0000:00/0000:00:01.1/sound/card0/input2
[ 2.488733] usb usb8: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 2.488740] usb usb8: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.488743] usb usb8: Product: OHCI PCI host controller
[ 2.488745] usb usb8: Manufacturer: Linux 6.3.0-rc3-00046-g8ba643d7e1c7 ohci_hcd
[ 2.488747] usb usb8: SerialNumber: 0000:00:14.5
[ 2.492816] snd_hda_codec_realtek hdaudioC1D0: ALC892: SKU not ready 0x00000100
[ 2.492902] hub 8-0:1.0: USB hub found
[ 2.492953] hub 8-0:1.0: 2 ports detected
[ 2.493612] snd_hda_codec_realtek hdaudioC1D0: autoconfig for ALC892: line_outs=4 (0x14/0x16/0x15/0x17/0x0) type:line
[ 2.493620] snd_hda_codec_realtek hdaudioC1D0: speaker_outs=0 (0x0/0x0/0x0/0x0/0x0)
[ 2.493623] snd_hda_codec_realtek hdaudioC1D0: hp_outs=1 (0x1b/0x0/0x0/0x0/0x0)
[ 2.493626] snd_hda_codec_realtek hdaudioC1D0: mono: mono_out=0x0
[ 2.493628] snd_hda_codec_realtek hdaudioC1D0: dig-out=0x1e/0x0
[ 2.493629] snd_hda_codec_realtek hdaudioC1D0: inputs:
[ 2.493631] snd_hda_codec_realtek hdaudioC1D0: Rear Mic=0x18
[ 2.493633] snd_hda_codec_realtek hdaudioC1D0: Front Mic=0x19
[ 2.493635] snd_hda_codec_realtek hdaudioC1D0: Line=0x1a
[ 2.493636] snd_hda_codec_realtek hdaudioC1D0: CD=0x1c
[ 2.495003] ohci-pci 0000:00:16.0: OHCI PCI host controller
[ 2.495036] ohci-pci 0000:00:16.0: new USB bus registered, assigned bus number 9
[ 2.495172] ohci-pci 0000:00:16.0: irq 18, io mem 0xf01cb000
[ 2.510556] [drm] radeon kernel modesetting enabled.
[ 2.513510] [drm] initializing kernel modesetting (ARUBA 0x1002:0x9996 0x1002:0x9996 0x00).
[ 2.513588] ATOM BIOS: 113
[ 2.513893] radeon 0000:00:01.0: VRAM: 512M 0x0000000000000000 - 0x000000001FFFFFFF (512M used)
[ 2.513898] radeon 0000:00:01.0: GTT: 1024M 0x0000000020000000 - 0x000000005FFFFFFF
[ 2.513911] [drm] Detected VRAM RAM=512M, BAR=256M
[ 2.513913] [drm] RAM width 64bits DDR
[ 2.514805] [drm] radeon: 512M of VRAM memory ready
[ 2.514815] [drm] radeon: 1024M of GTT memory ready.
[ 2.514888] [drm] Loading ARUBA Microcode
[ 2.517185] r8169 0000:04:00.0 enp4s0: Link is Down
[ 2.519927] input: HD-Audio Generic Rear Mic as /devices/pci0000:00/0000:00:14.2/sound/card1/input3
[ 2.520307] input: HD-Audio Generic Front Mic as /devices/pci0000:00/0000:00:14.2/sound/card1/input4
[ 2.520824] input: HD-Audio Generic Line as /devices/pci0000:00/0000:00:14.2/sound/card1/input5
[ 2.521180] input: HD-Audio Generic Line Out Front as /devices/pci0000:00/0000:00:14.2/sound/card1/input6
[ 2.521594] input: HD-Audio Generic Line Out Surround as /devices/pci0000:00/0000:00:14.2/sound/card1/input7
[ 2.522160] input: HD-Audio Generic Line Out CLFE as /devices/pci0000:00/0000:00:14.2/sound/card1/input8
[ 2.522536] input: HD-Audio Generic Line Out Side as /devices/pci0000:00/0000:00:14.2/sound/card1/input9
[ 2.522888] input: HD-Audio Generic Front Headphone as /devices/pci0000:00/0000:00:14.2/sound/card1/input10
[ 2.532990] [drm] Internal thermal controller without fan control
[ 2.558842] usb usb9: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 2.558849] usb usb9: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.558852] usb usb9: Product: OHCI PCI host controller
[ 2.558855] usb usb9: Manufacturer: Linux 6.3.0-rc3-00046-g8ba643d7e1c7 ohci_hcd
[ 2.558857] usb usb9: SerialNumber: 0000:00:16.0
[ 2.558984] [drm] radeon: dpm initialized
[ 2.564667] [drm] Found VCE firmware/feedback version 50.0.1 / 17!
[ 2.564728] [drm] GART: num cpu pages 262144, num gpu pages 262144
[ 2.594051] [drm] PCIE GART of 1024M enabled (table at 0x00000000001D6000).
[ 2.594314] radeon 0000:00:01.0: WB enabled
[ 2.594318] radeon 0000:00:01.0: fence driver on ring 0 use gpu addr 0x0000000020000c00
[ 2.594697] radeon 0000:00:01.0: fence driver on ring 5 use gpu addr 0x0000000000075a18
[ 2.595711] hub 9-0:1.0: USB hub found
[ 2.595753] hub 9-0:1.0: 4 ports detected
[ 2.618866] radeon 0000:00:01.0: fence driver on ring 6 use gpu addr 0x0000000020000c18
[ 2.618874] radeon 0000:00:01.0: fence driver on ring 7 use gpu addr 0x0000000020000c1c
[ 2.618877] radeon 0000:00:01.0: fence driver on ring 1 use gpu addr 0x0000000020000c04
[ 2.618879] radeon 0000:00:01.0: fence driver on ring 2 use gpu addr 0x0000000020000c08
[ 2.618881] radeon 0000:00:01.0: fence driver on ring 3 use gpu addr 0x0000000020000c0c
[ 2.618883] radeon 0000:00:01.0: fence driver on ring 4 use gpu addr 0x0000000020000c10
[ 2.647823] radeon 0000:00:01.0: radeon: MSI limited to 32-bit
[ 2.648048] radeon 0000:00:01.0: radeon: using MSI.
[ 2.648125] [drm] radeon: irq initialized.
[ 2.667958] [drm] ring test on 0 succeeded in 3 usecs
[ 2.667970] [drm] ring test on 3 succeeded in 4 usecs
[ 2.667977] [drm] ring test on 4 succeeded in 4 usecs
[ 2.713784] [drm] ring test on 5 succeeded in 2 usecs
[ 2.733666] [drm] UVD initialized successfully.
[ 2.741341] usb 4-1: new low-speed USB device number 2 using ohci-pci
[ 2.843040] [drm] ring test on 6 succeeded in 18 usecs
[ 2.843052] [drm] ring test on 7 succeeded in 3 usecs
[ 2.843053] [drm] VCE initialized successfully.
[ 2.843202] snd_hda_intel 0000:00:01.1: bound 0000:00:01.0 (ops radeon_audio_component_bind_ops [radeon])
[ 2.843370] [drm] ib test on ring 0 succeeded in 0 usecs
[ 2.843424] [drm] ib test on ring 3 succeeded in 0 usecs
[ 2.843474] [drm] ib test on ring 4 succeeded in 0 usecs
[ 2.942595] usb 4-1: New USB device found, idVendor=413c, idProduct=2106, bcdDevice= 1.01
[ 2.942607] usb 4-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 2.942611] usb 4-1: Product: Dell QuietKey Keyboard
[ 2.942615] usb 4-1: Manufacturer: DELL
[ 2.951006] input: DELL Dell QuietKey Keyboard as /devices/pci0000:00/0000:00:12.0/usb4/4-1/4-1:1.0/0003:413C:2106.0001/input/input11
[ 3.010775] hid-generic 0003:413C:2106.0001: input,hidraw0: USB HID v1.10 Keyboard [DELL Dell QuietKey Keyboard] on usb-0000:00:12.0-1/input0
[ 3.385454] [drm] ib test on ring 5 succeeded
[ 3.405349] usb 4-2: new low-speed USB device number 3 using ohci-pci
[ 3.600597] usb 4-2: New USB device found, idVendor=046d, idProduct=c016, bcdDevice= 3.40
[ 3.600609] usb 4-2: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 3.600613] usb 4-2: Product: Optical USB Mouse
[ 3.600617] usb 4-2: Manufacturer: Logitech
[ 3.610963] input: Logitech Optical USB Mouse as /devices/pci0000:00/0000:00:12.0/usb4/4-2/4-2:1.0/0003:046D:C016.0002/input/input12
[ 3.611917] hid-generic 0003:046D:C016.0002: input,hidraw1: USB HID v1.10 Mouse [Logitech Optical USB Mouse] on usb-0000:00:12.0-2/input0
[ 3.929454] [drm] ib test on ring 6 succeeded
[ 4.441455] [drm] ib test on ring 7 succeeded
[ 4.446983] [drm] Radeon Display Connectors
[ 4.446989] [drm] Connector 0:
[ 4.446992] [drm] DP-1
[ 4.446995] [drm] HPD1
[ 4.446997] [drm] DDC: 0x6530 0x6530 0x6534 0x6534 0x6538 0x6538 0x653c 0x653c
[ 4.447003] [drm] Encoders:
[ 4.447005] [drm] DFP1: INTERNAL_UNIPHY2
[ 4.447007] [drm] Connector 1:
[ 4.447009] [drm] VGA-1
[ 4.447011] [drm] HPD2
[ 4.447013] [drm] DDC: 0x6540 0x6540 0x6544 0x6544 0x6548 0x6548 0x654c 0x654c
[ 4.447018] [drm] Encoders:
[ 4.447020] [drm] CRT1: INTERNAL_UNIPHY2
[ 4.447022] [drm] CRT1: NUTMEG
[ 4.447024] [drm] Connector 2:
[ 4.447026] [drm] HDMI-A-1
[ 4.447028] [drm] HPD3
[ 4.447030] [drm] DDC: 0x6550 0x6550 0x6554 0x6554 0x6558 0x6558 0x655c 0x655c
[ 4.447035] [drm] Encoders:
[ 4.447036] [drm] DFP2: INTERNAL_UNIPHY
[ 4.716171] [drm] fb mappable at 0xE03E9000
[ 4.716179] [drm] vram apper at 0xE0000000
[ 4.716181] [drm] size 5242880
[ 4.716183] [drm] fb depth is 24
[ 4.716185] [drm] pitch is 5120
[ 4.716671] fbcon: radeondrmfb (fb0) is primary device
[ 4.908197] Console: switching to colour frame buffer device 160x64
[ 4.909974] radeon 0000:00:01.0: [drm] fb0: radeondrmfb frame buffer device
[ 4.933682] [drm] Initialized radeon 2.50.0 20080528 for 0000:00:01.0 on minor 0
[ 5.125043] r8169 0000:04:00.0 enp4s0: Link is Up - 1Gbps/Full - flow control rx/tx
[ 5.125059] IPv6: ADDRCONF(NETDEV_CHANGE): enp4s0: link becomes ready
[ 7.669093] [drm] amdgpu kernel modesetting enabled.
[ 11.216542] memfd_create() without MFD_EXEC nor MFD_NOEXEC_SEAL, pid=272 'systemd'
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-24 18:46 ` Paul Menzel
0 siblings, 0 replies; 236+ messages in thread
From: Paul Menzel @ 2023-04-24 18:46 UTC (permalink / raw)
To: Thomas Gleixner
Cc: Sean Christopherson, Andrew Cooper, linux-kernel, x86,
David Woodhouse, Brian Gerst, Arjan van de Veen, Paolo Bonzini,
Paul McKenney, Tom Lendacky, Oleksandr Natalenko,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
[-- Attachment #1: Type: text/plain, Size: 722 bytes --]
Dear Thomas,
Am 20.04.23 um 21:10 schrieb Thomas Gleixner:
> On Thu, Apr 20 2023 at 18:47, Paul Menzel wrote:
>> Am 20.04.23 um 17:57 schrieb Thomas Gleixner:
>> I quickly applied it on top of your branch, but I am getting:
>
> As I said it was untested. I was traveling and did not have access to a
> machine to even build it completely. Fixed up and tested version below.
Sorry, if it sounded like a complaint. I just wanted to give a quick
feedback.
[…]
I tested your new version even on Friday, and it worked fine – no ten
seconds delay. Please find the messages attached.
Thank you all for your great work.
Kind regards,
Paul
PS: I am going to try to test your updated branch at the end of the week.
[-- Attachment #2: kodi-linux-6.3-rc3-smp-tglx-with-apic-fix.txt --]
[-- Type: text/plain, Size: 57511 bytes --]
[ 0.000000] Linux version 6.3.0-rc3-00046-g8ba643d7e1c7 (root@bf16f3646a84) (gcc (Debian 11.2.0-12) 11.2.0, GNU ld (GNU Binutils for Debian) 2.40) #452 SMP PREEMPT_DYNAMIC Thu Apr 20 20:15:01 UTC 2023
[ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc3-00046-g8ba643d7e1c7 root=/dev/sda3 rw quiet noisapnp cryptomgr.notests ipv6.disable_ipv6=1 selinux=0
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256
[ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format.
[ 0.000000] signal: max sigframe size: 1776
[ 0.000000] BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
[ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005fe4cfff] usable
[ 0.000000] BIOS-e820: [mem 0x000000005fe4d000-0x000000007fffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000f8000000-0x00000000fbffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000fec10000-0x00000000fec10fff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000017effffff] usable
[ 0.000000] NX (Execute Disable) protection: active
[ 0.000000] SMBIOS 3.0.0 present.
[ 0.000000] DMI: ASUS F2A85-M_PRO/F2A85-M_PRO, BIOS 4.18-9-gb640ed51b2 04/17/2023
[ 0.000000] tsc: Fast TSC calibration using PIT
[ 0.000000] tsc: Detected 3900.440 MHz processor
[ 0.000756] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
[ 0.000759] e820: remove [mem 0x000a0000-0x000fffff] usable
[ 0.000763] last_pfn = 0x17f000 max_arch_pfn = 0x400000000
[ 0.000768] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT
[ 0.000938] last_pfn = 0x5fe4d max_arch_pfn = 0x400000000
[ 0.004000] Using GB pages for direct mapping
[ 0.004000] ACPI: Early table checksum verification disabled
[ 0.004000] ACPI: RSDP 0x00000000000F6830 000024 (v02 COREv4)
[ 0.004000] ACPI: XSDT 0x000000005FE5A0E0 000074 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: FACP 0x000000005FE5BBC0 000114 (v06 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: DSDT 0x000000005FE5A280 00193A (v02 COREv4 COREBOOT 00010001 INTL 20200925)
[ 0.004000] ACPI: FACS 0x000000005FE5A240 000040
[ 0.004000] ACPI: FACS 0x000000005FE5A240 000040
[ 0.004000] ACPI: SSDT 0x000000005FE5BCE0 00008A (v02 COREv4 COREBOOT 0000002A CORE 20200925)
[ 0.004000] ACPI: MCFG 0x000000005FE5BD70 00003C (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: APIC 0x000000005FE5BDB0 000062 (v03 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: HPET 0x000000005FE5BE20 000038 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: HEST 0x000000005FE5BE60 0001D0 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: IVRS 0x000000005FE5C030 000070 (v02 AMD AMDIOMMU 00000001 AMD 00000000)
[ 0.004000] ACPI: SSDT 0x000000005FE5C0A0 00051F (v02 AMD ALIB 00000001 MSFT 04000000)
[ 0.004000] ACPI: SSDT 0x000000005FE5C5C0 0006B2 (v01 AMD POWERNOW 00000001 AMD 00000001)
[ 0.004000] ACPI: VFCT 0x000000005FE5CC80 00F269 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: Reserving FACP table memory at [mem 0x5fe5bbc0-0x5fe5bcd3]
[ 0.004000] ACPI: Reserving DSDT table memory at [mem 0x5fe5a280-0x5fe5bbb9]
[ 0.004000] ACPI: Reserving FACS table memory at [mem 0x5fe5a240-0x5fe5a27f]
[ 0.004000] ACPI: Reserving FACS table memory at [mem 0x5fe5a240-0x5fe5a27f]
[ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe5bce0-0x5fe5bd69]
[ 0.004000] ACPI: Reserving MCFG table memory at [mem 0x5fe5bd70-0x5fe5bdab]
[ 0.004000] ACPI: Reserving APIC table memory at [mem 0x5fe5bdb0-0x5fe5be11]
[ 0.004000] ACPI: Reserving HPET table memory at [mem 0x5fe5be20-0x5fe5be57]
[ 0.004000] ACPI: Reserving HEST table memory at [mem 0x5fe5be60-0x5fe5c02f]
[ 0.004000] ACPI: Reserving IVRS table memory at [mem 0x5fe5c030-0x5fe5c09f]
[ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe5c0a0-0x5fe5c5be]
[ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe5c5c0-0x5fe5cc71]
[ 0.004000] ACPI: Reserving VFCT table memory at [mem 0x5fe5cc80-0x5fe6bee8]
[ 0.004000] No NUMA configuration found
[ 0.004000] Faking a node at [mem 0x0000000000000000-0x000000017effffff]
[ 0.004000] NODE_DATA(0) allocated [mem 0x17efe9000-0x17effffff]
[ 0.004000] Zone ranges:
[ 0.004000] DMA [mem 0x0000000000001000-0x0000000000ffffff]
[ 0.004000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]
[ 0.004000] Normal [mem 0x0000000100000000-0x000000017effffff]
[ 0.004000] Device empty
[ 0.004000] Movable zone start for each node
[ 0.004000] Early memory node ranges
[ 0.004000] node 0: [mem 0x0000000000001000-0x000000000009efff]
[ 0.004000] node 0: [mem 0x0000000000100000-0x000000005fe4cfff]
[ 0.004000] node 0: [mem 0x0000000100000000-0x000000017effffff]
[ 0.004000] Initmem setup node 0 [mem 0x0000000000001000-0x000000017effffff]
[ 0.004000] On node 0, zone DMA: 1 pages in unavailable ranges
[ 0.004000] On node 0, zone DMA: 97 pages in unavailable ranges
[ 0.004000] On node 0, zone Normal: 435 pages in unavailable ranges
[ 0.004000] On node 0, zone Normal: 4096 pages in unavailable ranges
[ 0.004000] ACPI: PM-Timer IO Port: 0x818
[ 0.004000] ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1])
[ 0.004000] IOAPIC[0]: apic_id 4, version 33, address 0xfec00000, GSI 0-23
[ 0.004000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[ 0.004000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 low level)
[ 0.004000] ACPI: Using ACPI (MADT) for SMP configuration information
[ 0.004000] ACPI: HPET id: 0x10228210 base: 0xfed00000
[ 0.004000] smpboot: Allowing 2 CPUs, 0 hotplug CPUs
[ 0.004000] [mem 0x80000000-0xf7ffffff] available for PCI devices
[ 0.004000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns
[ 0.004000] setup_percpu: NR_CPUS:64 nr_cpumask_bits:2 nr_cpu_ids:2 nr_node_ids:1
[ 0.004000] percpu: Embedded 55 pages/cpu s188392 r8192 d28696 u1048576
[ 0.004000] pcpu-alloc: s188392 r8192 d28696 u1048576 alloc=1*2097152
[ 0.004000] pcpu-alloc: [0] 0 1
[ 0.004000] Fallback order for Node 0: 0
[ 0.004000] Built 1 zonelists, mobility grouping on. Total pages: 898451
[ 0.004000] Policy zone: Normal
[ 0.004000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc3-00046-g8ba643d7e1c7 root=/dev/sda3 rw quiet noisapnp cryptomgr.notests ipv6.disable_ipv6=1 selinux=0
[ 0.004000] Unknown kernel command line parameters "noisapnp BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc3-00046-g8ba643d7e1c7", will be passed to user space.
[ 0.004000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[ 0.004000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[ 0.004000] mem auto-init: stack:off, heap alloc:off, heap free:off
[ 0.004000] stackdepot: allocating hash table via alloc_large_system_hash
[ 0.004000] stackdepot hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[ 0.004000] software IO TLB: area num 2.
[ 0.004000] Memory: 3477168K/3651500K available (14336K kernel code, 2340K rwdata, 5308K rodata, 2908K init, 11064K bss, 174072K reserved, 0K cma-reserved)
[ 0.004000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
[ 0.004000] ftrace: allocating 38652 entries in 151 pages
[ 0.004000] ftrace: allocated 151 pages with 5 groups
[ 0.004000] Dynamic Preempt: full
[ 0.004000] rcu: Preemptible hierarchical RCU implementation.
[ 0.004000] rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=2.
[ 0.004000] Trampoline variant of Tasks RCU enabled.
[ 0.004000] Rude variant of Tasks RCU enabled.
[ 0.004000] Tracing variant of Tasks RCU enabled.
[ 0.004000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[ 0.004000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2
[ 0.004000] NR_IRQS: 4352, nr_irqs: 440, preallocated irqs: 16
[ 0.004000] rcu: srcu_init: Setting srcu_struct sizes based on contention.
[ 0.004000] spurious 8259A interrupt: IRQ7.
[ 0.004000] Console: colour VGA+ 80x25
[ 0.004000] printk: console [tty0] enabled
[ 0.004000] ACPI: Core revision 20221020
[ 0.004000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 133484873504 ns
[ 0.004000] APIC: Switch to symmetric I/O mode setup
[ 0.004000] AMD-Vi: Using global IVHD EFR:0x0, EFR2:0x0
[ 0.004000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[ 0.004000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x7071f4ed18f, max_idle_ns: 881590420850 ns
[ 0.145332] Calibrating delay loop (skipped), value calculated using timer frequency.. 7800.88 BogoMIPS (lpj=15601760)
[ 0.145336] pid_max: default: 32768 minimum: 301
[ 0.145430] LSM: initializing lsm=capability
[ 0.145526] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[ 0.145542] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[ 0.145940] Last level iTLB entries: 4KB 512, 2MB 1024, 4MB 512
[ 0.145943] Last level dTLB entries: 4KB 1024, 2MB 1024, 4MB 512, 1GB 0
[ 0.145947] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization
[ 0.145950] Spectre V2 : Mitigation: Retpolines
[ 0.145951] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch
[ 0.145952] Spectre V2 : Spectre v2 / SpectreRSB : Filling RSB on VMEXIT
[ 0.145952] Spectre V2 : Enabling Speculation Barrier for firmware calls
[ 0.145953] RETBleed: Mitigation: untrained return thunk
[ 0.145955] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier
[ 0.145957] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl
[ 0.150469] Freeing SMP alternatives memory: 32K
[ 0.258608] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD Graphics (family: 0x15, model: 0x13, stepping: 0x1)
[ 0.258845] cblist_init_generic: Setting adjustable number of callback queues.
[ 0.258846] cblist_init_generic: Setting shift to 1 and lim to 1.
[ 0.258878] cblist_init_generic: Setting shift to 1 and lim to 1.
[ 0.258905] cblist_init_generic: Setting shift to 1 and lim to 1.
[ 0.258933] Performance Events: Fam15h core perfctr, AMD PMU driver.
[ 0.258955] ... version: 0
[ 0.258956] ... bit width: 48
[ 0.258957] ... generic registers: 6
[ 0.258958] ... value mask: 0000ffffffffffff
[ 0.258959] ... max period: 00007fffffffffff
[ 0.258960] ... fixed-purpose events: 0
[ 0.258961] ... event mask: 000000000000003f
[ 0.259083] rcu: Hierarchical SRCU implementation.
[ 0.259084] rcu: Max phase no-delay instances is 1000.
[ 0.259676] NMI watchdog: Enabled. Permanently consumes one hw-PMU counter.
[ 0.259750] smp: Bringing up secondary CPUs ...
[ 0.259952] x86: Booting SMP configuration:
[ 0.259953] .... node #0, CPUs: #1
[ 0.259958] smpboot: Kicking AP alive: 17
[ 0.260088] smp: Brought up 1 node, 2 CPUs
[ 0.260088] smpboot: Max logical packages: 1
[ 0.260088] smpboot: Total of 2 processors activated (15601.76 BogoMIPS)
[ 0.261513] devtmpfs: initialized
[ 0.261513] x86/mm: Memory block size: 128MB
[ 0.262387] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[ 0.262387] futex hash table entries: 512 (order: 3, 32768 bytes, linear)
[ 0.262387] pinctrl core: initialized pinctrl subsystem
[ 0.262387] PM: RTC time: 20:56:07, date: 2023-04-21
[ 0.262387] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[ 0.262600] audit: initializing netlink subsys (disabled)
[ 0.262617] audit: type=2000 audit(1682110567.140:1): state=initialized audit_enabled=0 res=1
[ 0.262617] thermal_sys: Registered thermal governor 'fair_share'
[ 0.262617] thermal_sys: Registered thermal governor 'bang_bang'
[ 0.262617] thermal_sys: Registered thermal governor 'step_wise'
[ 0.262617] thermal_sys: Registered thermal governor 'user_space'
[ 0.262617] cpuidle: using governor ladder
[ 0.262617] cpuidle: using governor menu
[ 0.262617] PCI: MMCONFIG for domain 0000 [bus 00-3f] at [mem 0xf8000000-0xfbffffff] (base 0xf8000000)
[ 0.262617] PCI: MMCONFIG at [mem 0xf8000000-0xfbffffff] reserved as E820 entry
[ 0.262617] PCI: Using configuration type 1 for base access
[ 0.262617] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.
[ 0.273356] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages
[ 0.273356] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page
[ 0.273356] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages
[ 0.273356] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page
[ 0.280509] cryptd: max_cpu_qlen set to 1000
[ 0.280509] ACPI: Added _OSI(Module Device)
[ 0.280509] ACPI: Added _OSI(Processor Device)
[ 0.280509] ACPI: Added _OSI(3.0 _SCP Extensions)
[ 0.280509] ACPI: Added _OSI(Processor Aggregator Device)
[ 0.287039] ACPI: 4 ACPI AML tables successfully acquired and loaded
[ 0.288072] ACPI: Interpreter enabled
[ 0.288072] ACPI: PM: (supports S0 S1 S3 S5)
[ 0.288072] ACPI: Using IOAPIC for interrupt routing
[ 0.288072] HEST: Table parsing has been initialized.
[ 0.288072] GHES: Failed to enable APEI firmware first mode.
[ 0.288072] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[ 0.288072] PCI: Ignoring E820 reservations for host bridge windows
[ 0.288072] ACPI: Enabled 8 GPEs in block 00 to 1F
[ 0.290922] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[ 0.290932] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]
[ 0.291018] acpi PNP0A03:00: _OSC: OS now controls [PME AER PCIeCapability LTR]
[ 0.291034] acpi PNP0A03:00: [Firmware Info]: MMCONFIG for domain 0000 [bus 00-3f] only partially covers this bridge
[ 0.291117] acpi PNP0A03:00: host bridge window expanded to [io 0x0000-0x0cf7 window]; [io 0x03b0-0x03df window] ignored
[ 0.291369] PCI host bridge to bus 0000:00
[ 0.291371] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]
[ 0.291374] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]
[ 0.291376] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000dffff]
[ 0.291379] pci_bus 0000:00: root bus resource [mem 0x80000000-0xffffffff]
[ 0.291381] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 0.291406] pci 0000:00:00.0: [1022:1410] type 00 class 0x060000
[ 0.291559] pci 0000:00:00.2: [1022:1419] type 00 class 0x080600
[ 0.291648] pci 0000:00:01.0: [1002:9996] type 00 class 0x030000
[ 0.291656] pci 0000:00:01.0: reg 0x10: [mem 0xe0000000-0xefffffff pref]
[ 0.291661] pci 0000:00:01.0: reg 0x14: [io 0x1000-0x10ff]
[ 0.291665] pci 0000:00:01.0: reg 0x18: [mem 0xf0180000-0xf01bffff]
[ 0.291681] pci 0000:00:01.0: enabling Extended Tags
[ 0.291694] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]
[ 0.291711] pci 0000:00:01.0: supports D1 D2
[ 0.291776] pci 0000:00:01.1: [1002:9902] type 00 class 0x040300
[ 0.291784] pci 0000:00:01.1: reg 0x10: [mem 0xf01c0000-0xf01c3fff]
[ 0.291805] pci 0000:00:01.1: enabling Extended Tags
[ 0.291828] pci 0000:00:01.1: supports D1 D2
[ 0.291918] pci 0000:00:11.0: [1022:7801] type 00 class 0x010601
[ 0.291931] pci 0000:00:11.0: reg 0x10: [io 0x1410-0x1417]
[ 0.291938] pci 0000:00:11.0: reg 0x14: [io 0x1420-0x1423]
[ 0.291946] pci 0000:00:11.0: reg 0x18: [io 0x1418-0x141f]
[ 0.291953] pci 0000:00:11.0: reg 0x1c: [io 0x1424-0x1427]
[ 0.291960] pci 0000:00:11.0: reg 0x20: [io 0x1400-0x140f]
[ 0.291968] pci 0000:00:11.0: reg 0x24: [mem 0xf01cc000-0xf01cc7ff]
[ 0.292125] pci 0000:00:12.0: [1022:7807] type 00 class 0x0c0310
[ 0.292139] pci 0000:00:12.0: reg 0x10: [mem 0xf01c8000-0xf01c8fff]
[ 0.292322] pci 0000:00:12.2: [1022:7808] type 00 class 0x0c0320
[ 0.292335] pci 0000:00:12.2: reg 0x10: [mem 0xf01cd000-0xf01cd0ff]
[ 0.292400] pci 0000:00:12.2: supports D1 D2
[ 0.292402] pci 0000:00:12.2: PME# supported from D0 D1 D2 D3hot
[ 0.292542] pci 0000:00:13.0: [1022:7807] type 00 class 0x0c0310
[ 0.292555] pci 0000:00:13.0: reg 0x10: [mem 0xf01c9000-0xf01c9fff]
[ 0.292737] pci 0000:00:13.2: [1022:7808] type 00 class 0x0c0320
[ 0.292750] pci 0000:00:13.2: reg 0x10: [mem 0xf01ce000-0xf01ce0ff]
[ 0.292815] pci 0000:00:13.2: supports D1 D2
[ 0.292816] pci 0000:00:13.2: PME# supported from D0 D1 D2 D3hot
[ 0.292955] pci 0000:00:14.0: [1022:780b] type 00 class 0x0c0500
[ 0.293139] pci 0000:00:14.2: [1022:780d] type 00 class 0x040300
[ 0.293156] pci 0000:00:14.2: reg 0x10: [mem 0xf01c4000-0xf01c7fff 64bit]
[ 0.293210] pci 0000:00:14.2: PME# supported from D0 D3hot D3cold
[ 0.293355] pci 0000:00:14.3: [1022:780e] type 00 class 0x060100
[ 0.293541] pci 0000:00:14.4: [1022:780f] type 01 class 0x060401
[ 0.293703] pci 0000:00:14.5: [1022:7809] type 00 class 0x0c0310
[ 0.293717] pci 0000:00:14.5: reg 0x10: [mem 0xf01ca000-0xf01cafff]
[ 0.293906] pci 0000:00:15.0: [1022:43a0] type 01 class 0x060400
[ 0.293935] pci 0000:00:15.0: enabling Extended Tags
[ 0.293975] pci 0000:00:15.0: supports D1 D2
[ 0.294151] pci 0000:00:15.1: [1022:43a1] type 01 class 0x060400
[ 0.294182] pci 0000:00:15.1: enabling Extended Tags
[ 0.294221] pci 0000:00:15.1: supports D1 D2
[ 0.294398] pci 0000:00:15.2: [1022:43a2] type 01 class 0x060400
[ 0.294426] pci 0000:00:15.2: enabling Extended Tags
[ 0.294468] pci 0000:00:15.2: supports D1 D2
[ 0.294547] pci 0000:00:16.0: [1022:7807] type 00 class 0x0c0310
[ 0.294560] pci 0000:00:16.0: reg 0x10: [mem 0xf01cb000-0xf01cbfff]
[ 0.294737] pci 0000:00:16.2: [1022:7808] type 00 class 0x0c0320
[ 0.294751] pci 0000:00:16.2: reg 0x10: [mem 0xf01cf000-0xf01cf0ff]
[ 0.294815] pci 0000:00:16.2: supports D1 D2
[ 0.294817] pci 0000:00:16.2: PME# supported from D0 D1 D2 D3hot
[ 0.294958] pci 0000:00:18.0: [1022:1400] type 00 class 0x060000
[ 0.295024] pci 0000:00:18.1: [1022:1401] type 00 class 0x060000
[ 0.295086] pci 0000:00:18.2: [1022:1402] type 00 class 0x060000
[ 0.295152] pci 0000:00:18.3: [1022:1403] type 00 class 0x060000
[ 0.295289] pci 0000:00:18.4: [1022:1404] type 00 class 0x060000
[ 0.295353] pci 0000:00:18.5: [1022:1405] type 00 class 0x060000
[ 0.295436] pci_bus 0000:01: extended config space not accessible
[ 0.295499] pci 0000:00:14.4: PCI bridge to [bus 01] (subtractive decode)
[ 0.295510] pci 0000:00:14.4: bridge window [io 0x0000-0x0cf7 window] (subtractive decode)
[ 0.295513] pci 0000:00:14.4: bridge window [io 0x0d00-0xffff window] (subtractive decode)
[ 0.295515] pci 0000:00:14.4: bridge window [mem 0x000a0000-0x000dffff] (subtractive decode)
[ 0.295517] pci 0000:00:14.4: bridge window [mem 0x80000000-0xffffffff] (subtractive decode)
[ 0.295571] pci 0000:00:15.0: PCI bridge to [bus 02]
[ 0.295653] pci 0000:03:00.0: [1b21:1042] type 00 class 0x0c0330
[ 0.295689] pci 0000:03:00.0: reg 0x10: [mem 0xf0000000-0xf0007fff 64bit]
[ 0.295863] pci 0000:03:00.0: PME# supported from D3hot D3cold
[ 0.295907] pci 0000:03:00.0: 2.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s PCIe x1 link at 0000:00:15.1 (capable of 4.000 Gb/s with 5.0 GT/s PCIe x1 link)
[ 0.309393] pci 0000:00:15.1: PCI bridge to [bus 03]
[ 0.309405] pci 0000:00:15.1: bridge window [mem 0xf0000000-0xf00fffff]
[ 0.309414] pci 0000:00:15.2: bridge configuration invalid ([bus 00-00]), reconfiguring
[ 0.309545] pci 0000:04:00.0: [10ec:8168] type 00 class 0x020000
[ 0.309563] pci 0000:04:00.0: reg 0x10: [io 0x0000-0x00ff]
[ 0.309585] pci 0000:04:00.0: reg 0x18: [mem 0x00000000-0x00000fff 64bit pref]
[ 0.309598] pci 0000:04:00.0: reg 0x20: [mem 0x00000000-0x00003fff 64bit pref]
[ 0.309705] pci 0000:04:00.0: supports D1 D2
[ 0.309707] pci 0000:04:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[ 0.325403] pci 0000:00:15.2: PCI bridge to [bus 04-ff]
[ 0.325414] pci 0000:00:15.2: bridge window [io 0x0000-0x0fff]
[ 0.325417] pci 0000:00:15.2: bridge window [mem 0x00000000-0x000fffff]
[ 0.325422] pci 0000:00:15.2: bridge window [mem 0x00000000-0x000fffff 64bit pref]
[ 0.325425] pci_bus 0000:04: busn_res: [bus 04-ff] end is updated to 04
[ 0.325919] ACPI: PCI: Interrupt link INTA configured for IRQ 0
[ 0.326011] ACPI: PCI: Interrupt link INTB configured for IRQ 0
[ 0.326102] ACPI: PCI: Interrupt link INTC configured for IRQ 0
[ 0.326193] ACPI: PCI: Interrupt link INTD configured for IRQ 0
[ 0.326284] ACPI: PCI: Interrupt link INTE configured for IRQ 0
[ 0.326375] ACPI: PCI: Interrupt link INTF configured for IRQ 0
[ 0.326465] ACPI: PCI: Interrupt link INTG configured for IRQ 0
[ 0.326556] ACPI: PCI: Interrupt link INTH configured for IRQ 0
[ 0.326777] iommu: Default domain type: Translated
[ 0.326778] iommu: DMA domain TLB invalidation policy: lazy mode
[ 0.326974] SCSI subsystem initialized
[ 0.329389] libata version 3.00 loaded.
[ 0.329395] ACPI: bus type USB registered
[ 0.329418] usbcore: registered new interface driver usbfs
[ 0.329428] usbcore: registered new interface driver hub
[ 0.329437] usbcore: registered new device driver usb
[ 0.329564] PCI: Using ACPI for IRQ routing
[ 0.331132] PCI: pci_cache_line_size set to 64 bytes
[ 0.331184] e820: reserve RAM buffer [mem 0x0009fc00-0x0009ffff]
[ 0.331187] e820: reserve RAM buffer [mem 0x5fe4d000-0x5fffffff]
[ 0.331189] e820: reserve RAM buffer [mem 0x17f000000-0x17fffffff]
[ 0.331235] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0
[ 0.331240] hpet0: 3 comparators, 32-bit 14.318180 MHz counter
[ 0.334410] clocksource: Switched to clocksource tsc-early
[ 0.334656] VFS: Disk quotas dquot_6.6.0
[ 0.334682] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[ 0.334803] pnp: PnP ACPI init
[ 0.335147] system 00:00: [mem 0xfec10002-0xfec11001] could not be reserved
[ 0.335525] pnp: PnP ACPI: found 2 devices
[ 0.342072] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
[ 0.342260] NET: Registered PF_INET protocol family
[ 0.342434] IP idents hash table entries: 65536 (order: 7, 524288 bytes, linear)
[ 0.344041] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear)
[ 0.344058] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
[ 0.344070] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear)
[ 0.344141] TCP bind hash table entries: 32768 (order: 8, 1048576 bytes, linear)
[ 0.344505] TCP: Hash tables configured (established 32768 bind 32768)
[ 0.344576] UDP hash table entries: 2048 (order: 4, 65536 bytes, linear)
[ 0.344601] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear)
[ 0.344722] NET: Registered PF_UNIX/PF_LOCAL protocol family
[ 0.344757] pci 0000:00:15.2: BAR 15: assigned [mem 0x80000000-0x800fffff 64bit pref]
[ 0.344763] pci 0000:00:15.2: BAR 13: assigned [io 0x2000-0x2fff]
[ 0.344769] pci 0000:00:14.4: PCI bridge to [bus 01]
[ 0.344781] pci 0000:00:15.0: PCI bridge to [bus 02]
[ 0.344790] pci 0000:00:15.1: PCI bridge to [bus 03]
[ 0.344793] pci 0000:00:15.1: bridge window [mem 0xf0000000-0xf00fffff]
[ 0.344807] pci 0000:04:00.0: BAR 4: assigned [mem 0x80000000-0x80003fff 64bit pref]
[ 0.344821] pci 0000:04:00.0: BAR 2: assigned [mem 0x80004000-0x80004fff 64bit pref]
[ 0.344833] pci 0000:04:00.0: BAR 0: assigned [io 0x2000-0x20ff]
[ 0.344840] pci 0000:00:15.2: PCI bridge to [bus 04]
[ 0.344842] pci 0000:00:15.2: bridge window [io 0x2000-0x2fff]
[ 0.344847] pci 0000:00:15.2: bridge window [mem 0x80000000-0x800fffff 64bit pref]
[ 0.344854] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]
[ 0.344857] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]
[ 0.344860] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000dffff]
[ 0.344862] pci_bus 0000:00: resource 7 [mem 0x80000000-0xffffffff]
[ 0.344864] pci_bus 0000:01: resource 4 [io 0x0000-0x0cf7 window]
[ 0.344867] pci_bus 0000:01: resource 5 [io 0x0d00-0xffff window]
[ 0.344869] pci_bus 0000:01: resource 6 [mem 0x000a0000-0x000dffff]
[ 0.344871] pci_bus 0000:01: resource 7 [mem 0x80000000-0xffffffff]
[ 0.344873] pci_bus 0000:03: resource 1 [mem 0xf0000000-0xf00fffff]
[ 0.344875] pci_bus 0000:04: resource 0 [io 0x2000-0x2fff]
[ 0.344877] pci_bus 0000:04: resource 2 [mem 0x80000000-0x800fffff 64bit pref]
[ 0.345003] pci 0000:00:01.1: D0 power state depends on 0000:00:01.0
[ 0.345887] pci 0000:00:12.2: PME# does not work under D3, disabling it
[ 0.346671] pci 0000:00:13.2: PME# does not work under D3, disabling it
[ 0.347980] pci 0000:00:16.2: PME# does not work under D3, disabling it
[ 0.348353] PCI: CLS 64 bytes, default 64
[ 0.348449] pci 0000:00:00.2: AMD-Vi: Applying erratum 746 workaround
[ 0.348560] pci 0000:00:01.0: Adding to iommu group 0
[ 0.348584] pci 0000:00:01.1: Adding to iommu group 0
[ 0.348616] pci 0000:00:11.0: Adding to iommu group 1
[ 0.348659] pci 0000:00:12.0: Adding to iommu group 2
[ 0.348681] pci 0000:00:12.2: Adding to iommu group 2
[ 0.348723] pci 0000:00:13.0: Adding to iommu group 3
[ 0.348743] pci 0000:00:13.2: Adding to iommu group 3
[ 0.348792] pci 0000:00:14.0: Adding to iommu group 4
[ 0.348817] pci 0000:00:14.2: Adding to iommu group 4
[ 0.348840] pci 0000:00:14.3: Adding to iommu group 4
[ 0.348879] pci 0000:00:14.4: Adding to iommu group 5
[ 0.348903] pci 0000:00:14.5: Adding to iommu group 6
[ 0.348946] pci 0000:00:15.0: Adding to iommu group 7
[ 0.348974] pci 0000:00:15.1: Adding to iommu group 7
[ 0.349000] pci 0000:00:15.2: Adding to iommu group 7
[ 0.349040] pci 0000:00:16.0: Adding to iommu group 8
[ 0.349063] pci 0000:00:16.2: Adding to iommu group 8
[ 0.349128] pci 0000:00:18.0: Adding to iommu group 9
[ 0.349150] pci 0000:00:18.1: Adding to iommu group 9
[ 0.349188] pci 0000:00:18.2: Adding to iommu group 9
[ 0.349210] pci 0000:00:18.3: Adding to iommu group 9
[ 0.349234] pci 0000:00:18.4: Adding to iommu group 9
[ 0.349255] pci 0000:00:18.5: Adding to iommu group 9
[ 0.349269] pci 0000:03:00.0: Adding to iommu group 7
[ 0.349279] pci 0000:04:00.0: Adding to iommu group 7
[ 0.352005] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[ 0.352011] AMD-Vi: Extended features (0x800000853, 0x0): PreF PPR GT IA
[ 0.352020] AMD-Vi: Interrupt remapping enabled
[ 0.352238] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[ 0.352240] software IO TLB: mapped [mem 0x000000005be4d000-0x000000005fe4d000] (64MB)
[ 0.352297] LVT offset 0 assigned for vector 0x400
[ 0.352343] perf: AMD IBS detected (0x000000ff)
[ 0.352351] amd_uncore: 4 amd_nb counters detected
[ 0.356598] workingset: timestamp_bits=37 max_order=20 bucket_order=0
[ 0.356626] zbud: loaded
[ 0.357112] NET: Registered PF_ALG protocol family
[ 0.357118] Key type asymmetric registered
[ 0.357119] Asymmetric key parser 'x509' registered
[ 0.357471] alg: self-tests disabled
[ 0.357567] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 251)
[ 0.357610] io scheduler mq-deadline registered
[ 0.357612] io scheduler kyber registered
[ 0.358782] pcieport 0000:00:15.0: PME: Signaling with IRQ 25
[ 0.358948] pcieport 0000:00:15.1: PME: Signaling with IRQ 26
[ 0.359013] pcieport 0000:00:15.2: enabling device (0000 -> 0003)
[ 0.359228] pcieport 0000:00:15.2: PME: Signaling with IRQ 27
[ 0.359503] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
[ 0.359567] ACPI: button: Power Button [PWRF]
[ 0.359621] ACPI: \_SB_.P000: Found 2 idle states
[ 0.359743] ACPI: \_SB_.P001: Found 2 idle states
[ 0.360670] thermal LNXTHERM:00: registered as thermal_zone0
[ 0.360673] ACPI: thermal: Thermal Zone [TZ00] (15 C)
[ 0.360984] Non-volatile memory driver v1.3
[ 0.361051] AMD-Vi: AMD IOMMUv2 loaded and initialized
[ 0.361079] ACPI: bus type drm_connector registered
[ 0.361292] ahci 0000:00:11.0: version 3.0
[ 0.361580] ahci 0000:00:11.0: AHCI 0001.0300 32 slots 8 ports 6 Gbps 0x40 impl SATA mode
[ 0.361584] ahci 0000:00:11.0: flags: 64bit ncq sntf ilck led clo pio
[ 0.362741] scsi host0: ahci
[ 0.362967] scsi host1: ahci
[ 0.363157] scsi host2: ahci
[ 0.363372] scsi host3: ahci
[ 0.363571] scsi host4: ahci
[ 0.363773] scsi host5: ahci
[ 0.363984] scsi host6: ahci
[ 0.364182] scsi host7: ahci
[ 0.364275] ata1: DUMMY
[ 0.364277] ata2: DUMMY
[ 0.364278] ata3: DUMMY
[ 0.364278] ata4: DUMMY
[ 0.364279] ata5: DUMMY
[ 0.364280] ata6: DUMMY
[ 0.364282] ata7: SATA max UDMA/133 abar m2048@0xf01cc000 port 0xf01cc400 irq 19
[ 0.364284] ata8: DUMMY
[ 0.364564] i8042: PNP: No PS/2 controller found.
[ 0.364565] i8042: Probing ports directly.
[ 0.366981] serio: i8042 KBD port at 0x60,0x64 irq 1
[ 0.366988] serio: i8042 AUX port at 0x60,0x64 irq 12
[ 0.367123] mousedev: PS/2 mouse device common for all mice
[ 0.367188] rtc_cmos 00:01: RTC can wake from S4
[ 0.367487] rtc_cmos 00:01: registered as rtc0
[ 0.367510] rtc_cmos 00:01: setting system clock to 2023-04-21T20:56:07 UTC (1682110567)
[ 0.367553] rtc_cmos 00:01: alarms up to one day, y3k, 114 bytes nvram, hpet irqs
[ 0.367590] device-mapper: uevent: version 1.0.3
[ 0.367665] device-mapper: ioctl: 4.47.0-ioctl (2022-07-28) initialised: dm-devel@redhat.com
[ 0.367827] hid: raw HID events driver (C) Jiri Kosina
[ 0.367913] usbcore: registered new interface driver usbhid
[ 0.367914] usbhid: USB HID core driver
[ 0.368010] Initializing XFRM netlink socket
[ 0.368020] NET: Registered PF_PACKET protocol family
[ 0.368021] x86/pm: family 0x15 cpu detected, MSR saving is needed during suspending.
[ 0.368340] microcode: CPU1: patch_level=0x0600111f
[ 0.368342] microcode: CPU0: patch_level=0x0600111f
[ 0.368353] microcode: Microcode Update Driver: v2.2.
[ 0.368358] IPI shorthand broadcast: enabled
[ 0.368370] AVX version of gcm_enc/dec engaged.
[ 0.368401] AES CTR mode by8 optimization enabled
[ 0.372495] sched_clock: Marking stable (254511749, 117328013)->(374590709, -2750947)
[ 0.372745] registered taskstats version 1
[ 0.372987] zswap: loaded using pool lzo/zbud
[ 0.377461] kmemleak: Kernel memory leak detector initialized (mem pool available: 15679)
[ 0.377464] kmemleak: Automatic memory scanning thread started
[ 0.377465] debug_vm_pgtable: [debug_vm_pgtable ]: Validating architecture page table helpers
[ 0.382250] Key type encrypted registered
[ 0.385395] PM: Magic number: 3:579:953
[ 0.385449] memory memory6: hash matches
[ 0.483426] ata7: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[ 0.483587] ata7.00: ATA-9: SanDisk SDSSDP064G, 2.0.0, max UDMA/133
[ 0.483590] ata7.00: 125045424 sectors, multi 1: LBA48 NCQ (depth 32)
[ 0.483792] ata7.00: configured for UDMA/133
[ 0.484004] scsi 6:0:0:0: Direct-Access ATA SanDisk SDSSDP06 0 PQ: 0 ANSI: 5
[ 0.485006] sd 6:0:0:0: [sda] 125045424 512-byte logical blocks: (64.0 GB/59.6 GiB)
[ 0.485028] sd 6:0:0:0: [sda] Write Protect is off
[ 0.485033] sd 6:0:0:0: [sda] Mode Sense: 00 3a 00 00
[ 0.485062] sd 6:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 0.485103] sd 6:0:0:0: [sda] Preferred minimum I/O size 512 bytes
[ 0.486554] sda: sda1 sda2 sda3
[ 0.487033] sd 6:0:0:0: [sda] Attached SCSI disk
[ 0.508543] EXT4-fs (sda3): mounted filesystem fe29e0dc-6303-4401-987c-8472bc1b9516 with ordered data mode. Quota mode: none.
[ 0.508591] VFS: Mounted root (ext4 filesystem) on device 8:3.
[ 0.510618] devtmpfs: mounted
[ 0.514506] Freeing unused kernel image (initmem) memory: 2908K
[ 0.532141] Write protecting the kernel read-only data: 20480k
[ 0.532425] Freeing unused kernel image (rodata/data gap) memory: 836K
[ 0.569736] x86/mm: Checked W+X mappings: passed, no W+X pages found.
[ 0.569742] rodata_test: all tests were successful
[ 0.569773] Run /sbin/init as init process
[ 0.569775] with arguments:
[ 0.569776] /sbin/init
[ 0.569777] noisapnp
[ 0.569778] with environment:
[ 0.569779] HOME=/
[ 0.569780] TERM=linux
[ 0.569781] BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc3-00046-g8ba643d7e1c7
[ 0.751269] systemd[1]: Inserted module 'autofs4'
[ 0.777767] NET: Registered PF_INET6 protocol family
[ 0.778674] Segment Routing with IPv6
[ 0.778702] In-situ OAM (IOAM) with IPv6
[ 0.804133] systemd[1]: systemd 252.6-1 running in system mode (+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
[ 0.804143] systemd[1]: Detected architecture x86-64.
[ 0.808903] systemd[1]: Hostname set to <kodi>.
[ 1.081619] systemd[1]: Queued start job for default target graphical.target.
[ 1.102404] systemd[1]: Created slice system-getty.slice - Slice /system/getty.
[ 1.103505] systemd[1]: Created slice system-modprobe.slice - Slice /system/modprobe.
[ 1.104345] systemd[1]: Created slice user.slice - User and Session Slice.
[ 1.104531] systemd[1]: Started systemd-ask-password-console.path - Dispatch Password Requests to Console Directory Watch.
[ 1.104655] systemd[1]: Started systemd-ask-password-wall.path - Forward Password Requests to Wall Directory Watch.
[ 1.105082] systemd[1]: Set up automount proc-sys-fs-binfmt_misc.automount - Arbitrary Executable File Formats File System Automount Point.
[ 1.105122] systemd[1]: Reached target cryptsetup.target - Local Encrypted Volumes.
[ 1.105171] systemd[1]: Reached target integritysetup.target - Local Integrity Protected Volumes.
[ 1.105213] systemd[1]: Reached target paths.target - Path Units.
[ 1.105248] systemd[1]: Reached target remote-fs.target - Remote File Systems.
[ 1.105276] systemd[1]: Reached target slices.target - Slice Units.
[ 1.105302] systemd[1]: Reached target swap.target - Swaps.
[ 1.105344] systemd[1]: Reached target veritysetup.target - Local Verity Protected Volumes.
[ 1.107911] systemd[1]: Listening on systemd-coredump.socket - Process Core Dump Socket.
[ 1.108154] systemd[1]: Listening on systemd-fsckd.socket - fsck to fsckd communication Socket.
[ 1.108318] systemd[1]: Listening on systemd-initctl.socket - initctl Compatibility Named Pipe.
[ 1.108624] systemd[1]: Listening on systemd-journald-audit.socket - Journal Audit Socket.
[ 1.108886] systemd[1]: Listening on systemd-journald-dev-log.socket - Journal Socket (/dev/log).
[ 1.109157] systemd[1]: Listening on systemd-journald.socket - Journal Socket.
[ 1.109414] systemd[1]: Listening on systemd-networkd.socket - Network Service Netlink Socket.
[ 1.110215] systemd[1]: Listening on systemd-udevd-control.socket - udev Control Socket.
[ 1.110475] systemd[1]: Listening on systemd-udevd-kernel.socket - udev Kernel Socket.
[ 1.113168] systemd[1]: Mounting dev-hugepages.mount - Huge Pages File System...
[ 1.116585] systemd[1]: Mounting dev-mqueue.mount - POSIX Message Queue File System...
[ 1.121085] systemd[1]: Mounting sys-kernel-debug.mount - Kernel Debug File System...
[ 1.124574] systemd[1]: Mounting sys-kernel-tracing.mount - Kernel Trace File System...
[ 1.128227] systemd[1]: Starting kmod-static-nodes.service - Create List of Static Device Nodes...
[ 1.132312] systemd[1]: Starting modprobe@configfs.service - Load Kernel Module configfs...
[ 1.135610] systemd[1]: Starting modprobe@dm_mod.service - Load Kernel Module dm_mod...
[ 1.138876] systemd[1]: Starting modprobe@drm.service - Load Kernel Module drm...
[ 1.147425] systemd[1]: Starting modprobe@efi_pstore.service - Load Kernel Module efi_pstore...
[ 1.150601] systemd[1]: Starting modprobe@fuse.service - Load Kernel Module fuse...
[ 1.153921] systemd[1]: Starting modprobe@loop.service - Load Kernel Module loop...
[ 1.154024] systemd[1]: systemd-firstboot.service - First Boot Wizard was skipped because of an unmet condition check (ConditionFirstBoot=yes).
[ 1.154112] systemd[1]: systemd-fsck-root.service - File System Check on Root Device was skipped because of an unmet condition check (ConditionPathIsReadWrite=!/).
[ 1.154158] systemd[1]: Reached target local-fs.target - Local File Systems.
[ 1.154259] systemd[1]: apparmor.service - Load AppArmor profiles was skipped because of an unmet condition check (ConditionSecurity=apparmor).
[ 1.167112] fuse: init (API version 7.38)
[ 1.167255] loop: module loaded
[ 1.168586] systemd[1]: Starting systemd-binfmt.service - Set Up Additional Binary Formats...
[ 1.174035] systemd[1]: Starting systemd-journald.service - Journal Service...
[ 1.177353] systemd[1]: Starting systemd-random-seed.service - Load/Save Random Seed...
[ 1.180848] systemd[1]: Starting systemd-sysctl.service - Apply Kernel Variables...
[ 1.195025] systemd[1]: Starting systemd-sysusers.service - Create System Users...
[ 1.201956] systemd[1]: Starting systemd-udev-trigger.service - Coldplug All udev Devices...
[ 1.217695] systemd[1]: Mounted dev-hugepages.mount - Huge Pages File System.
[ 1.218409] systemd[1]: Mounted dev-mqueue.mount - POSIX Message Queue File System.
[ 1.218683] systemd[1]: Mounted sys-kernel-debug.mount - Kernel Debug File System.
[ 1.218957] systemd[1]: Mounted sys-kernel-tracing.mount - Kernel Trace File System.
[ 1.225546] systemd[1]: Finished kmod-static-nodes.service - Create List of Static Device Nodes.
[ 1.226534] systemd[1]: modprobe@configfs.service: Deactivated successfully.
[ 1.226893] systemd[1]: Finished modprobe@configfs.service - Load Kernel Module configfs.
[ 1.227645] systemd[1]: modprobe@dm_mod.service: Deactivated successfully.
[ 1.228209] systemd[1]: Finished modprobe@dm_mod.service - Load Kernel Module dm_mod.
[ 1.230383] systemd[1]: modprobe@drm.service: Deactivated successfully.
[ 1.230725] systemd[1]: Finished modprobe@drm.service - Load Kernel Module drm.
[ 1.231459] systemd[1]: modprobe@efi_pstore.service: Deactivated successfully.
[ 1.231774] systemd[1]: Finished modprobe@efi_pstore.service - Load Kernel Module efi_pstore.
[ 1.232474] systemd[1]: modprobe@fuse.service: Deactivated successfully.
[ 1.232795] systemd[1]: Finished modprobe@fuse.service - Load Kernel Module fuse.
[ 1.234105] systemd[1]: modprobe@loop.service: Deactivated successfully.
[ 1.234462] systemd[1]: Finished modprobe@loop.service - Load Kernel Module loop.
[ 1.235099] systemd[1]: proc-sys-fs-binfmt_misc.automount: Got automount request for /proc/sys/fs/binfmt_misc, triggered by 140 (systemd-binfmt)
[ 1.261703] systemd[1]: Mounting proc-sys-fs-binfmt_misc.mount - Arbitrary Executable File Formats File System...
[ 1.270160] systemd[1]: Mounting sys-fs-fuse-connections.mount - FUSE Control File System...
[ 1.273274] systemd[1]: Mounting sys-kernel-config.mount - Kernel Configuration File System...
[ 1.273402] systemd[1]: systemd-pstore.service - Platform Persistent Storage Archival was skipped because of an unmet condition check (ConditionDirectoryNotEmpty=/sys/fs/pstore).
[ 1.273574] systemd[1]: systemd-repart.service - Repartition Root Disk was skipped because no trigger condition checks were met.
[ 1.290481] systemd[1]: Finished systemd-sysctl.service - Apply Kernel Variables.
[ 1.294159] systemd[1]: Finished systemd-sysusers.service - Create System Users.
[ 1.294555] systemd[1]: Mounted sys-fs-fuse-connections.mount - FUSE Control File System.
[ 1.303356] systemd[1]: Starting systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev...
[ 1.304817] systemd[1]: Mounted sys-kernel-config.mount - Kernel Configuration File System.
[ 1.320909] systemd[1]: Mounted proc-sys-fs-binfmt_misc.mount - Arbitrary Executable File Formats File System.
[ 1.323560] systemd[1]: Finished systemd-binfmt.service - Set Up Additional Binary Formats.
[ 1.369356] tsc: Refined TSC clocksource calibration: 3900.224 MHz
[ 1.369367] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x70705a6472c, max_idle_ns: 881590586812 ns
[ 1.370036] systemd[1]: Finished systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev.
[ 1.373207] clocksource: Switched to clocksource tsc
[ 1.389831] systemd[1]: Starting systemd-udevd.service - Rule-based Manager for Device Events and Files...
[ 1.427455] systemd[1]: Started systemd-journald.service - Journal Service.
[ 1.472737] systemd-journald[142]: Received client request to flush runtime journal.
[ 1.742223] sd 6:0:0:0: Attached scsi generic sg0 type 0
[ 1.866042] acpi_cpufreq: overriding BIOS provided _PSD data
[ 2.005347] random: crng init done
[ 2.137832] QUIRK: Enable AMD PLL fix
[ 2.137832] QUIRK: Enable AMD PLL fix
[ 2.137887] ehci-pci 0000:00:13.2: EHCI Host Controller
[ 2.137922] ehci-pci 0000:00:13.2: new USB bus registered, assigned bus number 1
[ 2.137936] ehci-pci 0000:00:13.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
[ 2.137945] ehci-pci 0000:00:13.2: debug port 1
[ 2.138141] ehci-pci 0000:00:13.2: irq 17, io mem 0xf01ce000
[ 2.139491] piix4_smbus 0000:00:14.0: SMBus Host Controller at 0xb00, revision 0
[ 2.139505] piix4_smbus 0000:00:14.0: Using register 0x2e for SMBus port selection
[ 2.140107] piix4_smbus 0000:00:14.0: Auxiliary SMBus Host Controller at 0xb20
[ 2.154632] ehci-pci 0000:00:13.2: USB 2.0 started, EHCI 1.00
[ 2.155284] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 2.155291] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.155294] usb usb1: Product: EHCI Host Controller
[ 2.155296] usb usb1: Manufacturer: Linux 6.3.0-rc3-00046-g8ba643d7e1c7 ehci_hcd
[ 2.155299] usb usb1: SerialNumber: 0000:00:13.2
[ 2.174521] hub 1-0:1.0: USB hub found
[ 2.174560] hub 1-0:1.0: 5 ports detected
[ 2.175582] ehci-pci 0000:00:12.2: EHCI Host Controller
[ 2.175610] ehci-pci 0000:00:12.2: new USB bus registered, assigned bus number 2
[ 2.175623] ehci-pci 0000:00:12.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
[ 2.175632] ehci-pci 0000:00:12.2: debug port 1
[ 2.175789] ehci-pci 0000:00:12.2: irq 17, io mem 0xf01cd000
[ 2.189373] ehci-pci 0000:00:12.2: USB 2.0 started, EHCI 1.00
[ 2.189720] usb usb2: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 2.189725] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.189727] usb usb2: Product: EHCI Host Controller
[ 2.189730] usb usb2: Manufacturer: Linux 6.3.0-rc3-00046-g8ba643d7e1c7 ehci_hcd
[ 2.189732] usb usb2: SerialNumber: 0000:00:12.2
[ 2.190283] hub 2-0:1.0: USB hub found
[ 2.190323] hub 2-0:1.0: 5 ports detected
[ 2.191801] ehci-pci 0000:00:16.2: EHCI Host Controller
[ 2.191831] ehci-pci 0000:00:16.2: new USB bus registered, assigned bus number 3
[ 2.191845] ehci-pci 0000:00:16.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
[ 2.191855] ehci-pci 0000:00:16.2: debug port 1
[ 2.192018] ehci-pci 0000:00:16.2: irq 17, io mem 0xf01cf000
[ 2.205373] ehci-pci 0000:00:16.2: USB 2.0 started, EHCI 1.00
[ 2.205651] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 2.205655] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.205658] usb usb3: Product: EHCI Host Controller
[ 2.205660] usb usb3: Manufacturer: Linux 6.3.0-rc3-00046-g8ba643d7e1c7 ehci_hcd
[ 2.205662] usb usb3: SerialNumber: 0000:00:16.2
[ 2.206200] hub 3-0:1.0: USB hub found
[ 2.206236] hub 3-0:1.0: 4 ports detected
[ 2.221053] ohci-pci 0000:00:12.0: OHCI PCI host controller
[ 2.221085] ohci-pci 0000:00:12.0: new USB bus registered, assigned bus number 4
[ 2.221264] ohci-pci 0000:00:12.0: irq 18, io mem 0xf01c8000
[ 2.237796] r8169 0000:04:00.0: enabling device (0000 -> 0003)
[ 2.261308] xhci_hcd 0000:03:00.0: xHCI Host Controller
[ 2.261394] xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 5
[ 2.272125] r8169 0000:04:00.0 eth0: RTL8168f/8111f, 08:60:6e:74:7a:51, XID 480, IRQ 28
[ 2.272143] r8169 0000:04:00.0 eth0: jumbo features [frames: 9194 bytes, tx checksumming: ko]
[ 2.282633] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 2.282644] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.282650] usb usb4: Product: OHCI PCI host controller
[ 2.282655] usb usb4: Manufacturer: Linux 6.3.0-rc3-00046-g8ba643d7e1c7 ohci_hcd
[ 2.282659] usb usb4: SerialNumber: 0000:00:12.0
[ 2.283372] hub 4-0:1.0: USB hub found
[ 2.283431] hub 4-0:1.0: 5 ports detected
[ 2.297576] ohci-pci 0000:00:13.0: OHCI PCI host controller
[ 2.297609] ohci-pci 0000:00:13.0: new USB bus registered, assigned bus number 6
[ 2.297727] ohci-pci 0000:00:13.0: irq 18, io mem 0xf01c9000
[ 2.320955] xhci_hcd 0000:03:00.0: hcc params 0x0200f180 hci version 0x96 quirks 0x0000000000080010
[ 2.322099] xhci_hcd 0000:03:00.0: xHCI Host Controller
[ 2.322121] xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 7
[ 2.322136] xhci_hcd 0000:03:00.0: Host supports USB 3.0 SuperSpeed
[ 2.324029] usb usb5: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 2.324036] usb usb5: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.324039] usb usb5: Product: xHCI Host Controller
[ 2.324041] usb usb5: Manufacturer: Linux 6.3.0-rc3-00046-g8ba643d7e1c7 xhci-hcd
[ 2.324043] usb usb5: SerialNumber: 0000:03:00.0
[ 2.324687] hub 5-0:1.0: USB hub found
[ 2.324727] hub 5-0:1.0: 2 ports detected
[ 2.325467] usb usb7: We don't know the algorithms for LPM for this host, disabling LPM.
[ 2.325624] usb usb7: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 6.03
[ 2.325628] usb usb7: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.325630] usb usb7: Product: xHCI Host Controller
[ 2.325632] usb usb7: Manufacturer: Linux 6.3.0-rc3-00046-g8ba643d7e1c7 xhci-hcd
[ 2.325634] usb usb7: SerialNumber: 0000:03:00.0
[ 2.326195] hub 7-0:1.0: USB hub found
[ 2.327034] hub 7-0:1.0: 2 ports detected
[ 2.349205] snd_hda_intel 0000:00:01.1: Force to non-snoop mode
[ 2.366399] usb usb6: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 2.366406] usb usb6: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.366409] usb usb6: Product: OHCI PCI host controller
[ 2.366411] usb usb6: Manufacturer: Linux 6.3.0-rc3-00046-g8ba643d7e1c7 ohci_hcd
[ 2.366413] usb usb6: SerialNumber: 0000:00:13.0
[ 2.367210] hub 6-0:1.0: USB hub found
[ 2.367303] hub 6-0:1.0: 5 ports detected
[ 2.391447] r8169 0000:04:00.0 enp4s0: renamed from eth0
[ 2.420037] ohci-pci 0000:00:14.5: OHCI PCI host controller
[ 2.420070] ohci-pci 0000:00:14.5: new USB bus registered, assigned bus number 8
[ 2.421556] ohci-pci 0000:00:14.5: irq 18, io mem 0xf01ca000
[ 2.445281] r8169 0000:04:00.0: Direct firmware load for rtl_nic/rtl8168f-1.fw failed with error -2
[ 2.445292] r8169 0000:04:00.0: Unable to load firmware rtl_nic/rtl8168f-1.fw (-2)
[ 2.445868] RTL8211E Gigabit Ethernet r8169-0-400:00: attached PHY driver (mii_bus:phy_addr=r8169-0-400:00, irq=MAC)
[ 2.481250] input: HDA ATI HDMI HDMI/DP,pcm=3 as /devices/pci0000:00/0000:00:01.1/sound/card0/input1
[ 2.481880] input: HDA ATI HDMI HDMI/DP,pcm=7 as /devices/pci0000:00/0000:00:01.1/sound/card0/input2
[ 2.488733] usb usb8: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 2.488740] usb usb8: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.488743] usb usb8: Product: OHCI PCI host controller
[ 2.488745] usb usb8: Manufacturer: Linux 6.3.0-rc3-00046-g8ba643d7e1c7 ohci_hcd
[ 2.488747] usb usb8: SerialNumber: 0000:00:14.5
[ 2.492816] snd_hda_codec_realtek hdaudioC1D0: ALC892: SKU not ready 0x00000100
[ 2.492902] hub 8-0:1.0: USB hub found
[ 2.492953] hub 8-0:1.0: 2 ports detected
[ 2.493612] snd_hda_codec_realtek hdaudioC1D0: autoconfig for ALC892: line_outs=4 (0x14/0x16/0x15/0x17/0x0) type:line
[ 2.493620] snd_hda_codec_realtek hdaudioC1D0: speaker_outs=0 (0x0/0x0/0x0/0x0/0x0)
[ 2.493623] snd_hda_codec_realtek hdaudioC1D0: hp_outs=1 (0x1b/0x0/0x0/0x0/0x0)
[ 2.493626] snd_hda_codec_realtek hdaudioC1D0: mono: mono_out=0x0
[ 2.493628] snd_hda_codec_realtek hdaudioC1D0: dig-out=0x1e/0x0
[ 2.493629] snd_hda_codec_realtek hdaudioC1D0: inputs:
[ 2.493631] snd_hda_codec_realtek hdaudioC1D0: Rear Mic=0x18
[ 2.493633] snd_hda_codec_realtek hdaudioC1D0: Front Mic=0x19
[ 2.493635] snd_hda_codec_realtek hdaudioC1D0: Line=0x1a
[ 2.493636] snd_hda_codec_realtek hdaudioC1D0: CD=0x1c
[ 2.495003] ohci-pci 0000:00:16.0: OHCI PCI host controller
[ 2.495036] ohci-pci 0000:00:16.0: new USB bus registered, assigned bus number 9
[ 2.495172] ohci-pci 0000:00:16.0: irq 18, io mem 0xf01cb000
[ 2.510556] [drm] radeon kernel modesetting enabled.
[ 2.513510] [drm] initializing kernel modesetting (ARUBA 0x1002:0x9996 0x1002:0x9996 0x00).
[ 2.513588] ATOM BIOS: 113
[ 2.513893] radeon 0000:00:01.0: VRAM: 512M 0x0000000000000000 - 0x000000001FFFFFFF (512M used)
[ 2.513898] radeon 0000:00:01.0: GTT: 1024M 0x0000000020000000 - 0x000000005FFFFFFF
[ 2.513911] [drm] Detected VRAM RAM=512M, BAR=256M
[ 2.513913] [drm] RAM width 64bits DDR
[ 2.514805] [drm] radeon: 512M of VRAM memory ready
[ 2.514815] [drm] radeon: 1024M of GTT memory ready.
[ 2.514888] [drm] Loading ARUBA Microcode
[ 2.517185] r8169 0000:04:00.0 enp4s0: Link is Down
[ 2.519927] input: HD-Audio Generic Rear Mic as /devices/pci0000:00/0000:00:14.2/sound/card1/input3
[ 2.520307] input: HD-Audio Generic Front Mic as /devices/pci0000:00/0000:00:14.2/sound/card1/input4
[ 2.520824] input: HD-Audio Generic Line as /devices/pci0000:00/0000:00:14.2/sound/card1/input5
[ 2.521180] input: HD-Audio Generic Line Out Front as /devices/pci0000:00/0000:00:14.2/sound/card1/input6
[ 2.521594] input: HD-Audio Generic Line Out Surround as /devices/pci0000:00/0000:00:14.2/sound/card1/input7
[ 2.522160] input: HD-Audio Generic Line Out CLFE as /devices/pci0000:00/0000:00:14.2/sound/card1/input8
[ 2.522536] input: HD-Audio Generic Line Out Side as /devices/pci0000:00/0000:00:14.2/sound/card1/input9
[ 2.522888] input: HD-Audio Generic Front Headphone as /devices/pci0000:00/0000:00:14.2/sound/card1/input10
[ 2.532990] [drm] Internal thermal controller without fan control
[ 2.558842] usb usb9: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 2.558849] usb usb9: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.558852] usb usb9: Product: OHCI PCI host controller
[ 2.558855] usb usb9: Manufacturer: Linux 6.3.0-rc3-00046-g8ba643d7e1c7 ohci_hcd
[ 2.558857] usb usb9: SerialNumber: 0000:00:16.0
[ 2.558984] [drm] radeon: dpm initialized
[ 2.564667] [drm] Found VCE firmware/feedback version 50.0.1 / 17!
[ 2.564728] [drm] GART: num cpu pages 262144, num gpu pages 262144
[ 2.594051] [drm] PCIE GART of 1024M enabled (table at 0x00000000001D6000).
[ 2.594314] radeon 0000:00:01.0: WB enabled
[ 2.594318] radeon 0000:00:01.0: fence driver on ring 0 use gpu addr 0x0000000020000c00
[ 2.594697] radeon 0000:00:01.0: fence driver on ring 5 use gpu addr 0x0000000000075a18
[ 2.595711] hub 9-0:1.0: USB hub found
[ 2.595753] hub 9-0:1.0: 4 ports detected
[ 2.618866] radeon 0000:00:01.0: fence driver on ring 6 use gpu addr 0x0000000020000c18
[ 2.618874] radeon 0000:00:01.0: fence driver on ring 7 use gpu addr 0x0000000020000c1c
[ 2.618877] radeon 0000:00:01.0: fence driver on ring 1 use gpu addr 0x0000000020000c04
[ 2.618879] radeon 0000:00:01.0: fence driver on ring 2 use gpu addr 0x0000000020000c08
[ 2.618881] radeon 0000:00:01.0: fence driver on ring 3 use gpu addr 0x0000000020000c0c
[ 2.618883] radeon 0000:00:01.0: fence driver on ring 4 use gpu addr 0x0000000020000c10
[ 2.647823] radeon 0000:00:01.0: radeon: MSI limited to 32-bit
[ 2.648048] radeon 0000:00:01.0: radeon: using MSI.
[ 2.648125] [drm] radeon: irq initialized.
[ 2.667958] [drm] ring test on 0 succeeded in 3 usecs
[ 2.667970] [drm] ring test on 3 succeeded in 4 usecs
[ 2.667977] [drm] ring test on 4 succeeded in 4 usecs
[ 2.713784] [drm] ring test on 5 succeeded in 2 usecs
[ 2.733666] [drm] UVD initialized successfully.
[ 2.741341] usb 4-1: new low-speed USB device number 2 using ohci-pci
[ 2.843040] [drm] ring test on 6 succeeded in 18 usecs
[ 2.843052] [drm] ring test on 7 succeeded in 3 usecs
[ 2.843053] [drm] VCE initialized successfully.
[ 2.843202] snd_hda_intel 0000:00:01.1: bound 0000:00:01.0 (ops radeon_audio_component_bind_ops [radeon])
[ 2.843370] [drm] ib test on ring 0 succeeded in 0 usecs
[ 2.843424] [drm] ib test on ring 3 succeeded in 0 usecs
[ 2.843474] [drm] ib test on ring 4 succeeded in 0 usecs
[ 2.942595] usb 4-1: New USB device found, idVendor=413c, idProduct=2106, bcdDevice= 1.01
[ 2.942607] usb 4-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 2.942611] usb 4-1: Product: Dell QuietKey Keyboard
[ 2.942615] usb 4-1: Manufacturer: DELL
[ 2.951006] input: DELL Dell QuietKey Keyboard as /devices/pci0000:00/0000:00:12.0/usb4/4-1/4-1:1.0/0003:413C:2106.0001/input/input11
[ 3.010775] hid-generic 0003:413C:2106.0001: input,hidraw0: USB HID v1.10 Keyboard [DELL Dell QuietKey Keyboard] on usb-0000:00:12.0-1/input0
[ 3.385454] [drm] ib test on ring 5 succeeded
[ 3.405349] usb 4-2: new low-speed USB device number 3 using ohci-pci
[ 3.600597] usb 4-2: New USB device found, idVendor=046d, idProduct=c016, bcdDevice= 3.40
[ 3.600609] usb 4-2: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 3.600613] usb 4-2: Product: Optical USB Mouse
[ 3.600617] usb 4-2: Manufacturer: Logitech
[ 3.610963] input: Logitech Optical USB Mouse as /devices/pci0000:00/0000:00:12.0/usb4/4-2/4-2:1.0/0003:046D:C016.0002/input/input12
[ 3.611917] hid-generic 0003:046D:C016.0002: input,hidraw1: USB HID v1.10 Mouse [Logitech Optical USB Mouse] on usb-0000:00:12.0-2/input0
[ 3.929454] [drm] ib test on ring 6 succeeded
[ 4.441455] [drm] ib test on ring 7 succeeded
[ 4.446983] [drm] Radeon Display Connectors
[ 4.446989] [drm] Connector 0:
[ 4.446992] [drm] DP-1
[ 4.446995] [drm] HPD1
[ 4.446997] [drm] DDC: 0x6530 0x6530 0x6534 0x6534 0x6538 0x6538 0x653c 0x653c
[ 4.447003] [drm] Encoders:
[ 4.447005] [drm] DFP1: INTERNAL_UNIPHY2
[ 4.447007] [drm] Connector 1:
[ 4.447009] [drm] VGA-1
[ 4.447011] [drm] HPD2
[ 4.447013] [drm] DDC: 0x6540 0x6540 0x6544 0x6544 0x6548 0x6548 0x654c 0x654c
[ 4.447018] [drm] Encoders:
[ 4.447020] [drm] CRT1: INTERNAL_UNIPHY2
[ 4.447022] [drm] CRT1: NUTMEG
[ 4.447024] [drm] Connector 2:
[ 4.447026] [drm] HDMI-A-1
[ 4.447028] [drm] HPD3
[ 4.447030] [drm] DDC: 0x6550 0x6550 0x6554 0x6554 0x6558 0x6558 0x655c 0x655c
[ 4.447035] [drm] Encoders:
[ 4.447036] [drm] DFP2: INTERNAL_UNIPHY
[ 4.716171] [drm] fb mappable at 0xE03E9000
[ 4.716179] [drm] vram apper at 0xE0000000
[ 4.716181] [drm] size 5242880
[ 4.716183] [drm] fb depth is 24
[ 4.716185] [drm] pitch is 5120
[ 4.716671] fbcon: radeondrmfb (fb0) is primary device
[ 4.908197] Console: switching to colour frame buffer device 160x64
[ 4.909974] radeon 0000:00:01.0: [drm] fb0: radeondrmfb frame buffer device
[ 4.933682] [drm] Initialized radeon 2.50.0 20080528 for 0000:00:01.0 on minor 0
[ 5.125043] r8169 0000:04:00.0 enp4s0: Link is Up - 1Gbps/Full - flow control rx/tx
[ 5.125059] IPv6: ADDRCONF(NETDEV_CHANGE): enp4s0: link becomes ready
[ 7.669093] [drm] amdgpu kernel modesetting enabled.
[ 11.216542] memfd_create() without MFD_EXEC nor MFD_NOEXEC_SEAL, pid=272 'systemd'
[-- Attachment #3: Type: text/plain, Size: 161 bytes --]
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-24 18:46 ` Paul Menzel
0 siblings, 0 replies; 236+ messages in thread
From: Paul Menzel @ 2023-04-24 18:46 UTC (permalink / raw)
To: Thomas Gleixner
Cc: Sean Christopherson, Andrew Cooper, linux-kernel, x86,
David Woodhouse, Brian Gerst, Arjan van de Veen, Paolo Bonzini,
Paul McKenney, Tom Lendacky, Oleksandr Natalenko,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
[-- Attachment #1: Type: text/plain, Size: 722 bytes --]
Dear Thomas,
Am 20.04.23 um 21:10 schrieb Thomas Gleixner:
> On Thu, Apr 20 2023 at 18:47, Paul Menzel wrote:
>> Am 20.04.23 um 17:57 schrieb Thomas Gleixner:
>> I quickly applied it on top of your branch, but I am getting:
>
> As I said it was untested. I was traveling and did not have access to a
> machine to even build it completely. Fixed up and tested version below.
Sorry, if it sounded like a complaint. I just wanted to give a quick
feedback.
[…]
I tested your new version even on Friday, and it worked fine – no ten
seconds delay. Please find the messages attached.
Thank you all for your great work.
Kind regards,
Paul
PS: I am going to try to test your updated branch at the end of the week.
[-- Attachment #2: kodi-linux-6.3-rc3-smp-tglx-with-apic-fix.txt --]
[-- Type: text/plain, Size: 57511 bytes --]
[ 0.000000] Linux version 6.3.0-rc3-00046-g8ba643d7e1c7 (root@bf16f3646a84) (gcc (Debian 11.2.0-12) 11.2.0, GNU ld (GNU Binutils for Debian) 2.40) #452 SMP PREEMPT_DYNAMIC Thu Apr 20 20:15:01 UTC 2023
[ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc3-00046-g8ba643d7e1c7 root=/dev/sda3 rw quiet noisapnp cryptomgr.notests ipv6.disable_ipv6=1 selinux=0
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[ 0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256
[ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format.
[ 0.000000] signal: max sigframe size: 1776
[ 0.000000] BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
[ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005fe4cfff] usable
[ 0.000000] BIOS-e820: [mem 0x000000005fe4d000-0x000000007fffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000f8000000-0x00000000fbffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000fec10000-0x00000000fec10fff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000017effffff] usable
[ 0.000000] NX (Execute Disable) protection: active
[ 0.000000] SMBIOS 3.0.0 present.
[ 0.000000] DMI: ASUS F2A85-M_PRO/F2A85-M_PRO, BIOS 4.18-9-gb640ed51b2 04/17/2023
[ 0.000000] tsc: Fast TSC calibration using PIT
[ 0.000000] tsc: Detected 3900.440 MHz processor
[ 0.000756] e820: update [mem 0x00000000-0x00000fff] usable ==> reserved
[ 0.000759] e820: remove [mem 0x000a0000-0x000fffff] usable
[ 0.000763] last_pfn = 0x17f000 max_arch_pfn = 0x400000000
[ 0.000768] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT
[ 0.000938] last_pfn = 0x5fe4d max_arch_pfn = 0x400000000
[ 0.004000] Using GB pages for direct mapping
[ 0.004000] ACPI: Early table checksum verification disabled
[ 0.004000] ACPI: RSDP 0x00000000000F6830 000024 (v02 COREv4)
[ 0.004000] ACPI: XSDT 0x000000005FE5A0E0 000074 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: FACP 0x000000005FE5BBC0 000114 (v06 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: DSDT 0x000000005FE5A280 00193A (v02 COREv4 COREBOOT 00010001 INTL 20200925)
[ 0.004000] ACPI: FACS 0x000000005FE5A240 000040
[ 0.004000] ACPI: FACS 0x000000005FE5A240 000040
[ 0.004000] ACPI: SSDT 0x000000005FE5BCE0 00008A (v02 COREv4 COREBOOT 0000002A CORE 20200925)
[ 0.004000] ACPI: MCFG 0x000000005FE5BD70 00003C (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: APIC 0x000000005FE5BDB0 000062 (v03 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: HPET 0x000000005FE5BE20 000038 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: HEST 0x000000005FE5BE60 0001D0 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: IVRS 0x000000005FE5C030 000070 (v02 AMD AMDIOMMU 00000001 AMD 00000000)
[ 0.004000] ACPI: SSDT 0x000000005FE5C0A0 00051F (v02 AMD ALIB 00000001 MSFT 04000000)
[ 0.004000] ACPI: SSDT 0x000000005FE5C5C0 0006B2 (v01 AMD POWERNOW 00000001 AMD 00000001)
[ 0.004000] ACPI: VFCT 0x000000005FE5CC80 00F269 (v01 COREv4 COREBOOT 00000000 CORE 20200925)
[ 0.004000] ACPI: Reserving FACP table memory at [mem 0x5fe5bbc0-0x5fe5bcd3]
[ 0.004000] ACPI: Reserving DSDT table memory at [mem 0x5fe5a280-0x5fe5bbb9]
[ 0.004000] ACPI: Reserving FACS table memory at [mem 0x5fe5a240-0x5fe5a27f]
[ 0.004000] ACPI: Reserving FACS table memory at [mem 0x5fe5a240-0x5fe5a27f]
[ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe5bce0-0x5fe5bd69]
[ 0.004000] ACPI: Reserving MCFG table memory at [mem 0x5fe5bd70-0x5fe5bdab]
[ 0.004000] ACPI: Reserving APIC table memory at [mem 0x5fe5bdb0-0x5fe5be11]
[ 0.004000] ACPI: Reserving HPET table memory at [mem 0x5fe5be20-0x5fe5be57]
[ 0.004000] ACPI: Reserving HEST table memory at [mem 0x5fe5be60-0x5fe5c02f]
[ 0.004000] ACPI: Reserving IVRS table memory at [mem 0x5fe5c030-0x5fe5c09f]
[ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe5c0a0-0x5fe5c5be]
[ 0.004000] ACPI: Reserving SSDT table memory at [mem 0x5fe5c5c0-0x5fe5cc71]
[ 0.004000] ACPI: Reserving VFCT table memory at [mem 0x5fe5cc80-0x5fe6bee8]
[ 0.004000] No NUMA configuration found
[ 0.004000] Faking a node at [mem 0x0000000000000000-0x000000017effffff]
[ 0.004000] NODE_DATA(0) allocated [mem 0x17efe9000-0x17effffff]
[ 0.004000] Zone ranges:
[ 0.004000] DMA [mem 0x0000000000001000-0x0000000000ffffff]
[ 0.004000] DMA32 [mem 0x0000000001000000-0x00000000ffffffff]
[ 0.004000] Normal [mem 0x0000000100000000-0x000000017effffff]
[ 0.004000] Device empty
[ 0.004000] Movable zone start for each node
[ 0.004000] Early memory node ranges
[ 0.004000] node 0: [mem 0x0000000000001000-0x000000000009efff]
[ 0.004000] node 0: [mem 0x0000000000100000-0x000000005fe4cfff]
[ 0.004000] node 0: [mem 0x0000000100000000-0x000000017effffff]
[ 0.004000] Initmem setup node 0 [mem 0x0000000000001000-0x000000017effffff]
[ 0.004000] On node 0, zone DMA: 1 pages in unavailable ranges
[ 0.004000] On node 0, zone DMA: 97 pages in unavailable ranges
[ 0.004000] On node 0, zone Normal: 435 pages in unavailable ranges
[ 0.004000] On node 0, zone Normal: 4096 pages in unavailable ranges
[ 0.004000] ACPI: PM-Timer IO Port: 0x818
[ 0.004000] ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1])
[ 0.004000] IOAPIC[0]: apic_id 4, version 33, address 0xfec00000, GSI 0-23
[ 0.004000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[ 0.004000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 low level)
[ 0.004000] ACPI: Using ACPI (MADT) for SMP configuration information
[ 0.004000] ACPI: HPET id: 0x10228210 base: 0xfed00000
[ 0.004000] smpboot: Allowing 2 CPUs, 0 hotplug CPUs
[ 0.004000] [mem 0x80000000-0xf7ffffff] available for PCI devices
[ 0.004000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns
[ 0.004000] setup_percpu: NR_CPUS:64 nr_cpumask_bits:2 nr_cpu_ids:2 nr_node_ids:1
[ 0.004000] percpu: Embedded 55 pages/cpu s188392 r8192 d28696 u1048576
[ 0.004000] pcpu-alloc: s188392 r8192 d28696 u1048576 alloc=1*2097152
[ 0.004000] pcpu-alloc: [0] 0 1
[ 0.004000] Fallback order for Node 0: 0
[ 0.004000] Built 1 zonelists, mobility grouping on. Total pages: 898451
[ 0.004000] Policy zone: Normal
[ 0.004000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc3-00046-g8ba643d7e1c7 root=/dev/sda3 rw quiet noisapnp cryptomgr.notests ipv6.disable_ipv6=1 selinux=0
[ 0.004000] Unknown kernel command line parameters "noisapnp BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc3-00046-g8ba643d7e1c7", will be passed to user space.
[ 0.004000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[ 0.004000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[ 0.004000] mem auto-init: stack:off, heap alloc:off, heap free:off
[ 0.004000] stackdepot: allocating hash table via alloc_large_system_hash
[ 0.004000] stackdepot hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[ 0.004000] software IO TLB: area num 2.
[ 0.004000] Memory: 3477168K/3651500K available (14336K kernel code, 2340K rwdata, 5308K rodata, 2908K init, 11064K bss, 174072K reserved, 0K cma-reserved)
[ 0.004000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
[ 0.004000] ftrace: allocating 38652 entries in 151 pages
[ 0.004000] ftrace: allocated 151 pages with 5 groups
[ 0.004000] Dynamic Preempt: full
[ 0.004000] rcu: Preemptible hierarchical RCU implementation.
[ 0.004000] rcu: RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=2.
[ 0.004000] Trampoline variant of Tasks RCU enabled.
[ 0.004000] Rude variant of Tasks RCU enabled.
[ 0.004000] Tracing variant of Tasks RCU enabled.
[ 0.004000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[ 0.004000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2
[ 0.004000] NR_IRQS: 4352, nr_irqs: 440, preallocated irqs: 16
[ 0.004000] rcu: srcu_init: Setting srcu_struct sizes based on contention.
[ 0.004000] spurious 8259A interrupt: IRQ7.
[ 0.004000] Console: colour VGA+ 80x25
[ 0.004000] printk: console [tty0] enabled
[ 0.004000] ACPI: Core revision 20221020
[ 0.004000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 133484873504 ns
[ 0.004000] APIC: Switch to symmetric I/O mode setup
[ 0.004000] AMD-Vi: Using global IVHD EFR:0x0, EFR2:0x0
[ 0.004000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[ 0.004000] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x7071f4ed18f, max_idle_ns: 881590420850 ns
[ 0.145332] Calibrating delay loop (skipped), value calculated using timer frequency.. 7800.88 BogoMIPS (lpj=15601760)
[ 0.145336] pid_max: default: 32768 minimum: 301
[ 0.145430] LSM: initializing lsm=capability
[ 0.145526] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[ 0.145542] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[ 0.145940] Last level iTLB entries: 4KB 512, 2MB 1024, 4MB 512
[ 0.145943] Last level dTLB entries: 4KB 1024, 2MB 1024, 4MB 512, 1GB 0
[ 0.145947] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization
[ 0.145950] Spectre V2 : Mitigation: Retpolines
[ 0.145951] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch
[ 0.145952] Spectre V2 : Spectre v2 / SpectreRSB : Filling RSB on VMEXIT
[ 0.145952] Spectre V2 : Enabling Speculation Barrier for firmware calls
[ 0.145953] RETBleed: Mitigation: untrained return thunk
[ 0.145955] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier
[ 0.145957] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl
[ 0.150469] Freeing SMP alternatives memory: 32K
[ 0.258608] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD Graphics (family: 0x15, model: 0x13, stepping: 0x1)
[ 0.258845] cblist_init_generic: Setting adjustable number of callback queues.
[ 0.258846] cblist_init_generic: Setting shift to 1 and lim to 1.
[ 0.258878] cblist_init_generic: Setting shift to 1 and lim to 1.
[ 0.258905] cblist_init_generic: Setting shift to 1 and lim to 1.
[ 0.258933] Performance Events: Fam15h core perfctr, AMD PMU driver.
[ 0.258955] ... version: 0
[ 0.258956] ... bit width: 48
[ 0.258957] ... generic registers: 6
[ 0.258958] ... value mask: 0000ffffffffffff
[ 0.258959] ... max period: 00007fffffffffff
[ 0.258960] ... fixed-purpose events: 0
[ 0.258961] ... event mask: 000000000000003f
[ 0.259083] rcu: Hierarchical SRCU implementation.
[ 0.259084] rcu: Max phase no-delay instances is 1000.
[ 0.259676] NMI watchdog: Enabled. Permanently consumes one hw-PMU counter.
[ 0.259750] smp: Bringing up secondary CPUs ...
[ 0.259952] x86: Booting SMP configuration:
[ 0.259953] .... node #0, CPUs: #1
[ 0.259958] smpboot: Kicking AP alive: 17
[ 0.260088] smp: Brought up 1 node, 2 CPUs
[ 0.260088] smpboot: Max logical packages: 1
[ 0.260088] smpboot: Total of 2 processors activated (15601.76 BogoMIPS)
[ 0.261513] devtmpfs: initialized
[ 0.261513] x86/mm: Memory block size: 128MB
[ 0.262387] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[ 0.262387] futex hash table entries: 512 (order: 3, 32768 bytes, linear)
[ 0.262387] pinctrl core: initialized pinctrl subsystem
[ 0.262387] PM: RTC time: 20:56:07, date: 2023-04-21
[ 0.262387] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[ 0.262600] audit: initializing netlink subsys (disabled)
[ 0.262617] audit: type=2000 audit(1682110567.140:1): state=initialized audit_enabled=0 res=1
[ 0.262617] thermal_sys: Registered thermal governor 'fair_share'
[ 0.262617] thermal_sys: Registered thermal governor 'bang_bang'
[ 0.262617] thermal_sys: Registered thermal governor 'step_wise'
[ 0.262617] thermal_sys: Registered thermal governor 'user_space'
[ 0.262617] cpuidle: using governor ladder
[ 0.262617] cpuidle: using governor menu
[ 0.262617] PCI: MMCONFIG for domain 0000 [bus 00-3f] at [mem 0xf8000000-0xfbffffff] (base 0xf8000000)
[ 0.262617] PCI: MMCONFIG at [mem 0xf8000000-0xfbffffff] reserved as E820 entry
[ 0.262617] PCI: Using configuration type 1 for base access
[ 0.262617] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.
[ 0.273356] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages
[ 0.273356] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page
[ 0.273356] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages
[ 0.273356] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page
[ 0.280509] cryptd: max_cpu_qlen set to 1000
[ 0.280509] ACPI: Added _OSI(Module Device)
[ 0.280509] ACPI: Added _OSI(Processor Device)
[ 0.280509] ACPI: Added _OSI(3.0 _SCP Extensions)
[ 0.280509] ACPI: Added _OSI(Processor Aggregator Device)
[ 0.287039] ACPI: 4 ACPI AML tables successfully acquired and loaded
[ 0.288072] ACPI: Interpreter enabled
[ 0.288072] ACPI: PM: (supports S0 S1 S3 S5)
[ 0.288072] ACPI: Using IOAPIC for interrupt routing
[ 0.288072] HEST: Table parsing has been initialized.
[ 0.288072] GHES: Failed to enable APEI firmware first mode.
[ 0.288072] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[ 0.288072] PCI: Ignoring E820 reservations for host bridge windows
[ 0.288072] ACPI: Enabled 8 GPEs in block 00 to 1F
[ 0.290922] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[ 0.290932] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]
[ 0.291018] acpi PNP0A03:00: _OSC: OS now controls [PME AER PCIeCapability LTR]
[ 0.291034] acpi PNP0A03:00: [Firmware Info]: MMCONFIG for domain 0000 [bus 00-3f] only partially covers this bridge
[ 0.291117] acpi PNP0A03:00: host bridge window expanded to [io 0x0000-0x0cf7 window]; [io 0x03b0-0x03df window] ignored
[ 0.291369] PCI host bridge to bus 0000:00
[ 0.291371] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]
[ 0.291374] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]
[ 0.291376] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000dffff]
[ 0.291379] pci_bus 0000:00: root bus resource [mem 0x80000000-0xffffffff]
[ 0.291381] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 0.291406] pci 0000:00:00.0: [1022:1410] type 00 class 0x060000
[ 0.291559] pci 0000:00:00.2: [1022:1419] type 00 class 0x080600
[ 0.291648] pci 0000:00:01.0: [1002:9996] type 00 class 0x030000
[ 0.291656] pci 0000:00:01.0: reg 0x10: [mem 0xe0000000-0xefffffff pref]
[ 0.291661] pci 0000:00:01.0: reg 0x14: [io 0x1000-0x10ff]
[ 0.291665] pci 0000:00:01.0: reg 0x18: [mem 0xf0180000-0xf01bffff]
[ 0.291681] pci 0000:00:01.0: enabling Extended Tags
[ 0.291694] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]
[ 0.291711] pci 0000:00:01.0: supports D1 D2
[ 0.291776] pci 0000:00:01.1: [1002:9902] type 00 class 0x040300
[ 0.291784] pci 0000:00:01.1: reg 0x10: [mem 0xf01c0000-0xf01c3fff]
[ 0.291805] pci 0000:00:01.1: enabling Extended Tags
[ 0.291828] pci 0000:00:01.1: supports D1 D2
[ 0.291918] pci 0000:00:11.0: [1022:7801] type 00 class 0x010601
[ 0.291931] pci 0000:00:11.0: reg 0x10: [io 0x1410-0x1417]
[ 0.291938] pci 0000:00:11.0: reg 0x14: [io 0x1420-0x1423]
[ 0.291946] pci 0000:00:11.0: reg 0x18: [io 0x1418-0x141f]
[ 0.291953] pci 0000:00:11.0: reg 0x1c: [io 0x1424-0x1427]
[ 0.291960] pci 0000:00:11.0: reg 0x20: [io 0x1400-0x140f]
[ 0.291968] pci 0000:00:11.0: reg 0x24: [mem 0xf01cc000-0xf01cc7ff]
[ 0.292125] pci 0000:00:12.0: [1022:7807] type 00 class 0x0c0310
[ 0.292139] pci 0000:00:12.0: reg 0x10: [mem 0xf01c8000-0xf01c8fff]
[ 0.292322] pci 0000:00:12.2: [1022:7808] type 00 class 0x0c0320
[ 0.292335] pci 0000:00:12.2: reg 0x10: [mem 0xf01cd000-0xf01cd0ff]
[ 0.292400] pci 0000:00:12.2: supports D1 D2
[ 0.292402] pci 0000:00:12.2: PME# supported from D0 D1 D2 D3hot
[ 0.292542] pci 0000:00:13.0: [1022:7807] type 00 class 0x0c0310
[ 0.292555] pci 0000:00:13.0: reg 0x10: [mem 0xf01c9000-0xf01c9fff]
[ 0.292737] pci 0000:00:13.2: [1022:7808] type 00 class 0x0c0320
[ 0.292750] pci 0000:00:13.2: reg 0x10: [mem 0xf01ce000-0xf01ce0ff]
[ 0.292815] pci 0000:00:13.2: supports D1 D2
[ 0.292816] pci 0000:00:13.2: PME# supported from D0 D1 D2 D3hot
[ 0.292955] pci 0000:00:14.0: [1022:780b] type 00 class 0x0c0500
[ 0.293139] pci 0000:00:14.2: [1022:780d] type 00 class 0x040300
[ 0.293156] pci 0000:00:14.2: reg 0x10: [mem 0xf01c4000-0xf01c7fff 64bit]
[ 0.293210] pci 0000:00:14.2: PME# supported from D0 D3hot D3cold
[ 0.293355] pci 0000:00:14.3: [1022:780e] type 00 class 0x060100
[ 0.293541] pci 0000:00:14.4: [1022:780f] type 01 class 0x060401
[ 0.293703] pci 0000:00:14.5: [1022:7809] type 00 class 0x0c0310
[ 0.293717] pci 0000:00:14.5: reg 0x10: [mem 0xf01ca000-0xf01cafff]
[ 0.293906] pci 0000:00:15.0: [1022:43a0] type 01 class 0x060400
[ 0.293935] pci 0000:00:15.0: enabling Extended Tags
[ 0.293975] pci 0000:00:15.0: supports D1 D2
[ 0.294151] pci 0000:00:15.1: [1022:43a1] type 01 class 0x060400
[ 0.294182] pci 0000:00:15.1: enabling Extended Tags
[ 0.294221] pci 0000:00:15.1: supports D1 D2
[ 0.294398] pci 0000:00:15.2: [1022:43a2] type 01 class 0x060400
[ 0.294426] pci 0000:00:15.2: enabling Extended Tags
[ 0.294468] pci 0000:00:15.2: supports D1 D2
[ 0.294547] pci 0000:00:16.0: [1022:7807] type 00 class 0x0c0310
[ 0.294560] pci 0000:00:16.0: reg 0x10: [mem 0xf01cb000-0xf01cbfff]
[ 0.294737] pci 0000:00:16.2: [1022:7808] type 00 class 0x0c0320
[ 0.294751] pci 0000:00:16.2: reg 0x10: [mem 0xf01cf000-0xf01cf0ff]
[ 0.294815] pci 0000:00:16.2: supports D1 D2
[ 0.294817] pci 0000:00:16.2: PME# supported from D0 D1 D2 D3hot
[ 0.294958] pci 0000:00:18.0: [1022:1400] type 00 class 0x060000
[ 0.295024] pci 0000:00:18.1: [1022:1401] type 00 class 0x060000
[ 0.295086] pci 0000:00:18.2: [1022:1402] type 00 class 0x060000
[ 0.295152] pci 0000:00:18.3: [1022:1403] type 00 class 0x060000
[ 0.295289] pci 0000:00:18.4: [1022:1404] type 00 class 0x060000
[ 0.295353] pci 0000:00:18.5: [1022:1405] type 00 class 0x060000
[ 0.295436] pci_bus 0000:01: extended config space not accessible
[ 0.295499] pci 0000:00:14.4: PCI bridge to [bus 01] (subtractive decode)
[ 0.295510] pci 0000:00:14.4: bridge window [io 0x0000-0x0cf7 window] (subtractive decode)
[ 0.295513] pci 0000:00:14.4: bridge window [io 0x0d00-0xffff window] (subtractive decode)
[ 0.295515] pci 0000:00:14.4: bridge window [mem 0x000a0000-0x000dffff] (subtractive decode)
[ 0.295517] pci 0000:00:14.4: bridge window [mem 0x80000000-0xffffffff] (subtractive decode)
[ 0.295571] pci 0000:00:15.0: PCI bridge to [bus 02]
[ 0.295653] pci 0000:03:00.0: [1b21:1042] type 00 class 0x0c0330
[ 0.295689] pci 0000:03:00.0: reg 0x10: [mem 0xf0000000-0xf0007fff 64bit]
[ 0.295863] pci 0000:03:00.0: PME# supported from D3hot D3cold
[ 0.295907] pci 0000:03:00.0: 2.000 Gb/s available PCIe bandwidth, limited by 2.5 GT/s PCIe x1 link at 0000:00:15.1 (capable of 4.000 Gb/s with 5.0 GT/s PCIe x1 link)
[ 0.309393] pci 0000:00:15.1: PCI bridge to [bus 03]
[ 0.309405] pci 0000:00:15.1: bridge window [mem 0xf0000000-0xf00fffff]
[ 0.309414] pci 0000:00:15.2: bridge configuration invalid ([bus 00-00]), reconfiguring
[ 0.309545] pci 0000:04:00.0: [10ec:8168] type 00 class 0x020000
[ 0.309563] pci 0000:04:00.0: reg 0x10: [io 0x0000-0x00ff]
[ 0.309585] pci 0000:04:00.0: reg 0x18: [mem 0x00000000-0x00000fff 64bit pref]
[ 0.309598] pci 0000:04:00.0: reg 0x20: [mem 0x00000000-0x00003fff 64bit pref]
[ 0.309705] pci 0000:04:00.0: supports D1 D2
[ 0.309707] pci 0000:04:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[ 0.325403] pci 0000:00:15.2: PCI bridge to [bus 04-ff]
[ 0.325414] pci 0000:00:15.2: bridge window [io 0x0000-0x0fff]
[ 0.325417] pci 0000:00:15.2: bridge window [mem 0x00000000-0x000fffff]
[ 0.325422] pci 0000:00:15.2: bridge window [mem 0x00000000-0x000fffff 64bit pref]
[ 0.325425] pci_bus 0000:04: busn_res: [bus 04-ff] end is updated to 04
[ 0.325919] ACPI: PCI: Interrupt link INTA configured for IRQ 0
[ 0.326011] ACPI: PCI: Interrupt link INTB configured for IRQ 0
[ 0.326102] ACPI: PCI: Interrupt link INTC configured for IRQ 0
[ 0.326193] ACPI: PCI: Interrupt link INTD configured for IRQ 0
[ 0.326284] ACPI: PCI: Interrupt link INTE configured for IRQ 0
[ 0.326375] ACPI: PCI: Interrupt link INTF configured for IRQ 0
[ 0.326465] ACPI: PCI: Interrupt link INTG configured for IRQ 0
[ 0.326556] ACPI: PCI: Interrupt link INTH configured for IRQ 0
[ 0.326777] iommu: Default domain type: Translated
[ 0.326778] iommu: DMA domain TLB invalidation policy: lazy mode
[ 0.326974] SCSI subsystem initialized
[ 0.329389] libata version 3.00 loaded.
[ 0.329395] ACPI: bus type USB registered
[ 0.329418] usbcore: registered new interface driver usbfs
[ 0.329428] usbcore: registered new interface driver hub
[ 0.329437] usbcore: registered new device driver usb
[ 0.329564] PCI: Using ACPI for IRQ routing
[ 0.331132] PCI: pci_cache_line_size set to 64 bytes
[ 0.331184] e820: reserve RAM buffer [mem 0x0009fc00-0x0009ffff]
[ 0.331187] e820: reserve RAM buffer [mem 0x5fe4d000-0x5fffffff]
[ 0.331189] e820: reserve RAM buffer [mem 0x17f000000-0x17fffffff]
[ 0.331235] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0
[ 0.331240] hpet0: 3 comparators, 32-bit 14.318180 MHz counter
[ 0.334410] clocksource: Switched to clocksource tsc-early
[ 0.334656] VFS: Disk quotas dquot_6.6.0
[ 0.334682] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[ 0.334803] pnp: PnP ACPI init
[ 0.335147] system 00:00: [mem 0xfec10002-0xfec11001] could not be reserved
[ 0.335525] pnp: PnP ACPI: found 2 devices
[ 0.342072] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
[ 0.342260] NET: Registered PF_INET protocol family
[ 0.342434] IP idents hash table entries: 65536 (order: 7, 524288 bytes, linear)
[ 0.344041] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear)
[ 0.344058] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
[ 0.344070] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear)
[ 0.344141] TCP bind hash table entries: 32768 (order: 8, 1048576 bytes, linear)
[ 0.344505] TCP: Hash tables configured (established 32768 bind 32768)
[ 0.344576] UDP hash table entries: 2048 (order: 4, 65536 bytes, linear)
[ 0.344601] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear)
[ 0.344722] NET: Registered PF_UNIX/PF_LOCAL protocol family
[ 0.344757] pci 0000:00:15.2: BAR 15: assigned [mem 0x80000000-0x800fffff 64bit pref]
[ 0.344763] pci 0000:00:15.2: BAR 13: assigned [io 0x2000-0x2fff]
[ 0.344769] pci 0000:00:14.4: PCI bridge to [bus 01]
[ 0.344781] pci 0000:00:15.0: PCI bridge to [bus 02]
[ 0.344790] pci 0000:00:15.1: PCI bridge to [bus 03]
[ 0.344793] pci 0000:00:15.1: bridge window [mem 0xf0000000-0xf00fffff]
[ 0.344807] pci 0000:04:00.0: BAR 4: assigned [mem 0x80000000-0x80003fff 64bit pref]
[ 0.344821] pci 0000:04:00.0: BAR 2: assigned [mem 0x80004000-0x80004fff 64bit pref]
[ 0.344833] pci 0000:04:00.0: BAR 0: assigned [io 0x2000-0x20ff]
[ 0.344840] pci 0000:00:15.2: PCI bridge to [bus 04]
[ 0.344842] pci 0000:00:15.2: bridge window [io 0x2000-0x2fff]
[ 0.344847] pci 0000:00:15.2: bridge window [mem 0x80000000-0x800fffff 64bit pref]
[ 0.344854] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]
[ 0.344857] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]
[ 0.344860] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000dffff]
[ 0.344862] pci_bus 0000:00: resource 7 [mem 0x80000000-0xffffffff]
[ 0.344864] pci_bus 0000:01: resource 4 [io 0x0000-0x0cf7 window]
[ 0.344867] pci_bus 0000:01: resource 5 [io 0x0d00-0xffff window]
[ 0.344869] pci_bus 0000:01: resource 6 [mem 0x000a0000-0x000dffff]
[ 0.344871] pci_bus 0000:01: resource 7 [mem 0x80000000-0xffffffff]
[ 0.344873] pci_bus 0000:03: resource 1 [mem 0xf0000000-0xf00fffff]
[ 0.344875] pci_bus 0000:04: resource 0 [io 0x2000-0x2fff]
[ 0.344877] pci_bus 0000:04: resource 2 [mem 0x80000000-0x800fffff 64bit pref]
[ 0.345003] pci 0000:00:01.1: D0 power state depends on 0000:00:01.0
[ 0.345887] pci 0000:00:12.2: PME# does not work under D3, disabling it
[ 0.346671] pci 0000:00:13.2: PME# does not work under D3, disabling it
[ 0.347980] pci 0000:00:16.2: PME# does not work under D3, disabling it
[ 0.348353] PCI: CLS 64 bytes, default 64
[ 0.348449] pci 0000:00:00.2: AMD-Vi: Applying erratum 746 workaround
[ 0.348560] pci 0000:00:01.0: Adding to iommu group 0
[ 0.348584] pci 0000:00:01.1: Adding to iommu group 0
[ 0.348616] pci 0000:00:11.0: Adding to iommu group 1
[ 0.348659] pci 0000:00:12.0: Adding to iommu group 2
[ 0.348681] pci 0000:00:12.2: Adding to iommu group 2
[ 0.348723] pci 0000:00:13.0: Adding to iommu group 3
[ 0.348743] pci 0000:00:13.2: Adding to iommu group 3
[ 0.348792] pci 0000:00:14.0: Adding to iommu group 4
[ 0.348817] pci 0000:00:14.2: Adding to iommu group 4
[ 0.348840] pci 0000:00:14.3: Adding to iommu group 4
[ 0.348879] pci 0000:00:14.4: Adding to iommu group 5
[ 0.348903] pci 0000:00:14.5: Adding to iommu group 6
[ 0.348946] pci 0000:00:15.0: Adding to iommu group 7
[ 0.348974] pci 0000:00:15.1: Adding to iommu group 7
[ 0.349000] pci 0000:00:15.2: Adding to iommu group 7
[ 0.349040] pci 0000:00:16.0: Adding to iommu group 8
[ 0.349063] pci 0000:00:16.2: Adding to iommu group 8
[ 0.349128] pci 0000:00:18.0: Adding to iommu group 9
[ 0.349150] pci 0000:00:18.1: Adding to iommu group 9
[ 0.349188] pci 0000:00:18.2: Adding to iommu group 9
[ 0.349210] pci 0000:00:18.3: Adding to iommu group 9
[ 0.349234] pci 0000:00:18.4: Adding to iommu group 9
[ 0.349255] pci 0000:00:18.5: Adding to iommu group 9
[ 0.349269] pci 0000:03:00.0: Adding to iommu group 7
[ 0.349279] pci 0000:04:00.0: Adding to iommu group 7
[ 0.352005] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[ 0.352011] AMD-Vi: Extended features (0x800000853, 0x0): PreF PPR GT IA
[ 0.352020] AMD-Vi: Interrupt remapping enabled
[ 0.352238] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[ 0.352240] software IO TLB: mapped [mem 0x000000005be4d000-0x000000005fe4d000] (64MB)
[ 0.352297] LVT offset 0 assigned for vector 0x400
[ 0.352343] perf: AMD IBS detected (0x000000ff)
[ 0.352351] amd_uncore: 4 amd_nb counters detected
[ 0.356598] workingset: timestamp_bits=37 max_order=20 bucket_order=0
[ 0.356626] zbud: loaded
[ 0.357112] NET: Registered PF_ALG protocol family
[ 0.357118] Key type asymmetric registered
[ 0.357119] Asymmetric key parser 'x509' registered
[ 0.357471] alg: self-tests disabled
[ 0.357567] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 251)
[ 0.357610] io scheduler mq-deadline registered
[ 0.357612] io scheduler kyber registered
[ 0.358782] pcieport 0000:00:15.0: PME: Signaling with IRQ 25
[ 0.358948] pcieport 0000:00:15.1: PME: Signaling with IRQ 26
[ 0.359013] pcieport 0000:00:15.2: enabling device (0000 -> 0003)
[ 0.359228] pcieport 0000:00:15.2: PME: Signaling with IRQ 27
[ 0.359503] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
[ 0.359567] ACPI: button: Power Button [PWRF]
[ 0.359621] ACPI: \_SB_.P000: Found 2 idle states
[ 0.359743] ACPI: \_SB_.P001: Found 2 idle states
[ 0.360670] thermal LNXTHERM:00: registered as thermal_zone0
[ 0.360673] ACPI: thermal: Thermal Zone [TZ00] (15 C)
[ 0.360984] Non-volatile memory driver v1.3
[ 0.361051] AMD-Vi: AMD IOMMUv2 loaded and initialized
[ 0.361079] ACPI: bus type drm_connector registered
[ 0.361292] ahci 0000:00:11.0: version 3.0
[ 0.361580] ahci 0000:00:11.0: AHCI 0001.0300 32 slots 8 ports 6 Gbps 0x40 impl SATA mode
[ 0.361584] ahci 0000:00:11.0: flags: 64bit ncq sntf ilck led clo pio
[ 0.362741] scsi host0: ahci
[ 0.362967] scsi host1: ahci
[ 0.363157] scsi host2: ahci
[ 0.363372] scsi host3: ahci
[ 0.363571] scsi host4: ahci
[ 0.363773] scsi host5: ahci
[ 0.363984] scsi host6: ahci
[ 0.364182] scsi host7: ahci
[ 0.364275] ata1: DUMMY
[ 0.364277] ata2: DUMMY
[ 0.364278] ata3: DUMMY
[ 0.364278] ata4: DUMMY
[ 0.364279] ata5: DUMMY
[ 0.364280] ata6: DUMMY
[ 0.364282] ata7: SATA max UDMA/133 abar m2048@0xf01cc000 port 0xf01cc400 irq 19
[ 0.364284] ata8: DUMMY
[ 0.364564] i8042: PNP: No PS/2 controller found.
[ 0.364565] i8042: Probing ports directly.
[ 0.366981] serio: i8042 KBD port at 0x60,0x64 irq 1
[ 0.366988] serio: i8042 AUX port at 0x60,0x64 irq 12
[ 0.367123] mousedev: PS/2 mouse device common for all mice
[ 0.367188] rtc_cmos 00:01: RTC can wake from S4
[ 0.367487] rtc_cmos 00:01: registered as rtc0
[ 0.367510] rtc_cmos 00:01: setting system clock to 2023-04-21T20:56:07 UTC (1682110567)
[ 0.367553] rtc_cmos 00:01: alarms up to one day, y3k, 114 bytes nvram, hpet irqs
[ 0.367590] device-mapper: uevent: version 1.0.3
[ 0.367665] device-mapper: ioctl: 4.47.0-ioctl (2022-07-28) initialised: dm-devel@redhat.com
[ 0.367827] hid: raw HID events driver (C) Jiri Kosina
[ 0.367913] usbcore: registered new interface driver usbhid
[ 0.367914] usbhid: USB HID core driver
[ 0.368010] Initializing XFRM netlink socket
[ 0.368020] NET: Registered PF_PACKET protocol family
[ 0.368021] x86/pm: family 0x15 cpu detected, MSR saving is needed during suspending.
[ 0.368340] microcode: CPU1: patch_level=0x0600111f
[ 0.368342] microcode: CPU0: patch_level=0x0600111f
[ 0.368353] microcode: Microcode Update Driver: v2.2.
[ 0.368358] IPI shorthand broadcast: enabled
[ 0.368370] AVX version of gcm_enc/dec engaged.
[ 0.368401] AES CTR mode by8 optimization enabled
[ 0.372495] sched_clock: Marking stable (254511749, 117328013)->(374590709, -2750947)
[ 0.372745] registered taskstats version 1
[ 0.372987] zswap: loaded using pool lzo/zbud
[ 0.377461] kmemleak: Kernel memory leak detector initialized (mem pool available: 15679)
[ 0.377464] kmemleak: Automatic memory scanning thread started
[ 0.377465] debug_vm_pgtable: [debug_vm_pgtable ]: Validating architecture page table helpers
[ 0.382250] Key type encrypted registered
[ 0.385395] PM: Magic number: 3:579:953
[ 0.385449] memory memory6: hash matches
[ 0.483426] ata7: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[ 0.483587] ata7.00: ATA-9: SanDisk SDSSDP064G, 2.0.0, max UDMA/133
[ 0.483590] ata7.00: 125045424 sectors, multi 1: LBA48 NCQ (depth 32)
[ 0.483792] ata7.00: configured for UDMA/133
[ 0.484004] scsi 6:0:0:0: Direct-Access ATA SanDisk SDSSDP06 0 PQ: 0 ANSI: 5
[ 0.485006] sd 6:0:0:0: [sda] 125045424 512-byte logical blocks: (64.0 GB/59.6 GiB)
[ 0.485028] sd 6:0:0:0: [sda] Write Protect is off
[ 0.485033] sd 6:0:0:0: [sda] Mode Sense: 00 3a 00 00
[ 0.485062] sd 6:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[ 0.485103] sd 6:0:0:0: [sda] Preferred minimum I/O size 512 bytes
[ 0.486554] sda: sda1 sda2 sda3
[ 0.487033] sd 6:0:0:0: [sda] Attached SCSI disk
[ 0.508543] EXT4-fs (sda3): mounted filesystem fe29e0dc-6303-4401-987c-8472bc1b9516 with ordered data mode. Quota mode: none.
[ 0.508591] VFS: Mounted root (ext4 filesystem) on device 8:3.
[ 0.510618] devtmpfs: mounted
[ 0.514506] Freeing unused kernel image (initmem) memory: 2908K
[ 0.532141] Write protecting the kernel read-only data: 20480k
[ 0.532425] Freeing unused kernel image (rodata/data gap) memory: 836K
[ 0.569736] x86/mm: Checked W+X mappings: passed, no W+X pages found.
[ 0.569742] rodata_test: all tests were successful
[ 0.569773] Run /sbin/init as init process
[ 0.569775] with arguments:
[ 0.569776] /sbin/init
[ 0.569777] noisapnp
[ 0.569778] with environment:
[ 0.569779] HOME=/
[ 0.569780] TERM=linux
[ 0.569781] BOOT_IMAGE=/boot/vmlinuz-6.3.0-rc3-00046-g8ba643d7e1c7
[ 0.751269] systemd[1]: Inserted module 'autofs4'
[ 0.777767] NET: Registered PF_INET6 protocol family
[ 0.778674] Segment Routing with IPv6
[ 0.778702] In-situ OAM (IOAM) with IPv6
[ 0.804133] systemd[1]: systemd 252.6-1 running in system mode (+PAM +AUDIT +SELINUX +APPARMOR +IMA +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 -IDN +IPTC +KMOD +LIBCRYPTSETUP +LIBFDISK +PCRE2 -PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD -BPF_FRAMEWORK -XKBCOMMON +UTMP +SYSVINIT default-hierarchy=unified)
[ 0.804143] systemd[1]: Detected architecture x86-64.
[ 0.808903] systemd[1]: Hostname set to <kodi>.
[ 1.081619] systemd[1]: Queued start job for default target graphical.target.
[ 1.102404] systemd[1]: Created slice system-getty.slice - Slice /system/getty.
[ 1.103505] systemd[1]: Created slice system-modprobe.slice - Slice /system/modprobe.
[ 1.104345] systemd[1]: Created slice user.slice - User and Session Slice.
[ 1.104531] systemd[1]: Started systemd-ask-password-console.path - Dispatch Password Requests to Console Directory Watch.
[ 1.104655] systemd[1]: Started systemd-ask-password-wall.path - Forward Password Requests to Wall Directory Watch.
[ 1.105082] systemd[1]: Set up automount proc-sys-fs-binfmt_misc.automount - Arbitrary Executable File Formats File System Automount Point.
[ 1.105122] systemd[1]: Reached target cryptsetup.target - Local Encrypted Volumes.
[ 1.105171] systemd[1]: Reached target integritysetup.target - Local Integrity Protected Volumes.
[ 1.105213] systemd[1]: Reached target paths.target - Path Units.
[ 1.105248] systemd[1]: Reached target remote-fs.target - Remote File Systems.
[ 1.105276] systemd[1]: Reached target slices.target - Slice Units.
[ 1.105302] systemd[1]: Reached target swap.target - Swaps.
[ 1.105344] systemd[1]: Reached target veritysetup.target - Local Verity Protected Volumes.
[ 1.107911] systemd[1]: Listening on systemd-coredump.socket - Process Core Dump Socket.
[ 1.108154] systemd[1]: Listening on systemd-fsckd.socket - fsck to fsckd communication Socket.
[ 1.108318] systemd[1]: Listening on systemd-initctl.socket - initctl Compatibility Named Pipe.
[ 1.108624] systemd[1]: Listening on systemd-journald-audit.socket - Journal Audit Socket.
[ 1.108886] systemd[1]: Listening on systemd-journald-dev-log.socket - Journal Socket (/dev/log).
[ 1.109157] systemd[1]: Listening on systemd-journald.socket - Journal Socket.
[ 1.109414] systemd[1]: Listening on systemd-networkd.socket - Network Service Netlink Socket.
[ 1.110215] systemd[1]: Listening on systemd-udevd-control.socket - udev Control Socket.
[ 1.110475] systemd[1]: Listening on systemd-udevd-kernel.socket - udev Kernel Socket.
[ 1.113168] systemd[1]: Mounting dev-hugepages.mount - Huge Pages File System...
[ 1.116585] systemd[1]: Mounting dev-mqueue.mount - POSIX Message Queue File System...
[ 1.121085] systemd[1]: Mounting sys-kernel-debug.mount - Kernel Debug File System...
[ 1.124574] systemd[1]: Mounting sys-kernel-tracing.mount - Kernel Trace File System...
[ 1.128227] systemd[1]: Starting kmod-static-nodes.service - Create List of Static Device Nodes...
[ 1.132312] systemd[1]: Starting modprobe@configfs.service - Load Kernel Module configfs...
[ 1.135610] systemd[1]: Starting modprobe@dm_mod.service - Load Kernel Module dm_mod...
[ 1.138876] systemd[1]: Starting modprobe@drm.service - Load Kernel Module drm...
[ 1.147425] systemd[1]: Starting modprobe@efi_pstore.service - Load Kernel Module efi_pstore...
[ 1.150601] systemd[1]: Starting modprobe@fuse.service - Load Kernel Module fuse...
[ 1.153921] systemd[1]: Starting modprobe@loop.service - Load Kernel Module loop...
[ 1.154024] systemd[1]: systemd-firstboot.service - First Boot Wizard was skipped because of an unmet condition check (ConditionFirstBoot=yes).
[ 1.154112] systemd[1]: systemd-fsck-root.service - File System Check on Root Device was skipped because of an unmet condition check (ConditionPathIsReadWrite=!/).
[ 1.154158] systemd[1]: Reached target local-fs.target - Local File Systems.
[ 1.154259] systemd[1]: apparmor.service - Load AppArmor profiles was skipped because of an unmet condition check (ConditionSecurity=apparmor).
[ 1.167112] fuse: init (API version 7.38)
[ 1.167255] loop: module loaded
[ 1.168586] systemd[1]: Starting systemd-binfmt.service - Set Up Additional Binary Formats...
[ 1.174035] systemd[1]: Starting systemd-journald.service - Journal Service...
[ 1.177353] systemd[1]: Starting systemd-random-seed.service - Load/Save Random Seed...
[ 1.180848] systemd[1]: Starting systemd-sysctl.service - Apply Kernel Variables...
[ 1.195025] systemd[1]: Starting systemd-sysusers.service - Create System Users...
[ 1.201956] systemd[1]: Starting systemd-udev-trigger.service - Coldplug All udev Devices...
[ 1.217695] systemd[1]: Mounted dev-hugepages.mount - Huge Pages File System.
[ 1.218409] systemd[1]: Mounted dev-mqueue.mount - POSIX Message Queue File System.
[ 1.218683] systemd[1]: Mounted sys-kernel-debug.mount - Kernel Debug File System.
[ 1.218957] systemd[1]: Mounted sys-kernel-tracing.mount - Kernel Trace File System.
[ 1.225546] systemd[1]: Finished kmod-static-nodes.service - Create List of Static Device Nodes.
[ 1.226534] systemd[1]: modprobe@configfs.service: Deactivated successfully.
[ 1.226893] systemd[1]: Finished modprobe@configfs.service - Load Kernel Module configfs.
[ 1.227645] systemd[1]: modprobe@dm_mod.service: Deactivated successfully.
[ 1.228209] systemd[1]: Finished modprobe@dm_mod.service - Load Kernel Module dm_mod.
[ 1.230383] systemd[1]: modprobe@drm.service: Deactivated successfully.
[ 1.230725] systemd[1]: Finished modprobe@drm.service - Load Kernel Module drm.
[ 1.231459] systemd[1]: modprobe@efi_pstore.service: Deactivated successfully.
[ 1.231774] systemd[1]: Finished modprobe@efi_pstore.service - Load Kernel Module efi_pstore.
[ 1.232474] systemd[1]: modprobe@fuse.service: Deactivated successfully.
[ 1.232795] systemd[1]: Finished modprobe@fuse.service - Load Kernel Module fuse.
[ 1.234105] systemd[1]: modprobe@loop.service: Deactivated successfully.
[ 1.234462] systemd[1]: Finished modprobe@loop.service - Load Kernel Module loop.
[ 1.235099] systemd[1]: proc-sys-fs-binfmt_misc.automount: Got automount request for /proc/sys/fs/binfmt_misc, triggered by 140 (systemd-binfmt)
[ 1.261703] systemd[1]: Mounting proc-sys-fs-binfmt_misc.mount - Arbitrary Executable File Formats File System...
[ 1.270160] systemd[1]: Mounting sys-fs-fuse-connections.mount - FUSE Control File System...
[ 1.273274] systemd[1]: Mounting sys-kernel-config.mount - Kernel Configuration File System...
[ 1.273402] systemd[1]: systemd-pstore.service - Platform Persistent Storage Archival was skipped because of an unmet condition check (ConditionDirectoryNotEmpty=/sys/fs/pstore).
[ 1.273574] systemd[1]: systemd-repart.service - Repartition Root Disk was skipped because no trigger condition checks were met.
[ 1.290481] systemd[1]: Finished systemd-sysctl.service - Apply Kernel Variables.
[ 1.294159] systemd[1]: Finished systemd-sysusers.service - Create System Users.
[ 1.294555] systemd[1]: Mounted sys-fs-fuse-connections.mount - FUSE Control File System.
[ 1.303356] systemd[1]: Starting systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev...
[ 1.304817] systemd[1]: Mounted sys-kernel-config.mount - Kernel Configuration File System.
[ 1.320909] systemd[1]: Mounted proc-sys-fs-binfmt_misc.mount - Arbitrary Executable File Formats File System.
[ 1.323560] systemd[1]: Finished systemd-binfmt.service - Set Up Additional Binary Formats.
[ 1.369356] tsc: Refined TSC clocksource calibration: 3900.224 MHz
[ 1.369367] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x70705a6472c, max_idle_ns: 881590586812 ns
[ 1.370036] systemd[1]: Finished systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev.
[ 1.373207] clocksource: Switched to clocksource tsc
[ 1.389831] systemd[1]: Starting systemd-udevd.service - Rule-based Manager for Device Events and Files...
[ 1.427455] systemd[1]: Started systemd-journald.service - Journal Service.
[ 1.472737] systemd-journald[142]: Received client request to flush runtime journal.
[ 1.742223] sd 6:0:0:0: Attached scsi generic sg0 type 0
[ 1.866042] acpi_cpufreq: overriding BIOS provided _PSD data
[ 2.005347] random: crng init done
[ 2.137832] QUIRK: Enable AMD PLL fix
[ 2.137832] QUIRK: Enable AMD PLL fix
[ 2.137887] ehci-pci 0000:00:13.2: EHCI Host Controller
[ 2.137922] ehci-pci 0000:00:13.2: new USB bus registered, assigned bus number 1
[ 2.137936] ehci-pci 0000:00:13.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
[ 2.137945] ehci-pci 0000:00:13.2: debug port 1
[ 2.138141] ehci-pci 0000:00:13.2: irq 17, io mem 0xf01ce000
[ 2.139491] piix4_smbus 0000:00:14.0: SMBus Host Controller at 0xb00, revision 0
[ 2.139505] piix4_smbus 0000:00:14.0: Using register 0x2e for SMBus port selection
[ 2.140107] piix4_smbus 0000:00:14.0: Auxiliary SMBus Host Controller at 0xb20
[ 2.154632] ehci-pci 0000:00:13.2: USB 2.0 started, EHCI 1.00
[ 2.155284] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 2.155291] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.155294] usb usb1: Product: EHCI Host Controller
[ 2.155296] usb usb1: Manufacturer: Linux 6.3.0-rc3-00046-g8ba643d7e1c7 ehci_hcd
[ 2.155299] usb usb1: SerialNumber: 0000:00:13.2
[ 2.174521] hub 1-0:1.0: USB hub found
[ 2.174560] hub 1-0:1.0: 5 ports detected
[ 2.175582] ehci-pci 0000:00:12.2: EHCI Host Controller
[ 2.175610] ehci-pci 0000:00:12.2: new USB bus registered, assigned bus number 2
[ 2.175623] ehci-pci 0000:00:12.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
[ 2.175632] ehci-pci 0000:00:12.2: debug port 1
[ 2.175789] ehci-pci 0000:00:12.2: irq 17, io mem 0xf01cd000
[ 2.189373] ehci-pci 0000:00:12.2: USB 2.0 started, EHCI 1.00
[ 2.189720] usb usb2: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 2.189725] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.189727] usb usb2: Product: EHCI Host Controller
[ 2.189730] usb usb2: Manufacturer: Linux 6.3.0-rc3-00046-g8ba643d7e1c7 ehci_hcd
[ 2.189732] usb usb2: SerialNumber: 0000:00:12.2
[ 2.190283] hub 2-0:1.0: USB hub found
[ 2.190323] hub 2-0:1.0: 5 ports detected
[ 2.191801] ehci-pci 0000:00:16.2: EHCI Host Controller
[ 2.191831] ehci-pci 0000:00:16.2: new USB bus registered, assigned bus number 3
[ 2.191845] ehci-pci 0000:00:16.2: applying AMD SB700/SB800/Hudson-2/3 EHCI dummy qh workaround
[ 2.191855] ehci-pci 0000:00:16.2: debug port 1
[ 2.192018] ehci-pci 0000:00:16.2: irq 17, io mem 0xf01cf000
[ 2.205373] ehci-pci 0000:00:16.2: USB 2.0 started, EHCI 1.00
[ 2.205651] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 2.205655] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.205658] usb usb3: Product: EHCI Host Controller
[ 2.205660] usb usb3: Manufacturer: Linux 6.3.0-rc3-00046-g8ba643d7e1c7 ehci_hcd
[ 2.205662] usb usb3: SerialNumber: 0000:00:16.2
[ 2.206200] hub 3-0:1.0: USB hub found
[ 2.206236] hub 3-0:1.0: 4 ports detected
[ 2.221053] ohci-pci 0000:00:12.0: OHCI PCI host controller
[ 2.221085] ohci-pci 0000:00:12.0: new USB bus registered, assigned bus number 4
[ 2.221264] ohci-pci 0000:00:12.0: irq 18, io mem 0xf01c8000
[ 2.237796] r8169 0000:04:00.0: enabling device (0000 -> 0003)
[ 2.261308] xhci_hcd 0000:03:00.0: xHCI Host Controller
[ 2.261394] xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 5
[ 2.272125] r8169 0000:04:00.0 eth0: RTL8168f/8111f, 08:60:6e:74:7a:51, XID 480, IRQ 28
[ 2.272143] r8169 0000:04:00.0 eth0: jumbo features [frames: 9194 bytes, tx checksumming: ko]
[ 2.282633] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 2.282644] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.282650] usb usb4: Product: OHCI PCI host controller
[ 2.282655] usb usb4: Manufacturer: Linux 6.3.0-rc3-00046-g8ba643d7e1c7 ohci_hcd
[ 2.282659] usb usb4: SerialNumber: 0000:00:12.0
[ 2.283372] hub 4-0:1.0: USB hub found
[ 2.283431] hub 4-0:1.0: 5 ports detected
[ 2.297576] ohci-pci 0000:00:13.0: OHCI PCI host controller
[ 2.297609] ohci-pci 0000:00:13.0: new USB bus registered, assigned bus number 6
[ 2.297727] ohci-pci 0000:00:13.0: irq 18, io mem 0xf01c9000
[ 2.320955] xhci_hcd 0000:03:00.0: hcc params 0x0200f180 hci version 0x96 quirks 0x0000000000080010
[ 2.322099] xhci_hcd 0000:03:00.0: xHCI Host Controller
[ 2.322121] xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 7
[ 2.322136] xhci_hcd 0000:03:00.0: Host supports USB 3.0 SuperSpeed
[ 2.324029] usb usb5: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.03
[ 2.324036] usb usb5: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.324039] usb usb5: Product: xHCI Host Controller
[ 2.324041] usb usb5: Manufacturer: Linux 6.3.0-rc3-00046-g8ba643d7e1c7 xhci-hcd
[ 2.324043] usb usb5: SerialNumber: 0000:03:00.0
[ 2.324687] hub 5-0:1.0: USB hub found
[ 2.324727] hub 5-0:1.0: 2 ports detected
[ 2.325467] usb usb7: We don't know the algorithms for LPM for this host, disabling LPM.
[ 2.325624] usb usb7: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 6.03
[ 2.325628] usb usb7: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.325630] usb usb7: Product: xHCI Host Controller
[ 2.325632] usb usb7: Manufacturer: Linux 6.3.0-rc3-00046-g8ba643d7e1c7 xhci-hcd
[ 2.325634] usb usb7: SerialNumber: 0000:03:00.0
[ 2.326195] hub 7-0:1.0: USB hub found
[ 2.327034] hub 7-0:1.0: 2 ports detected
[ 2.349205] snd_hda_intel 0000:00:01.1: Force to non-snoop mode
[ 2.366399] usb usb6: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 2.366406] usb usb6: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.366409] usb usb6: Product: OHCI PCI host controller
[ 2.366411] usb usb6: Manufacturer: Linux 6.3.0-rc3-00046-g8ba643d7e1c7 ohci_hcd
[ 2.366413] usb usb6: SerialNumber: 0000:00:13.0
[ 2.367210] hub 6-0:1.0: USB hub found
[ 2.367303] hub 6-0:1.0: 5 ports detected
[ 2.391447] r8169 0000:04:00.0 enp4s0: renamed from eth0
[ 2.420037] ohci-pci 0000:00:14.5: OHCI PCI host controller
[ 2.420070] ohci-pci 0000:00:14.5: new USB bus registered, assigned bus number 8
[ 2.421556] ohci-pci 0000:00:14.5: irq 18, io mem 0xf01ca000
[ 2.445281] r8169 0000:04:00.0: Direct firmware load for rtl_nic/rtl8168f-1.fw failed with error -2
[ 2.445292] r8169 0000:04:00.0: Unable to load firmware rtl_nic/rtl8168f-1.fw (-2)
[ 2.445868] RTL8211E Gigabit Ethernet r8169-0-400:00: attached PHY driver (mii_bus:phy_addr=r8169-0-400:00, irq=MAC)
[ 2.481250] input: HDA ATI HDMI HDMI/DP,pcm=3 as /devices/pci0000:00/0000:00:01.1/sound/card0/input1
[ 2.481880] input: HDA ATI HDMI HDMI/DP,pcm=7 as /devices/pci0000:00/0000:00:01.1/sound/card0/input2
[ 2.488733] usb usb8: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 2.488740] usb usb8: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.488743] usb usb8: Product: OHCI PCI host controller
[ 2.488745] usb usb8: Manufacturer: Linux 6.3.0-rc3-00046-g8ba643d7e1c7 ohci_hcd
[ 2.488747] usb usb8: SerialNumber: 0000:00:14.5
[ 2.492816] snd_hda_codec_realtek hdaudioC1D0: ALC892: SKU not ready 0x00000100
[ 2.492902] hub 8-0:1.0: USB hub found
[ 2.492953] hub 8-0:1.0: 2 ports detected
[ 2.493612] snd_hda_codec_realtek hdaudioC1D0: autoconfig for ALC892: line_outs=4 (0x14/0x16/0x15/0x17/0x0) type:line
[ 2.493620] snd_hda_codec_realtek hdaudioC1D0: speaker_outs=0 (0x0/0x0/0x0/0x0/0x0)
[ 2.493623] snd_hda_codec_realtek hdaudioC1D0: hp_outs=1 (0x1b/0x0/0x0/0x0/0x0)
[ 2.493626] snd_hda_codec_realtek hdaudioC1D0: mono: mono_out=0x0
[ 2.493628] snd_hda_codec_realtek hdaudioC1D0: dig-out=0x1e/0x0
[ 2.493629] snd_hda_codec_realtek hdaudioC1D0: inputs:
[ 2.493631] snd_hda_codec_realtek hdaudioC1D0: Rear Mic=0x18
[ 2.493633] snd_hda_codec_realtek hdaudioC1D0: Front Mic=0x19
[ 2.493635] snd_hda_codec_realtek hdaudioC1D0: Line=0x1a
[ 2.493636] snd_hda_codec_realtek hdaudioC1D0: CD=0x1c
[ 2.495003] ohci-pci 0000:00:16.0: OHCI PCI host controller
[ 2.495036] ohci-pci 0000:00:16.0: new USB bus registered, assigned bus number 9
[ 2.495172] ohci-pci 0000:00:16.0: irq 18, io mem 0xf01cb000
[ 2.510556] [drm] radeon kernel modesetting enabled.
[ 2.513510] [drm] initializing kernel modesetting (ARUBA 0x1002:0x9996 0x1002:0x9996 0x00).
[ 2.513588] ATOM BIOS: 113
[ 2.513893] radeon 0000:00:01.0: VRAM: 512M 0x0000000000000000 - 0x000000001FFFFFFF (512M used)
[ 2.513898] radeon 0000:00:01.0: GTT: 1024M 0x0000000020000000 - 0x000000005FFFFFFF
[ 2.513911] [drm] Detected VRAM RAM=512M, BAR=256M
[ 2.513913] [drm] RAM width 64bits DDR
[ 2.514805] [drm] radeon: 512M of VRAM memory ready
[ 2.514815] [drm] radeon: 1024M of GTT memory ready.
[ 2.514888] [drm] Loading ARUBA Microcode
[ 2.517185] r8169 0000:04:00.0 enp4s0: Link is Down
[ 2.519927] input: HD-Audio Generic Rear Mic as /devices/pci0000:00/0000:00:14.2/sound/card1/input3
[ 2.520307] input: HD-Audio Generic Front Mic as /devices/pci0000:00/0000:00:14.2/sound/card1/input4
[ 2.520824] input: HD-Audio Generic Line as /devices/pci0000:00/0000:00:14.2/sound/card1/input5
[ 2.521180] input: HD-Audio Generic Line Out Front as /devices/pci0000:00/0000:00:14.2/sound/card1/input6
[ 2.521594] input: HD-Audio Generic Line Out Surround as /devices/pci0000:00/0000:00:14.2/sound/card1/input7
[ 2.522160] input: HD-Audio Generic Line Out CLFE as /devices/pci0000:00/0000:00:14.2/sound/card1/input8
[ 2.522536] input: HD-Audio Generic Line Out Side as /devices/pci0000:00/0000:00:14.2/sound/card1/input9
[ 2.522888] input: HD-Audio Generic Front Headphone as /devices/pci0000:00/0000:00:14.2/sound/card1/input10
[ 2.532990] [drm] Internal thermal controller without fan control
[ 2.558842] usb usb9: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 6.03
[ 2.558849] usb usb9: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 2.558852] usb usb9: Product: OHCI PCI host controller
[ 2.558855] usb usb9: Manufacturer: Linux 6.3.0-rc3-00046-g8ba643d7e1c7 ohci_hcd
[ 2.558857] usb usb9: SerialNumber: 0000:00:16.0
[ 2.558984] [drm] radeon: dpm initialized
[ 2.564667] [drm] Found VCE firmware/feedback version 50.0.1 / 17!
[ 2.564728] [drm] GART: num cpu pages 262144, num gpu pages 262144
[ 2.594051] [drm] PCIE GART of 1024M enabled (table at 0x00000000001D6000).
[ 2.594314] radeon 0000:00:01.0: WB enabled
[ 2.594318] radeon 0000:00:01.0: fence driver on ring 0 use gpu addr 0x0000000020000c00
[ 2.594697] radeon 0000:00:01.0: fence driver on ring 5 use gpu addr 0x0000000000075a18
[ 2.595711] hub 9-0:1.0: USB hub found
[ 2.595753] hub 9-0:1.0: 4 ports detected
[ 2.618866] radeon 0000:00:01.0: fence driver on ring 6 use gpu addr 0x0000000020000c18
[ 2.618874] radeon 0000:00:01.0: fence driver on ring 7 use gpu addr 0x0000000020000c1c
[ 2.618877] radeon 0000:00:01.0: fence driver on ring 1 use gpu addr 0x0000000020000c04
[ 2.618879] radeon 0000:00:01.0: fence driver on ring 2 use gpu addr 0x0000000020000c08
[ 2.618881] radeon 0000:00:01.0: fence driver on ring 3 use gpu addr 0x0000000020000c0c
[ 2.618883] radeon 0000:00:01.0: fence driver on ring 4 use gpu addr 0x0000000020000c10
[ 2.647823] radeon 0000:00:01.0: radeon: MSI limited to 32-bit
[ 2.648048] radeon 0000:00:01.0: radeon: using MSI.
[ 2.648125] [drm] radeon: irq initialized.
[ 2.667958] [drm] ring test on 0 succeeded in 3 usecs
[ 2.667970] [drm] ring test on 3 succeeded in 4 usecs
[ 2.667977] [drm] ring test on 4 succeeded in 4 usecs
[ 2.713784] [drm] ring test on 5 succeeded in 2 usecs
[ 2.733666] [drm] UVD initialized successfully.
[ 2.741341] usb 4-1: new low-speed USB device number 2 using ohci-pci
[ 2.843040] [drm] ring test on 6 succeeded in 18 usecs
[ 2.843052] [drm] ring test on 7 succeeded in 3 usecs
[ 2.843053] [drm] VCE initialized successfully.
[ 2.843202] snd_hda_intel 0000:00:01.1: bound 0000:00:01.0 (ops radeon_audio_component_bind_ops [radeon])
[ 2.843370] [drm] ib test on ring 0 succeeded in 0 usecs
[ 2.843424] [drm] ib test on ring 3 succeeded in 0 usecs
[ 2.843474] [drm] ib test on ring 4 succeeded in 0 usecs
[ 2.942595] usb 4-1: New USB device found, idVendor=413c, idProduct=2106, bcdDevice= 1.01
[ 2.942607] usb 4-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 2.942611] usb 4-1: Product: Dell QuietKey Keyboard
[ 2.942615] usb 4-1: Manufacturer: DELL
[ 2.951006] input: DELL Dell QuietKey Keyboard as /devices/pci0000:00/0000:00:12.0/usb4/4-1/4-1:1.0/0003:413C:2106.0001/input/input11
[ 3.010775] hid-generic 0003:413C:2106.0001: input,hidraw0: USB HID v1.10 Keyboard [DELL Dell QuietKey Keyboard] on usb-0000:00:12.0-1/input0
[ 3.385454] [drm] ib test on ring 5 succeeded
[ 3.405349] usb 4-2: new low-speed USB device number 3 using ohci-pci
[ 3.600597] usb 4-2: New USB device found, idVendor=046d, idProduct=c016, bcdDevice= 3.40
[ 3.600609] usb 4-2: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 3.600613] usb 4-2: Product: Optical USB Mouse
[ 3.600617] usb 4-2: Manufacturer: Logitech
[ 3.610963] input: Logitech Optical USB Mouse as /devices/pci0000:00/0000:00:12.0/usb4/4-2/4-2:1.0/0003:046D:C016.0002/input/input12
[ 3.611917] hid-generic 0003:046D:C016.0002: input,hidraw1: USB HID v1.10 Mouse [Logitech Optical USB Mouse] on usb-0000:00:12.0-2/input0
[ 3.929454] [drm] ib test on ring 6 succeeded
[ 4.441455] [drm] ib test on ring 7 succeeded
[ 4.446983] [drm] Radeon Display Connectors
[ 4.446989] [drm] Connector 0:
[ 4.446992] [drm] DP-1
[ 4.446995] [drm] HPD1
[ 4.446997] [drm] DDC: 0x6530 0x6530 0x6534 0x6534 0x6538 0x6538 0x653c 0x653c
[ 4.447003] [drm] Encoders:
[ 4.447005] [drm] DFP1: INTERNAL_UNIPHY2
[ 4.447007] [drm] Connector 1:
[ 4.447009] [drm] VGA-1
[ 4.447011] [drm] HPD2
[ 4.447013] [drm] DDC: 0x6540 0x6540 0x6544 0x6544 0x6548 0x6548 0x654c 0x654c
[ 4.447018] [drm] Encoders:
[ 4.447020] [drm] CRT1: INTERNAL_UNIPHY2
[ 4.447022] [drm] CRT1: NUTMEG
[ 4.447024] [drm] Connector 2:
[ 4.447026] [drm] HDMI-A-1
[ 4.447028] [drm] HPD3
[ 4.447030] [drm] DDC: 0x6550 0x6550 0x6554 0x6554 0x6558 0x6558 0x655c 0x655c
[ 4.447035] [drm] Encoders:
[ 4.447036] [drm] DFP2: INTERNAL_UNIPHY
[ 4.716171] [drm] fb mappable at 0xE03E9000
[ 4.716179] [drm] vram apper at 0xE0000000
[ 4.716181] [drm] size 5242880
[ 4.716183] [drm] fb depth is 24
[ 4.716185] [drm] pitch is 5120
[ 4.716671] fbcon: radeondrmfb (fb0) is primary device
[ 4.908197] Console: switching to colour frame buffer device 160x64
[ 4.909974] radeon 0000:00:01.0: [drm] fb0: radeondrmfb frame buffer device
[ 4.933682] [drm] Initialized radeon 2.50.0 20080528 for 0000:00:01.0 on minor 0
[ 5.125043] r8169 0000:04:00.0 enp4s0: Link is Up - 1Gbps/Full - flow control rx/tx
[ 5.125059] IPv6: ADDRCONF(NETDEV_CHANGE): enp4s0: link becomes ready
[ 7.669093] [drm] amdgpu kernel modesetting enabled.
[ 11.216542] memfd_create() without MFD_EXEC nor MFD_NOEXEC_SEAL, pid=272 'systemd'
[-- Attachment #3: Type: text/plain, Size: 176 bytes --]
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 22/37] arm64: smp: Switch to hotplug core state synchronization
2023-04-17 15:50 ` Mark Rutland
(?)
@ 2023-04-25 19:51 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-25 19:51 UTC (permalink / raw)
To: Mark Rutland
Cc: LKML, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Catalin Marinas, Will Deacon,
linux-arm-kernel, David Woodhouse, Usama Arif, Juergen Gross,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Sabin Rapan
On Mon, Apr 17 2023 at 16:50, Mark Rutland wrote:
> On Sat, Apr 15, 2023 at 01:44:49AM +0200, Thomas Gleixner wrote:
> I gave this a spin on arm64 (in a 64-vCPU VM on an M1 host), and it seems to
> work fine with a bunch of vCPUs being hotplugged off and on again randomly.
>
> FWIW:
>
> Tested-by: Mark Rutland <mark.rutland@arm.com>
>
> I also hacked the code to have the dying CPU spin forever before the call to
> cpuhp_ap_report_dead(). In that case I see a warning, and that we don't call
> arch_cpuhp_cleanup_dead_cpu(), and that the CPU is marked as offline (per
> /sys/devices/system/cpu/$N/online).
Nice!
> As a tangent/aside, we might need to improve that for confidential compute
> architectures, and we might want to generically track cpus which might still be
> using kernel text/data. On arm64 we ensure that via our cpu_kill() callback
> (which'll use PSCI CPU_AFFINITY_INFO), but I'm not sure if TDX and/or SEV-SNP
> have a similar mechanism.
>
> Otherwise, a malicious hypervisor can pause a vCPU just before it leaves the
> kernel (e.g. immediately after the arch_cpuhp_cleanup_dead_cpu() call), wait
> for a kexec (or resuse of stack memroy), and unpause the vCPU to cause things
> to blow up.
There are a gazillion ways for a malicious hypervisor to blow up a
'squint enough to be confident' guest.
The real question is whether it can utilize such a blow up to extract
confidential information from the guest.
If not then it's just yet another way of DoS which is an "acceptable"
attack as it only affects availability but not confidentiality.
Thanks,
tglx
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 22/37] arm64: smp: Switch to hotplug core state synchronization
@ 2023-04-25 19:51 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-25 19:51 UTC (permalink / raw)
To: Mark Rutland
Cc: LKML, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Catalin Marinas, Will Deacon,
linux-arm-kernel, David Woodhouse, Usama Arif, Juergen Gross,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Sabin Rapan
On Mon, Apr 17 2023 at 16:50, Mark Rutland wrote:
> On Sat, Apr 15, 2023 at 01:44:49AM +0200, Thomas Gleixner wrote:
> I gave this a spin on arm64 (in a 64-vCPU VM on an M1 host), and it seems to
> work fine with a bunch of vCPUs being hotplugged off and on again randomly.
>
> FWIW:
>
> Tested-by: Mark Rutland <mark.rutland@arm.com>
>
> I also hacked the code to have the dying CPU spin forever before the call to
> cpuhp_ap_report_dead(). In that case I see a warning, and that we don't call
> arch_cpuhp_cleanup_dead_cpu(), and that the CPU is marked as offline (per
> /sys/devices/system/cpu/$N/online).
Nice!
> As a tangent/aside, we might need to improve that for confidential compute
> architectures, and we might want to generically track cpus which might still be
> using kernel text/data. On arm64 we ensure that via our cpu_kill() callback
> (which'll use PSCI CPU_AFFINITY_INFO), but I'm not sure if TDX and/or SEV-SNP
> have a similar mechanism.
>
> Otherwise, a malicious hypervisor can pause a vCPU just before it leaves the
> kernel (e.g. immediately after the arch_cpuhp_cleanup_dead_cpu() call), wait
> for a kexec (or resuse of stack memroy), and unpause the vCPU to cause things
> to blow up.
There are a gazillion ways for a malicious hypervisor to blow up a
'squint enough to be confident' guest.
The real question is whether it can utilize such a blow up to extract
confidential information from the guest.
If not then it's just yet another way of DoS which is an "acceptable"
attack as it only affects availability but not confidentiality.
Thanks,
tglx
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 22/37] arm64: smp: Switch to hotplug core state synchronization
@ 2023-04-25 19:51 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-25 19:51 UTC (permalink / raw)
To: Mark Rutland
Cc: LKML, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Catalin Marinas, Will Deacon,
linux-arm-kernel, David Woodhouse, Usama Arif, Juergen Gross,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Sabin Rapan
On Mon, Apr 17 2023 at 16:50, Mark Rutland wrote:
> On Sat, Apr 15, 2023 at 01:44:49AM +0200, Thomas Gleixner wrote:
> I gave this a spin on arm64 (in a 64-vCPU VM on an M1 host), and it seems to
> work fine with a bunch of vCPUs being hotplugged off and on again randomly.
>
> FWIW:
>
> Tested-by: Mark Rutland <mark.rutland@arm.com>
>
> I also hacked the code to have the dying CPU spin forever before the call to
> cpuhp_ap_report_dead(). In that case I see a warning, and that we don't call
> arch_cpuhp_cleanup_dead_cpu(), and that the CPU is marked as offline (per
> /sys/devices/system/cpu/$N/online).
Nice!
> As a tangent/aside, we might need to improve that for confidential compute
> architectures, and we might want to generically track cpus which might still be
> using kernel text/data. On arm64 we ensure that via our cpu_kill() callback
> (which'll use PSCI CPU_AFFINITY_INFO), but I'm not sure if TDX and/or SEV-SNP
> have a similar mechanism.
>
> Otherwise, a malicious hypervisor can pause a vCPU just before it leaves the
> kernel (e.g. immediately after the arch_cpuhp_cleanup_dead_cpu() call), wait
> for a kexec (or resuse of stack memroy), and unpause the vCPU to cause things
> to blow up.
There are a gazillion ways for a malicious hypervisor to blow up a
'squint enough to be confident' guest.
The real question is whether it can utilize such a blow up to extract
confidential information from the guest.
If not then it's just yet another way of DoS which is an "acceptable"
attack as it only affects availability but not confidentiality.
Thanks,
tglx
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
2023-04-20 15:57 ` Thomas Gleixner
(?)
@ 2023-04-25 20:07 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-25 20:07 UTC (permalink / raw)
To: Sean Christopherson
Cc: Andrew Cooper, Paul Menzel, linux-kernel, x86, David Woodhouse,
Brian Gerst, Arjan van de Veen, Paolo Bonzini, Paul McKenney,
Tom Lendacky, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Thu, Apr 20 2023 at 17:57, Thomas Gleixner wrote:
> On Thu, Apr 20 2023 at 07:51, Sean Christopherson wrote:
> Something like the completely untested below should just work whatever
> APIC ID the BIOS decided to dice.
>
> That might just work on SEV too without that GHCB muck, but what do I
> know.
It does not.
RDMSR(X2APIC_ID) is trapped via #VC which cannot be handled at that
point. Unfortunately the GHCB protocol does not provide a RDMSR
mechanism similar to the CPUID mechanism. Neither does the secure
firmware enforce CPUID(0xb):APICID to real APIC ID consistency.
So the hypervisor can dice the APIC IDs as long as they are consistent
with the provided ACPI/MADT table.
So no parallel startup for SEV for now.
Thanks,
tglx
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-25 20:07 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-25 20:07 UTC (permalink / raw)
To: Sean Christopherson
Cc: Andrew Cooper, Paul Menzel, linux-kernel, x86, David Woodhouse,
Brian Gerst, Arjan van de Veen, Paolo Bonzini, Paul McKenney,
Tom Lendacky, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Thu, Apr 20 2023 at 17:57, Thomas Gleixner wrote:
> On Thu, Apr 20 2023 at 07:51, Sean Christopherson wrote:
> Something like the completely untested below should just work whatever
> APIC ID the BIOS decided to dice.
>
> That might just work on SEV too without that GHCB muck, but what do I
> know.
It does not.
RDMSR(X2APIC_ID) is trapped via #VC which cannot be handled at that
point. Unfortunately the GHCB protocol does not provide a RDMSR
mechanism similar to the CPUID mechanism. Neither does the secure
firmware enforce CPUID(0xb):APICID to real APIC ID consistency.
So the hypervisor can dice the APIC IDs as long as they are consistent
with the provided ACPI/MADT table.
So no parallel startup for SEV for now.
Thanks,
tglx
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-25 20:07 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-25 20:07 UTC (permalink / raw)
To: Sean Christopherson
Cc: Andrew Cooper, Paul Menzel, linux-kernel, x86, David Woodhouse,
Brian Gerst, Arjan van de Veen, Paolo Bonzini, Paul McKenney,
Tom Lendacky, Oleksandr Natalenko, Guilherme G. Piccoli,
Piotr Gorski, David Woodhouse, Usama Arif, Jürgen Groß,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann,
linux-arm-kernel, Catalin Marinas, Will Deacon, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E. J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Thu, Apr 20 2023 at 17:57, Thomas Gleixner wrote:
> On Thu, Apr 20 2023 at 07:51, Sean Christopherson wrote:
> Something like the completely untested below should just work whatever
> APIC ID the BIOS decided to dice.
>
> That might just work on SEV too without that GHCB muck, but what do I
> know.
It does not.
RDMSR(X2APIC_ID) is trapped via #VC which cannot be handled at that
point. Unfortunately the GHCB protocol does not provide a RDMSR
mechanism similar to the CPUID mechanism. Neither does the secure
firmware enforce CPUID(0xb):APICID to real APIC ID consistency.
So the hypervisor can dice the APIC IDs as long as they are consistent
with the provided ACPI/MADT table.
So no parallel startup for SEV for now.
Thanks,
tglx
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 22/37] arm64: smp: Switch to hotplug core state synchronization
2023-04-25 19:51 ` Thomas Gleixner
(?)
@ 2023-04-26 7:59 ` Mark Rutland
-1 siblings, 0 replies; 236+ messages in thread
From: Mark Rutland @ 2023-04-26 7:59 UTC (permalink / raw)
To: Thomas Gleixner
Cc: LKML, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Catalin Marinas, Will Deacon,
linux-arm-kernel, David Woodhouse, Usama Arif, Juergen Gross,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Sabin Rapan
On Tue, Apr 25, 2023 at 09:51:12PM +0200, Thomas Gleixner wrote:
> On Mon, Apr 17 2023 at 16:50, Mark Rutland wrote:
> > As a tangent/aside, we might need to improve that for confidential compute
> > architectures, and we might want to generically track cpus which might still be
> > using kernel text/data. On arm64 we ensure that via our cpu_kill() callback
> > (which'll use PSCI CPU_AFFINITY_INFO), but I'm not sure if TDX and/or SEV-SNP
> > have a similar mechanism.
> >
> > Otherwise, a malicious hypervisor can pause a vCPU just before it leaves the
> > kernel (e.g. immediately after the arch_cpuhp_cleanup_dead_cpu() call), wait
> > for a kexec (or resuse of stack memroy), and unpause the vCPU to cause things
> > to blow up.
>
> There are a gazillion ways for a malicious hypervisor to blow up a
> 'squint enough to be confident' guest.
>
> The real question is whether it can utilize such a blow up to extract
> confidential information from the guest.
>
> If not then it's just yet another way of DoS which is an "acceptable"
> attack as it only affects availability but not confidentiality.
Sure.
My thinking is that this is an attack against the *integrity* of the guest
(since the vCPU that gets unpasued may write to memory), and so it's
potentially more than just a DoS.
I only mention this because I'd like to account for that on arm64, and if other
architectures also wanted to handle that it might make sense to have some
common infrastructure to track whether CPUs are potentially still within the
kernel.
Thanks,
Mark.
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 22/37] arm64: smp: Switch to hotplug core state synchronization
@ 2023-04-26 7:59 ` Mark Rutland
0 siblings, 0 replies; 236+ messages in thread
From: Mark Rutland @ 2023-04-26 7:59 UTC (permalink / raw)
To: Thomas Gleixner
Cc: LKML, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Catalin Marinas, Will Deacon,
linux-arm-kernel, David Woodhouse, Usama Arif, Juergen Gross,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Sabin Rapan
On Tue, Apr 25, 2023 at 09:51:12PM +0200, Thomas Gleixner wrote:
> On Mon, Apr 17 2023 at 16:50, Mark Rutland wrote:
> > As a tangent/aside, we might need to improve that for confidential compute
> > architectures, and we might want to generically track cpus which might still be
> > using kernel text/data. On arm64 we ensure that via our cpu_kill() callback
> > (which'll use PSCI CPU_AFFINITY_INFO), but I'm not sure if TDX and/or SEV-SNP
> > have a similar mechanism.
> >
> > Otherwise, a malicious hypervisor can pause a vCPU just before it leaves the
> > kernel (e.g. immediately after the arch_cpuhp_cleanup_dead_cpu() call), wait
> > for a kexec (or resuse of stack memroy), and unpause the vCPU to cause things
> > to blow up.
>
> There are a gazillion ways for a malicious hypervisor to blow up a
> 'squint enough to be confident' guest.
>
> The real question is whether it can utilize such a blow up to extract
> confidential information from the guest.
>
> If not then it's just yet another way of DoS which is an "acceptable"
> attack as it only affects availability but not confidentiality.
Sure.
My thinking is that this is an attack against the *integrity* of the guest
(since the vCPU that gets unpasued may write to memory), and so it's
potentially more than just a DoS.
I only mention this because I'd like to account for that on arm64, and if other
architectures also wanted to handle that it might make sense to have some
common infrastructure to track whether CPUs are potentially still within the
kernel.
Thanks,
Mark.
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 22/37] arm64: smp: Switch to hotplug core state synchronization
@ 2023-04-26 7:59 ` Mark Rutland
0 siblings, 0 replies; 236+ messages in thread
From: Mark Rutland @ 2023-04-26 7:59 UTC (permalink / raw)
To: Thomas Gleixner
Cc: LKML, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Catalin Marinas, Will Deacon,
linux-arm-kernel, David Woodhouse, Usama Arif, Juergen Gross,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Sabin Rapan
On Tue, Apr 25, 2023 at 09:51:12PM +0200, Thomas Gleixner wrote:
> On Mon, Apr 17 2023 at 16:50, Mark Rutland wrote:
> > As a tangent/aside, we might need to improve that for confidential compute
> > architectures, and we might want to generically track cpus which might still be
> > using kernel text/data. On arm64 we ensure that via our cpu_kill() callback
> > (which'll use PSCI CPU_AFFINITY_INFO), but I'm not sure if TDX and/or SEV-SNP
> > have a similar mechanism.
> >
> > Otherwise, a malicious hypervisor can pause a vCPU just before it leaves the
> > kernel (e.g. immediately after the arch_cpuhp_cleanup_dead_cpu() call), wait
> > for a kexec (or resuse of stack memroy), and unpause the vCPU to cause things
> > to blow up.
>
> There are a gazillion ways for a malicious hypervisor to blow up a
> 'squint enough to be confident' guest.
>
> The real question is whether it can utilize such a blow up to extract
> confidential information from the guest.
>
> If not then it's just yet another way of DoS which is an "acceptable"
> attack as it only affects availability but not confidentiality.
Sure.
My thinking is that this is an attack against the *integrity* of the guest
(since the vCPU that gets unpasued may write to memory), and so it's
potentially more than just a DoS.
I only mention this because I'd like to account for that on arm64, and if other
architectures also wanted to handle that it might make sense to have some
common infrastructure to track whether CPUs are potentially still within the
kernel.
Thanks,
Mark.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 22/37] arm64: smp: Switch to hotplug core state synchronization
2023-04-26 7:59 ` Mark Rutland
(?)
@ 2023-04-26 8:15 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-26 8:15 UTC (permalink / raw)
To: Mark Rutland
Cc: LKML, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Catalin Marinas, Will Deacon,
linux-arm-kernel, David Woodhouse, Usama Arif, Juergen Gross,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Sabin Rapan
On Wed, Apr 26 2023 at 08:59, Mark Rutland wrote:
> On Tue, Apr 25, 2023 at 09:51:12PM +0200, Thomas Gleixner wrote:
>> If not then it's just yet another way of DoS which is an "acceptable"
>> attack as it only affects availability but not confidentiality.
>
> Sure.
>
> My thinking is that this is an attack against the *integrity* of the guest
> (since the vCPU that gets unpasued may write to memory), and so it's
> potentially more than just a DoS.
>
> I only mention this because I'd like to account for that on arm64, and if other
> architectures also wanted to handle that it might make sense to have some
> common infrastructure to track whether CPUs are potentially still within the
> kernel.
Fair enough.
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 22/37] arm64: smp: Switch to hotplug core state synchronization
@ 2023-04-26 8:15 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-26 8:15 UTC (permalink / raw)
To: Mark Rutland
Cc: LKML, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Catalin Marinas, Will Deacon,
linux-arm-kernel, David Woodhouse, Usama Arif, Juergen Gross,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Sabin Rapan
On Wed, Apr 26 2023 at 08:59, Mark Rutland wrote:
> On Tue, Apr 25, 2023 at 09:51:12PM +0200, Thomas Gleixner wrote:
>> If not then it's just yet another way of DoS which is an "acceptable"
>> attack as it only affects availability but not confidentiality.
>
> Sure.
>
> My thinking is that this is an attack against the *integrity* of the guest
> (since the vCPU that gets unpasued may write to memory), and so it's
> potentially more than just a DoS.
>
> I only mention this because I'd like to account for that on arm64, and if other
> architectures also wanted to handle that it might make sense to have some
> common infrastructure to track whether CPUs are potentially still within the
> kernel.
Fair enough.
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 22/37] arm64: smp: Switch to hotplug core state synchronization
@ 2023-04-26 8:15 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-04-26 8:15 UTC (permalink / raw)
To: Mark Rutland
Cc: LKML, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, Catalin Marinas, Will Deacon,
linux-arm-kernel, David Woodhouse, Usama Arif, Juergen Gross,
Boris Ostrovsky, xen-devel, Russell King, Arnd Bergmann, Guo Ren,
linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Sabin Rapan
On Wed, Apr 26 2023 at 08:59, Mark Rutland wrote:
> On Tue, Apr 25, 2023 at 09:51:12PM +0200, Thomas Gleixner wrote:
>> If not then it's just yet another way of DoS which is an "acceptable"
>> attack as it only affects availability but not confidentiality.
>
> Sure.
>
> My thinking is that this is an attack against the *integrity* of the guest
> (since the vCPU that gets unpasued may write to memory), and so it's
> potentially more than just a DoS.
>
> I only mention this because I'd like to account for that on arm64, and if other
> architectures also wanted to handle that it might make sense to have some
> common infrastructure to track whether CPUs are potentially still within the
> kernel.
Fair enough.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* RE: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-04-27 14:48 ` Michael Kelley (LINUX)
-1 siblings, 0 replies; 236+ messages in thread
From: Michael Kelley (LINUX) @ 2023-04-27 14:48 UTC (permalink / raw)
To: Thomas Gleixner, LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
From: Thomas Gleixner <tglx@linutronix.de> Sent: Friday, April 14, 2023 4:44 PM
[snip]
>
> Conclusion
> ----------
>
> Adding the basic parallel bringup mechanism as provided by this series
> makes a lot of sense. Improving particular issues as pointed out in the
> analysis makes sense too.
>
> But trying to solve an application specific problem fully in the kernel
> with tons of complexity, without exploring straight forward and simple
> approaches first, does not make any sense at all.
>
> Thanks,
>
> tglx
>
> ---
> Documentation/admin-guide/kernel-parameters.txt | 20
> Documentation/core-api/cpu_hotplug.rst | 13
> arch/Kconfig | 23 +
> arch/arm/Kconfig | 1
> arch/arm/include/asm/smp.h | 2
> arch/arm/kernel/smp.c | 18
> arch/arm64/Kconfig | 1
> arch/arm64/include/asm/smp.h | 2
> arch/arm64/kernel/smp.c | 14
> arch/csky/Kconfig | 1
> arch/csky/include/asm/smp.h | 2
> arch/csky/kernel/smp.c | 8
> arch/mips/Kconfig | 1
> arch/mips/cavium-octeon/smp.c | 1
> arch/mips/include/asm/smp-ops.h | 1
> arch/mips/kernel/smp-bmips.c | 1
> arch/mips/kernel/smp-cps.c | 14
> arch/mips/kernel/smp.c | 8
> arch/mips/loongson64/smp.c | 1
> arch/parisc/Kconfig | 1
> arch/parisc/kernel/process.c | 4
> arch/parisc/kernel/smp.c | 7
> arch/riscv/Kconfig | 1
> arch/riscv/include/asm/smp.h | 2
> arch/riscv/kernel/cpu-hotplug.c | 14
> arch/x86/Kconfig | 45 --
> arch/x86/include/asm/apic.h | 5
> arch/x86/include/asm/cpu.h | 5
> arch/x86/include/asm/cpumask.h | 5
> arch/x86/include/asm/processor.h | 1
> arch/x86/include/asm/realmode.h | 3
> arch/x86/include/asm/sev-common.h | 3
> arch/x86/include/asm/smp.h | 26 -
> arch/x86/include/asm/topology.h | 23 -
> arch/x86/include/asm/tsc.h | 2
> arch/x86/kernel/acpi/sleep.c | 9
> arch/x86/kernel/apic/apic.c | 22 -
> arch/x86/kernel/callthunks.c | 4
> arch/x86/kernel/cpu/amd.c | 2
> arch/x86/kernel/cpu/cacheinfo.c | 21
> arch/x86/kernel/cpu/common.c | 50 --
> arch/x86/kernel/cpu/topology.c | 3
> arch/x86/kernel/head_32.S | 14
> arch/x86/kernel/head_64.S | 121 +++++
> arch/x86/kernel/sev.c | 2
> arch/x86/kernel/smp.c | 3
> arch/x86/kernel/smpboot.c | 508 ++++++++----------------
> arch/x86/kernel/topology.c | 98 ----
> arch/x86/kernel/tsc.c | 20
> arch/x86/kernel/tsc_sync.c | 36 -
> arch/x86/power/cpu.c | 37 -
> arch/x86/realmode/init.c | 3
> arch/x86/realmode/rm/trampoline_64.S | 27 +
> arch/x86/xen/enlighten_hvm.c | 11
> arch/x86/xen/smp_hvm.c | 16
> arch/x86/xen/smp_pv.c | 56 +-
> drivers/acpi/processor_idle.c | 4
> include/linux/cpu.h | 4
> include/linux/cpuhotplug.h | 17
> kernel/cpu.c | 397 +++++++++++++++++-
> kernel/smp.c | 2
> kernel/smpboot.c | 163 -------
> 62 files changed, 953 insertions(+), 976 deletions(-)
>
I smoke-tested several Linux guest configurations running on Hyper-V,
using the "kernel/git/tglx/devel.git hotplug" tree as updated on April 26th.
No functional issues, but encountered one cosmetic issue (details below).
Configurations tested:
* 16 vCPUs and 32 vCPUs
* 1 NUMA node and 2 NUMA nodes
* Parallel bring-up enabled and disabled via kernel boot line
* "Normal" VMs and SEV-SNP VMs running with a paravisor on Hyper-V.
This config can use parallel bring-up because most of the SNP-ness is
hidden in the paravisor. I was glad to see this work properly.
There's not much difference in performance with and without parallel
bring-up on the 32 vCPU VM. Without parallel, the time is about 26
milliseconds. With parallel, it's about 24 ms. So bring-up is already
fast in the virtual environment.
The cosmetic issue is in the dmesg log, and arises because Hyper-V
enumerates SMT CPUs differently from many other environments. In
a Hyper-V guest, the SMT threads in a core are numbered as <even, odd>
pairs. Guest CPUs #0 & #1 are SMT threads in core, as are #2 & #3, etc. With
parallel bring-up, here's the dmesg output:
[ 0.444345] smp: Bringing up secondary CPUs ...
[ 0.445139] .... node #0, CPUs: #2 #4 #6 #8 #10 #12 #14 #16 #18 #20 #22 #24 #26 #28 #30
[ 0.454112] x86: Booting SMP configuration:
[ 0.456035] #1 #3 #5 #7 #9 #11 #13 #15 #17 #19 #21 #23 #25 #27 #29 #31
[ 0.466120] smp: Brought up 1 node, 32 CPUs
[ 0.467036] smpboot: Max logical packages: 1
[ 0.468035] smpboot: Total of 32 processors activated (153240.06 BogoMIPS)
The function announce_cpu() is specifically testing for CPU #1 to output the
"Booting SMP configuration" message. In a Hyper-V guest, CPU #1 is the second
SMT thread in a core, so it isn't started until all the even-numbered CPUs are
started.
I don't know if this cosmetic issue is worth fixing, but I thought I'd point it out.
In any case,
Tested-by: Michael Kelley <mikelley@microsoft.com>
^ permalink raw reply [flat|nested] 236+ messages in thread
* RE: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-27 14:48 ` Michael Kelley (LINUX)
0 siblings, 0 replies; 236+ messages in thread
From: Michael Kelley (LINUX) @ 2023-04-27 14:48 UTC (permalink / raw)
To: Thomas Gleixner, LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
From: Thomas Gleixner <tglx@linutronix.de> Sent: Friday, April 14, 2023 4:44 PM
[snip]
>
> Conclusion
> ----------
>
> Adding the basic parallel bringup mechanism as provided by this series
> makes a lot of sense. Improving particular issues as pointed out in the
> analysis makes sense too.
>
> But trying to solve an application specific problem fully in the kernel
> with tons of complexity, without exploring straight forward and simple
> approaches first, does not make any sense at all.
>
> Thanks,
>
> tglx
>
> ---
> Documentation/admin-guide/kernel-parameters.txt | 20
> Documentation/core-api/cpu_hotplug.rst | 13
> arch/Kconfig | 23 +
> arch/arm/Kconfig | 1
> arch/arm/include/asm/smp.h | 2
> arch/arm/kernel/smp.c | 18
> arch/arm64/Kconfig | 1
> arch/arm64/include/asm/smp.h | 2
> arch/arm64/kernel/smp.c | 14
> arch/csky/Kconfig | 1
> arch/csky/include/asm/smp.h | 2
> arch/csky/kernel/smp.c | 8
> arch/mips/Kconfig | 1
> arch/mips/cavium-octeon/smp.c | 1
> arch/mips/include/asm/smp-ops.h | 1
> arch/mips/kernel/smp-bmips.c | 1
> arch/mips/kernel/smp-cps.c | 14
> arch/mips/kernel/smp.c | 8
> arch/mips/loongson64/smp.c | 1
> arch/parisc/Kconfig | 1
> arch/parisc/kernel/process.c | 4
> arch/parisc/kernel/smp.c | 7
> arch/riscv/Kconfig | 1
> arch/riscv/include/asm/smp.h | 2
> arch/riscv/kernel/cpu-hotplug.c | 14
> arch/x86/Kconfig | 45 --
> arch/x86/include/asm/apic.h | 5
> arch/x86/include/asm/cpu.h | 5
> arch/x86/include/asm/cpumask.h | 5
> arch/x86/include/asm/processor.h | 1
> arch/x86/include/asm/realmode.h | 3
> arch/x86/include/asm/sev-common.h | 3
> arch/x86/include/asm/smp.h | 26 -
> arch/x86/include/asm/topology.h | 23 -
> arch/x86/include/asm/tsc.h | 2
> arch/x86/kernel/acpi/sleep.c | 9
> arch/x86/kernel/apic/apic.c | 22 -
> arch/x86/kernel/callthunks.c | 4
> arch/x86/kernel/cpu/amd.c | 2
> arch/x86/kernel/cpu/cacheinfo.c | 21
> arch/x86/kernel/cpu/common.c | 50 --
> arch/x86/kernel/cpu/topology.c | 3
> arch/x86/kernel/head_32.S | 14
> arch/x86/kernel/head_64.S | 121 +++++
> arch/x86/kernel/sev.c | 2
> arch/x86/kernel/smp.c | 3
> arch/x86/kernel/smpboot.c | 508 ++++++++----------------
> arch/x86/kernel/topology.c | 98 ----
> arch/x86/kernel/tsc.c | 20
> arch/x86/kernel/tsc_sync.c | 36 -
> arch/x86/power/cpu.c | 37 -
> arch/x86/realmode/init.c | 3
> arch/x86/realmode/rm/trampoline_64.S | 27 +
> arch/x86/xen/enlighten_hvm.c | 11
> arch/x86/xen/smp_hvm.c | 16
> arch/x86/xen/smp_pv.c | 56 +-
> drivers/acpi/processor_idle.c | 4
> include/linux/cpu.h | 4
> include/linux/cpuhotplug.h | 17
> kernel/cpu.c | 397 +++++++++++++++++-
> kernel/smp.c | 2
> kernel/smpboot.c | 163 -------
> 62 files changed, 953 insertions(+), 976 deletions(-)
>
I smoke-tested several Linux guest configurations running on Hyper-V,
using the "kernel/git/tglx/devel.git hotplug" tree as updated on April 26th.
No functional issues, but encountered one cosmetic issue (details below).
Configurations tested:
* 16 vCPUs and 32 vCPUs
* 1 NUMA node and 2 NUMA nodes
* Parallel bring-up enabled and disabled via kernel boot line
* "Normal" VMs and SEV-SNP VMs running with a paravisor on Hyper-V.
This config can use parallel bring-up because most of the SNP-ness is
hidden in the paravisor. I was glad to see this work properly.
There's not much difference in performance with and without parallel
bring-up on the 32 vCPU VM. Without parallel, the time is about 26
milliseconds. With parallel, it's about 24 ms. So bring-up is already
fast in the virtual environment.
The cosmetic issue is in the dmesg log, and arises because Hyper-V
enumerates SMT CPUs differently from many other environments. In
a Hyper-V guest, the SMT threads in a core are numbered as <even, odd>
pairs. Guest CPUs #0 & #1 are SMT threads in core, as are #2 & #3, etc. With
parallel bring-up, here's the dmesg output:
[ 0.444345] smp: Bringing up secondary CPUs ...
[ 0.445139] .... node #0, CPUs: #2 #4 #6 #8 #10 #12 #14 #16 #18 #20 #22 #24 #26 #28 #30
[ 0.454112] x86: Booting SMP configuration:
[ 0.456035] #1 #3 #5 #7 #9 #11 #13 #15 #17 #19 #21 #23 #25 #27 #29 #31
[ 0.466120] smp: Brought up 1 node, 32 CPUs
[ 0.467036] smpboot: Max logical packages: 1
[ 0.468035] smpboot: Total of 32 processors activated (153240.06 BogoMIPS)
The function announce_cpu() is specifically testing for CPU #1 to output the
"Booting SMP configuration" message. In a Hyper-V guest, CPU #1 is the second
SMT thread in a core, so it isn't started until all the even-numbered CPUs are
started.
I don't know if this cosmetic issue is worth fixing, but I thought I'd point it out.
In any case,
Tested-by: Michael Kelley <mikelley@microsoft.com>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* RE: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-04-27 14:48 ` Michael Kelley (LINUX)
0 siblings, 0 replies; 236+ messages in thread
From: Michael Kelley (LINUX) @ 2023-04-27 14:48 UTC (permalink / raw)
To: Thomas Gleixner, LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
From: Thomas Gleixner <tglx@linutronix.de> Sent: Friday, April 14, 2023 4:44 PM
[snip]
>
> Conclusion
> ----------
>
> Adding the basic parallel bringup mechanism as provided by this series
> makes a lot of sense. Improving particular issues as pointed out in the
> analysis makes sense too.
>
> But trying to solve an application specific problem fully in the kernel
> with tons of complexity, without exploring straight forward and simple
> approaches first, does not make any sense at all.
>
> Thanks,
>
> tglx
>
> ---
> Documentation/admin-guide/kernel-parameters.txt | 20
> Documentation/core-api/cpu_hotplug.rst | 13
> arch/Kconfig | 23 +
> arch/arm/Kconfig | 1
> arch/arm/include/asm/smp.h | 2
> arch/arm/kernel/smp.c | 18
> arch/arm64/Kconfig | 1
> arch/arm64/include/asm/smp.h | 2
> arch/arm64/kernel/smp.c | 14
> arch/csky/Kconfig | 1
> arch/csky/include/asm/smp.h | 2
> arch/csky/kernel/smp.c | 8
> arch/mips/Kconfig | 1
> arch/mips/cavium-octeon/smp.c | 1
> arch/mips/include/asm/smp-ops.h | 1
> arch/mips/kernel/smp-bmips.c | 1
> arch/mips/kernel/smp-cps.c | 14
> arch/mips/kernel/smp.c | 8
> arch/mips/loongson64/smp.c | 1
> arch/parisc/Kconfig | 1
> arch/parisc/kernel/process.c | 4
> arch/parisc/kernel/smp.c | 7
> arch/riscv/Kconfig | 1
> arch/riscv/include/asm/smp.h | 2
> arch/riscv/kernel/cpu-hotplug.c | 14
> arch/x86/Kconfig | 45 --
> arch/x86/include/asm/apic.h | 5
> arch/x86/include/asm/cpu.h | 5
> arch/x86/include/asm/cpumask.h | 5
> arch/x86/include/asm/processor.h | 1
> arch/x86/include/asm/realmode.h | 3
> arch/x86/include/asm/sev-common.h | 3
> arch/x86/include/asm/smp.h | 26 -
> arch/x86/include/asm/topology.h | 23 -
> arch/x86/include/asm/tsc.h | 2
> arch/x86/kernel/acpi/sleep.c | 9
> arch/x86/kernel/apic/apic.c | 22 -
> arch/x86/kernel/callthunks.c | 4
> arch/x86/kernel/cpu/amd.c | 2
> arch/x86/kernel/cpu/cacheinfo.c | 21
> arch/x86/kernel/cpu/common.c | 50 --
> arch/x86/kernel/cpu/topology.c | 3
> arch/x86/kernel/head_32.S | 14
> arch/x86/kernel/head_64.S | 121 +++++
> arch/x86/kernel/sev.c | 2
> arch/x86/kernel/smp.c | 3
> arch/x86/kernel/smpboot.c | 508 ++++++++----------------
> arch/x86/kernel/topology.c | 98 ----
> arch/x86/kernel/tsc.c | 20
> arch/x86/kernel/tsc_sync.c | 36 -
> arch/x86/power/cpu.c | 37 -
> arch/x86/realmode/init.c | 3
> arch/x86/realmode/rm/trampoline_64.S | 27 +
> arch/x86/xen/enlighten_hvm.c | 11
> arch/x86/xen/smp_hvm.c | 16
> arch/x86/xen/smp_pv.c | 56 +-
> drivers/acpi/processor_idle.c | 4
> include/linux/cpu.h | 4
> include/linux/cpuhotplug.h | 17
> kernel/cpu.c | 397 +++++++++++++++++-
> kernel/smp.c | 2
> kernel/smpboot.c | 163 -------
> 62 files changed, 953 insertions(+), 976 deletions(-)
>
I smoke-tested several Linux guest configurations running on Hyper-V,
using the "kernel/git/tglx/devel.git hotplug" tree as updated on April 26th.
No functional issues, but encountered one cosmetic issue (details below).
Configurations tested:
* 16 vCPUs and 32 vCPUs
* 1 NUMA node and 2 NUMA nodes
* Parallel bring-up enabled and disabled via kernel boot line
* "Normal" VMs and SEV-SNP VMs running with a paravisor on Hyper-V.
This config can use parallel bring-up because most of the SNP-ness is
hidden in the paravisor. I was glad to see this work properly.
There's not much difference in performance with and without parallel
bring-up on the 32 vCPU VM. Without parallel, the time is about 26
milliseconds. With parallel, it's about 24 ms. So bring-up is already
fast in the virtual environment.
The cosmetic issue is in the dmesg log, and arises because Hyper-V
enumerates SMT CPUs differently from many other environments. In
a Hyper-V guest, the SMT threads in a core are numbered as <even, odd>
pairs. Guest CPUs #0 & #1 are SMT threads in core, as are #2 & #3, etc. With
parallel bring-up, here's the dmesg output:
[ 0.444345] smp: Bringing up secondary CPUs ...
[ 0.445139] .... node #0, CPUs: #2 #4 #6 #8 #10 #12 #14 #16 #18 #20 #22 #24 #26 #28 #30
[ 0.454112] x86: Booting SMP configuration:
[ 0.456035] #1 #3 #5 #7 #9 #11 #13 #15 #17 #19 #21 #23 #25 #27 #29 #31
[ 0.466120] smp: Brought up 1 node, 32 CPUs
[ 0.467036] smpboot: Max logical packages: 1
[ 0.468035] smpboot: Total of 32 processors activated (153240.06 BogoMIPS)
The function announce_cpu() is specifically testing for CPU #1 to output the
"Booting SMP configuration" message. In a Hyper-V guest, CPU #1 is the second
SMT thread in a core, so it isn't started until all the even-numbered CPUs are
started.
I don't know if this cosmetic issue is worth fixing, but I thought I'd point it out.
In any case,
Tested-by: Michael Kelley <mikelley@microsoft.com>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 26/37] riscv: Switch to hotplug core state synchronization
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-05-01 23:55 ` Palmer Dabbelt
-1 siblings, 0 replies; 236+ messages in thread
From: Palmer Dabbelt @ 2023-05-01 23:55 UTC (permalink / raw)
To: tglx
Cc: linux-kernel, x86, dwmw, andrew.cooper3, brgerst, arjan,
pbonzini, paulmck, thomas.lendacky, seanjc, oleksandr, pmenzel,
gpiccoli, lucjan.lucjanov, Paul Walmsley, linux-riscv, dwmw,
usama.arif, jgross, boris.ostrovsky, xen-devel, linux,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
guoren, linux-csky, tsbogend, linux-mips, James.Bottomley,
deller, linux-parisc, Mark Rutland, sabrapan
On Fri, 14 Apr 2023 16:44:55 PDT (-0700), tglx@linutronix.de wrote:
> Switch to the CPU hotplug core state tracking and synchronization
> mechanim. No functional change intended.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Paul Walmsley <paul.walmsley@sifive.com>
> Cc: Palmer Dabbelt <palmer@dabbelt.com>
> Cc: linux-riscv@lists.infradead.org
> ---
> arch/riscv/Kconfig | 1 +
> arch/riscv/include/asm/smp.h | 2 +-
> arch/riscv/kernel/cpu-hotplug.c | 14 +++++++-------
> 3 files changed, 9 insertions(+), 8 deletions(-)
>
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -116,6 +116,7 @@ config RISCV
> select HAVE_RSEQ
> select HAVE_STACKPROTECTOR
> select HAVE_SYSCALL_TRACEPOINTS
> + select HOTPLUG_CORE_SYNC_DEAD if HOTPLUG_CPU
> select IRQ_DOMAIN
> select IRQ_FORCED_THREADING
> select MODULES_USE_ELF_RELA if MODULES
> --- a/arch/riscv/include/asm/smp.h
> +++ b/arch/riscv/include/asm/smp.h
> @@ -64,7 +64,7 @@ asmlinkage void smp_callin(void);
>
> #if defined CONFIG_HOTPLUG_CPU
> int __cpu_disable(void);
> -void __cpu_die(unsigned int cpu);
> +static inline void __cpu_die(unsigned int cpu) { }
> #endif /* CONFIG_HOTPLUG_CPU */
>
> #else
> --- a/arch/riscv/kernel/cpu-hotplug.c
> +++ b/arch/riscv/kernel/cpu-hotplug.c
> @@ -8,6 +8,7 @@
> #include <linux/sched.h>
> #include <linux/err.h>
> #include <linux/irq.h>
> +#include <linux/cpuhotplug.h>
> #include <linux/cpu.h>
> #include <linux/sched/hotplug.h>
> #include <asm/irq.h>
> @@ -48,17 +49,15 @@ int __cpu_disable(void)
> return ret;
> }
>
> +#ifdef CONFIG_HOTPLUG_CPU
> /*
> - * Called on the thread which is asking for a CPU to be shutdown.
> + * Called on the thread which is asking for a CPU to be shutdown, if the
> + * CPU reported dead to the hotplug core.
> */
> -void __cpu_die(unsigned int cpu)
> +void arch_cpuhp_cleanup_dead_cpu(unsigned int cpu)
> {
> int ret = 0;
>
> - if (!cpu_wait_death(cpu, 5)) {
> - pr_err("CPU %u: didn't die\n", cpu);
> - return;
> - }
> pr_notice("CPU%u: off\n", cpu);
>
> /* Verify from the firmware if the cpu is really stopped*/
> @@ -75,9 +74,10 @@ void arch_cpu_idle_dead(void)
> {
> idle_task_exit();
>
> - (void)cpu_report_death();
> + cpuhp_ap_report_dead();
>
> cpu_ops[smp_processor_id()]->cpu_stop();
> /* It should never reach here */
> BUG();
> }
> +#endif
Acked-by: Palmer Dabbelt <palmer@rivosinc.com>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 26/37] riscv: Switch to hotplug core state synchronization
@ 2023-05-01 23:55 ` Palmer Dabbelt
0 siblings, 0 replies; 236+ messages in thread
From: Palmer Dabbelt @ 2023-05-01 23:55 UTC (permalink / raw)
To: tglx
Cc: linux-kernel, x86, dwmw, andrew.cooper3, brgerst, arjan,
pbonzini, paulmck, thomas.lendacky, seanjc, oleksandr, pmenzel,
gpiccoli, lucjan.lucjanov, Paul Walmsley, linux-riscv, dwmw,
usama.arif, jgross, boris.ostrovsky, xen-devel, linux,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
guoren, linux-csky, tsbogend, linux-mips, James.Bottomley,
deller, linux-parisc, Mark Rutland, sabrapan
On Fri, 14 Apr 2023 16:44:55 PDT (-0700), tglx@linutronix.de wrote:
> Switch to the CPU hotplug core state tracking and synchronization
> mechanim. No functional change intended.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Paul Walmsley <paul.walmsley@sifive.com>
> Cc: Palmer Dabbelt <palmer@dabbelt.com>
> Cc: linux-riscv@lists.infradead.org
> ---
> arch/riscv/Kconfig | 1 +
> arch/riscv/include/asm/smp.h | 2 +-
> arch/riscv/kernel/cpu-hotplug.c | 14 +++++++-------
> 3 files changed, 9 insertions(+), 8 deletions(-)
>
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -116,6 +116,7 @@ config RISCV
> select HAVE_RSEQ
> select HAVE_STACKPROTECTOR
> select HAVE_SYSCALL_TRACEPOINTS
> + select HOTPLUG_CORE_SYNC_DEAD if HOTPLUG_CPU
> select IRQ_DOMAIN
> select IRQ_FORCED_THREADING
> select MODULES_USE_ELF_RELA if MODULES
> --- a/arch/riscv/include/asm/smp.h
> +++ b/arch/riscv/include/asm/smp.h
> @@ -64,7 +64,7 @@ asmlinkage void smp_callin(void);
>
> #if defined CONFIG_HOTPLUG_CPU
> int __cpu_disable(void);
> -void __cpu_die(unsigned int cpu);
> +static inline void __cpu_die(unsigned int cpu) { }
> #endif /* CONFIG_HOTPLUG_CPU */
>
> #else
> --- a/arch/riscv/kernel/cpu-hotplug.c
> +++ b/arch/riscv/kernel/cpu-hotplug.c
> @@ -8,6 +8,7 @@
> #include <linux/sched.h>
> #include <linux/err.h>
> #include <linux/irq.h>
> +#include <linux/cpuhotplug.h>
> #include <linux/cpu.h>
> #include <linux/sched/hotplug.h>
> #include <asm/irq.h>
> @@ -48,17 +49,15 @@ int __cpu_disable(void)
> return ret;
> }
>
> +#ifdef CONFIG_HOTPLUG_CPU
> /*
> - * Called on the thread which is asking for a CPU to be shutdown.
> + * Called on the thread which is asking for a CPU to be shutdown, if the
> + * CPU reported dead to the hotplug core.
> */
> -void __cpu_die(unsigned int cpu)
> +void arch_cpuhp_cleanup_dead_cpu(unsigned int cpu)
> {
> int ret = 0;
>
> - if (!cpu_wait_death(cpu, 5)) {
> - pr_err("CPU %u: didn't die\n", cpu);
> - return;
> - }
> pr_notice("CPU%u: off\n", cpu);
>
> /* Verify from the firmware if the cpu is really stopped*/
> @@ -75,9 +74,10 @@ void arch_cpu_idle_dead(void)
> {
> idle_task_exit();
>
> - (void)cpu_report_death();
> + cpuhp_ap_report_dead();
>
> cpu_ops[smp_processor_id()]->cpu_stop();
> /* It should never reach here */
> BUG();
> }
> +#endif
Acked-by: Palmer Dabbelt <palmer@rivosinc.com>
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 26/37] riscv: Switch to hotplug core state synchronization
@ 2023-05-01 23:55 ` Palmer Dabbelt
0 siblings, 0 replies; 236+ messages in thread
From: Palmer Dabbelt @ 2023-05-01 23:55 UTC (permalink / raw)
To: tglx
Cc: linux-kernel, x86, dwmw, andrew.cooper3, brgerst, arjan,
pbonzini, paulmck, thomas.lendacky, seanjc, oleksandr, pmenzel,
gpiccoli, lucjan.lucjanov, Paul Walmsley, linux-riscv, dwmw,
usama.arif, jgross, boris.ostrovsky, xen-devel, linux,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
guoren, linux-csky, tsbogend, linux-mips, James.Bottomley,
deller, linux-parisc, Mark Rutland, sabrapan
On Fri, 14 Apr 2023 16:44:55 PDT (-0700), tglx@linutronix.de wrote:
> Switch to the CPU hotplug core state tracking and synchronization
> mechanim. No functional change intended.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Paul Walmsley <paul.walmsley@sifive.com>
> Cc: Palmer Dabbelt <palmer@dabbelt.com>
> Cc: linux-riscv@lists.infradead.org
> ---
> arch/riscv/Kconfig | 1 +
> arch/riscv/include/asm/smp.h | 2 +-
> arch/riscv/kernel/cpu-hotplug.c | 14 +++++++-------
> 3 files changed, 9 insertions(+), 8 deletions(-)
>
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -116,6 +116,7 @@ config RISCV
> select HAVE_RSEQ
> select HAVE_STACKPROTECTOR
> select HAVE_SYSCALL_TRACEPOINTS
> + select HOTPLUG_CORE_SYNC_DEAD if HOTPLUG_CPU
> select IRQ_DOMAIN
> select IRQ_FORCED_THREADING
> select MODULES_USE_ELF_RELA if MODULES
> --- a/arch/riscv/include/asm/smp.h
> +++ b/arch/riscv/include/asm/smp.h
> @@ -64,7 +64,7 @@ asmlinkage void smp_callin(void);
>
> #if defined CONFIG_HOTPLUG_CPU
> int __cpu_disable(void);
> -void __cpu_die(unsigned int cpu);
> +static inline void __cpu_die(unsigned int cpu) { }
> #endif /* CONFIG_HOTPLUG_CPU */
>
> #else
> --- a/arch/riscv/kernel/cpu-hotplug.c
> +++ b/arch/riscv/kernel/cpu-hotplug.c
> @@ -8,6 +8,7 @@
> #include <linux/sched.h>
> #include <linux/err.h>
> #include <linux/irq.h>
> +#include <linux/cpuhotplug.h>
> #include <linux/cpu.h>
> #include <linux/sched/hotplug.h>
> #include <asm/irq.h>
> @@ -48,17 +49,15 @@ int __cpu_disable(void)
> return ret;
> }
>
> +#ifdef CONFIG_HOTPLUG_CPU
> /*
> - * Called on the thread which is asking for a CPU to be shutdown.
> + * Called on the thread which is asking for a CPU to be shutdown, if the
> + * CPU reported dead to the hotplug core.
> */
> -void __cpu_die(unsigned int cpu)
> +void arch_cpuhp_cleanup_dead_cpu(unsigned int cpu)
> {
> int ret = 0;
>
> - if (!cpu_wait_death(cpu, 5)) {
> - pr_err("CPU %u: didn't die\n", cpu);
> - return;
> - }
> pr_notice("CPU%u: off\n", cpu);
>
> /* Verify from the firmware if the cpu is really stopped*/
> @@ -75,9 +74,10 @@ void arch_cpu_idle_dead(void)
> {
> idle_task_exit();
>
> - (void)cpu_report_death();
> + cpuhp_ap_report_dead();
>
> cpu_ops[smp_processor_id()]->cpu_stop();
> /* It should never reach here */
> BUG();
> }
> +#endif
Acked-by: Palmer Dabbelt <palmer@rivosinc.com>
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* RE: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
2023-04-27 14:48 ` Michael Kelley (LINUX)
(?)
@ 2023-05-04 18:46 ` Thomas Gleixner
-1 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-05-04 18:46 UTC (permalink / raw)
To: Michael Kelley (LINUX), LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Michael!
On Thu, Apr 27 2023 at 14:48, Michael Kelley wrote:
> From: Thomas Gleixner <tglx@linutronix.de> Sent: Friday, April 14, 2023 4:44 PM
>
> I smoke-tested several Linux guest configurations running on Hyper-V,
> using the "kernel/git/tglx/devel.git hotplug" tree as updated on April 26th.
> No functional issues, but encountered one cosmetic issue (details below).
>
> Configurations tested:
> * 16 vCPUs and 32 vCPUs
> * 1 NUMA node and 2 NUMA nodes
> * Parallel bring-up enabled and disabled via kernel boot line
> * "Normal" VMs and SEV-SNP VMs running with a paravisor on Hyper-V.
> This config can use parallel bring-up because most of the SNP-ness is
> hidden in the paravisor. I was glad to see this work properly.
>
> There's not much difference in performance with and without parallel
> bring-up on the 32 vCPU VM. Without parallel, the time is about 26
> milliseconds. With parallel, it's about 24 ms. So bring-up is already
> fast in the virtual environment.
Depends on the environment :)
> The cosmetic issue is in the dmesg log, and arises because Hyper-V
> enumerates SMT CPUs differently from many other environments. In
> a Hyper-V guest, the SMT threads in a core are numbered as <even, odd>
> pairs. Guest CPUs #0 & #1 are SMT threads in core, as are #2 & #3, etc. With
> parallel bring-up, here's the dmesg output:
>
> [ 0.444345] smp: Bringing up secondary CPUs ...
> [ 0.445139] .... node #0, CPUs: #2 #4 #6 #8 #10 #12 #14 #16 #18 #20 #22 #24 #26 #28 #30
> [ 0.454112] x86: Booting SMP configuration:
> [ 0.456035] #1 #3 #5 #7 #9 #11 #13 #15 #17 #19 #21 #23 #25 #27 #29 #31
> [ 0.466120] smp: Brought up 1 node, 32 CPUs
> [ 0.467036] smpboot: Max logical packages: 1
> [ 0.468035] smpboot: Total of 32 processors activated (153240.06 BogoMIPS)
>
> The function announce_cpu() is specifically testing for CPU #1 to output the
> "Booting SMP configuration" message. In a Hyper-V guest, CPU #1 is the second
> SMT thread in a core, so it isn't started until all the even-numbered CPUs are
> started.
Ah. Didn't notice that because SMT siblings are usually enumerated after
all primary ones in ACPI.
> I don't know if this cosmetic issue is worth fixing, but I thought I'd point it out.
That's trivial enough to fix. I'll amend the topmost patch before
posting V2.
Thanks for giving it a ride!
tglx
^ permalink raw reply [flat|nested] 236+ messages in thread
* RE: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-05-04 18:46 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-05-04 18:46 UTC (permalink / raw)
To: Michael Kelley (LINUX), LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Michael!
On Thu, Apr 27 2023 at 14:48, Michael Kelley wrote:
> From: Thomas Gleixner <tglx@linutronix.de> Sent: Friday, April 14, 2023 4:44 PM
>
> I smoke-tested several Linux guest configurations running on Hyper-V,
> using the "kernel/git/tglx/devel.git hotplug" tree as updated on April 26th.
> No functional issues, but encountered one cosmetic issue (details below).
>
> Configurations tested:
> * 16 vCPUs and 32 vCPUs
> * 1 NUMA node and 2 NUMA nodes
> * Parallel bring-up enabled and disabled via kernel boot line
> * "Normal" VMs and SEV-SNP VMs running with a paravisor on Hyper-V.
> This config can use parallel bring-up because most of the SNP-ness is
> hidden in the paravisor. I was glad to see this work properly.
>
> There's not much difference in performance with and without parallel
> bring-up on the 32 vCPU VM. Without parallel, the time is about 26
> milliseconds. With parallel, it's about 24 ms. So bring-up is already
> fast in the virtual environment.
Depends on the environment :)
> The cosmetic issue is in the dmesg log, and arises because Hyper-V
> enumerates SMT CPUs differently from many other environments. In
> a Hyper-V guest, the SMT threads in a core are numbered as <even, odd>
> pairs. Guest CPUs #0 & #1 are SMT threads in core, as are #2 & #3, etc. With
> parallel bring-up, here's the dmesg output:
>
> [ 0.444345] smp: Bringing up secondary CPUs ...
> [ 0.445139] .... node #0, CPUs: #2 #4 #6 #8 #10 #12 #14 #16 #18 #20 #22 #24 #26 #28 #30
> [ 0.454112] x86: Booting SMP configuration:
> [ 0.456035] #1 #3 #5 #7 #9 #11 #13 #15 #17 #19 #21 #23 #25 #27 #29 #31
> [ 0.466120] smp: Brought up 1 node, 32 CPUs
> [ 0.467036] smpboot: Max logical packages: 1
> [ 0.468035] smpboot: Total of 32 processors activated (153240.06 BogoMIPS)
>
> The function announce_cpu() is specifically testing for CPU #1 to output the
> "Booting SMP configuration" message. In a Hyper-V guest, CPU #1 is the second
> SMT thread in a core, so it isn't started until all the even-numbered CPUs are
> started.
Ah. Didn't notice that because SMT siblings are usually enumerated after
all primary ones in ACPI.
> I don't know if this cosmetic issue is worth fixing, but I thought I'd point it out.
That's trivial enough to fix. I'll amend the topmost patch before
posting V2.
Thanks for giving it a ride!
tglx
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply [flat|nested] 236+ messages in thread
* RE: [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup
@ 2023-05-04 18:46 ` Thomas Gleixner
0 siblings, 0 replies; 236+ messages in thread
From: Thomas Gleixner @ 2023-05-04 18:46 UTC (permalink / raw)
To: Michael Kelley (LINUX), LKML
Cc: x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Paul McKenney, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
Michael!
On Thu, Apr 27 2023 at 14:48, Michael Kelley wrote:
> From: Thomas Gleixner <tglx@linutronix.de> Sent: Friday, April 14, 2023 4:44 PM
>
> I smoke-tested several Linux guest configurations running on Hyper-V,
> using the "kernel/git/tglx/devel.git hotplug" tree as updated on April 26th.
> No functional issues, but encountered one cosmetic issue (details below).
>
> Configurations tested:
> * 16 vCPUs and 32 vCPUs
> * 1 NUMA node and 2 NUMA nodes
> * Parallel bring-up enabled and disabled via kernel boot line
> * "Normal" VMs and SEV-SNP VMs running with a paravisor on Hyper-V.
> This config can use parallel bring-up because most of the SNP-ness is
> hidden in the paravisor. I was glad to see this work properly.
>
> There's not much difference in performance with and without parallel
> bring-up on the 32 vCPU VM. Without parallel, the time is about 26
> milliseconds. With parallel, it's about 24 ms. So bring-up is already
> fast in the virtual environment.
Depends on the environment :)
> The cosmetic issue is in the dmesg log, and arises because Hyper-V
> enumerates SMT CPUs differently from many other environments. In
> a Hyper-V guest, the SMT threads in a core are numbered as <even, odd>
> pairs. Guest CPUs #0 & #1 are SMT threads in core, as are #2 & #3, etc. With
> parallel bring-up, here's the dmesg output:
>
> [ 0.444345] smp: Bringing up secondary CPUs ...
> [ 0.445139] .... node #0, CPUs: #2 #4 #6 #8 #10 #12 #14 #16 #18 #20 #22 #24 #26 #28 #30
> [ 0.454112] x86: Booting SMP configuration:
> [ 0.456035] #1 #3 #5 #7 #9 #11 #13 #15 #17 #19 #21 #23 #25 #27 #29 #31
> [ 0.466120] smp: Brought up 1 node, 32 CPUs
> [ 0.467036] smpboot: Max logical packages: 1
> [ 0.468035] smpboot: Total of 32 processors activated (153240.06 BogoMIPS)
>
> The function announce_cpu() is specifically testing for CPU #1 to output the
> "Booting SMP configuration" message. In a Hyper-V guest, CPU #1 is the second
> SMT thread in a core, so it isn't started until all the even-numbered CPUs are
> started.
Ah. Didn't notice that because SMT siblings are usually enumerated after
all primary ones in ACPI.
> I don't know if this cosmetic issue is worth fixing, but I thought I'd point it out.
That's trivial enough to fix. I'll amend the topmost patch before
posting V2.
Thanks for giving it a ride!
tglx
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 236+ messages in thread
* Re: [patch 05/37] x86/topology: Remove CPU0 hotplug option
2023-04-14 23:44 ` Thomas Gleixner
(?)
@ 2023-06-21 16:50 ` Paul E. McKenney
-1 siblings, 0 replies; 236+ messages in thread
From: Paul E. McKenney @ 2023-06-21 16:50 UTC (permalink / raw)
To: Thomas Gleixner
Cc: LKML, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Sat, Apr 15, 2023 at 01:44:21AM +0200, Thomas Gleixner wrote:
> This was introduced together with commit e1c467e69040 ("x86, hotplug: Wake
> up CPU0 via NMI instead of INIT, SIPI, SIPI") to eventually support
> physical hotplug of CPU0:
>
> "We'll change this code in the future to wake up hard offlined CPU0 if
> real platform and request are available."
>
> 11 years later this has not happened and physical hotplug is not officially
> supported. Remove the cruft.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> ---
> Documentation/admin-guide/kernel-parameters.txt | 14 ---
> Documentation/core-api/cpu_hotplug.rst | 13 ---
> arch/x86/Kconfig | 43 ----------
> arch/x86/include/asm/cpu.h | 3
> arch/x86/kernel/topology.c | 98 ------------------------
> arch/x86/power/cpu.c | 37 ---------
> 6 files changed, 6 insertions(+), 202 deletions(-)
[ . . . ]
> --- a/arch/x86/Kconfig
> +++ b/arch/x86/Kconfig
> @@ -2294,49 +2294,6 @@ config HOTPLUG_CPU
> def_bool y
> depends on SMP
>
> -config BOOTPARAM_HOTPLUG_CPU0
Removing this requires also removing its use in rcutorture.
I have therefore queued the commit below in -rcu, but please feel
free to take it along with the BOOTPARAM_HOTPLUG_CPU0-removal patch.
Just please let me know if you do.
(Yes, I finally got back to testing -next. Why do you ask?)
Thanx, Paul
------------------------------------------------------------------------
commit 95588de780c0e81004b72526aa3e3ef5ce054719
Author: Paul E. McKenney <paulmck@kernel.org>
Date: Wed Jun 21 09:44:52 2023 -0700
rcutorture: Remove obsolete BOOTPARAM_HOTPLUG_CPU0 Kconfig option
Now that the BOOTPARAM_HOTPLUG_CPU0 Kconfig option is in the process of
being removed, it is time to remove rcutorture's use of it.
Link: https://lore.kernel.org/lkml/20230414232309.510911744@linutronix.de/
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: <x86@kernel.org>
diff --git a/tools/testing/selftests/rcutorture/configs/rcu/TREE01 b/tools/testing/selftests/rcutorture/configs/rcu/TREE01
index 04831ef1f9b5..8ae41d5f81a3 100644
--- a/tools/testing/selftests/rcutorture/configs/rcu/TREE01
+++ b/tools/testing/selftests/rcutorture/configs/rcu/TREE01
@@ -15,4 +15,3 @@ CONFIG_DEBUG_LOCK_ALLOC=n
CONFIG_RCU_BOOST=n
CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
CONFIG_RCU_EXPERT=y
-CONFIG_BOOTPARAM_HOTPLUG_CPU0=y
_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv
^ permalink raw reply related [flat|nested] 236+ messages in thread
* Re: [patch 05/37] x86/topology: Remove CPU0 hotplug option
@ 2023-06-21 16:50 ` Paul E. McKenney
0 siblings, 0 replies; 236+ messages in thread
From: Paul E. McKenney @ 2023-06-21 16:50 UTC (permalink / raw)
To: Thomas Gleixner
Cc: LKML, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Sat, Apr 15, 2023 at 01:44:21AM +0200, Thomas Gleixner wrote:
> This was introduced together with commit e1c467e69040 ("x86, hotplug: Wake
> up CPU0 via NMI instead of INIT, SIPI, SIPI") to eventually support
> physical hotplug of CPU0:
>
> "We'll change this code in the future to wake up hard offlined CPU0 if
> real platform and request are available."
>
> 11 years later this has not happened and physical hotplug is not officially
> supported. Remove the cruft.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> ---
> Documentation/admin-guide/kernel-parameters.txt | 14 ---
> Documentation/core-api/cpu_hotplug.rst | 13 ---
> arch/x86/Kconfig | 43 ----------
> arch/x86/include/asm/cpu.h | 3
> arch/x86/kernel/topology.c | 98 ------------------------
> arch/x86/power/cpu.c | 37 ---------
> 6 files changed, 6 insertions(+), 202 deletions(-)
[ . . . ]
> --- a/arch/x86/Kconfig
> +++ b/arch/x86/Kconfig
> @@ -2294,49 +2294,6 @@ config HOTPLUG_CPU
> def_bool y
> depends on SMP
>
> -config BOOTPARAM_HOTPLUG_CPU0
Removing this requires also removing its use in rcutorture.
I have therefore queued the commit below in -rcu, but please feel
free to take it along with the BOOTPARAM_HOTPLUG_CPU0-removal patch.
Just please let me know if you do.
(Yes, I finally got back to testing -next. Why do you ask?)
Thanx, Paul
------------------------------------------------------------------------
commit 95588de780c0e81004b72526aa3e3ef5ce054719
Author: Paul E. McKenney <paulmck@kernel.org>
Date: Wed Jun 21 09:44:52 2023 -0700
rcutorture: Remove obsolete BOOTPARAM_HOTPLUG_CPU0 Kconfig option
Now that the BOOTPARAM_HOTPLUG_CPU0 Kconfig option is in the process of
being removed, it is time to remove rcutorture's use of it.
Link: https://lore.kernel.org/lkml/20230414232309.510911744@linutronix.de/
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: <x86@kernel.org>
diff --git a/tools/testing/selftests/rcutorture/configs/rcu/TREE01 b/tools/testing/selftests/rcutorture/configs/rcu/TREE01
index 04831ef1f9b5..8ae41d5f81a3 100644
--- a/tools/testing/selftests/rcutorture/configs/rcu/TREE01
+++ b/tools/testing/selftests/rcutorture/configs/rcu/TREE01
@@ -15,4 +15,3 @@ CONFIG_DEBUG_LOCK_ALLOC=n
CONFIG_RCU_BOOST=n
CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
CONFIG_RCU_EXPERT=y
-CONFIG_BOOTPARAM_HOTPLUG_CPU0=y
^ permalink raw reply related [flat|nested] 236+ messages in thread
* Re: [patch 05/37] x86/topology: Remove CPU0 hotplug option
@ 2023-06-21 16:50 ` Paul E. McKenney
0 siblings, 0 replies; 236+ messages in thread
From: Paul E. McKenney @ 2023-06-21 16:50 UTC (permalink / raw)
To: Thomas Gleixner
Cc: LKML, x86, David Woodhouse, Andrew Cooper, Brian Gerst,
Arjan van de Veen, Paolo Bonzini, Tom Lendacky,
Sean Christopherson, Oleksandr Natalenko, Paul Menzel,
Guilherme G. Piccoli, Piotr Gorski, David Woodhouse, Usama Arif,
Juergen Gross, Boris Ostrovsky, xen-devel, Russell King,
Arnd Bergmann, linux-arm-kernel, Catalin Marinas, Will Deacon,
Guo Ren, linux-csky, Thomas Bogendoerfer, linux-mips,
James E.J. Bottomley, Helge Deller, linux-parisc, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Mark Rutland, Sabin Rapan
On Sat, Apr 15, 2023 at 01:44:21AM +0200, Thomas Gleixner wrote:
> This was introduced together with commit e1c467e69040 ("x86, hotplug: Wake
> up CPU0 via NMI instead of INIT, SIPI, SIPI") to eventually support
> physical hotplug of CPU0:
>
> "We'll change this code in the future to wake up hard offlined CPU0 if
> real platform and request are available."
>
> 11 years later this has not happened and physical hotplug is not officially
> supported. Remove the cruft.
>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> ---
> Documentation/admin-guide/kernel-parameters.txt | 14 ---
> Documentation/core-api/cpu_hotplug.rst | 13 ---
> arch/x86/Kconfig | 43 ----------
> arch/x86/include/asm/cpu.h | 3
> arch/x86/kernel/topology.c | 98 ------------------------
> arch/x86/power/cpu.c | 37 ---------
> 6 files changed, 6 insertions(+), 202 deletions(-)
[ . . . ]
> --- a/arch/x86/Kconfig
> +++ b/arch/x86/Kconfig
> @@ -2294,49 +2294,6 @@ config HOTPLUG_CPU
> def_bool y
> depends on SMP
>
> -config BOOTPARAM_HOTPLUG_CPU0
Removing this requires also removing its use in rcutorture.
I have therefore queued the commit below in -rcu, but please feel
free to take it along with the BOOTPARAM_HOTPLUG_CPU0-removal patch.
Just please let me know if you do.
(Yes, I finally got back to testing -next. Why do you ask?)
Thanx, Paul
------------------------------------------------------------------------
commit 95588de780c0e81004b72526aa3e3ef5ce054719
Author: Paul E. McKenney <paulmck@kernel.org>
Date: Wed Jun 21 09:44:52 2023 -0700
rcutorture: Remove obsolete BOOTPARAM_HOTPLUG_CPU0 Kconfig option
Now that the BOOTPARAM_HOTPLUG_CPU0 Kconfig option is in the process of
being removed, it is time to remove rcutorture's use of it.
Link: https://lore.kernel.org/lkml/20230414232309.510911744@linutronix.de/
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: <x86@kernel.org>
diff --git a/tools/testing/selftests/rcutorture/configs/rcu/TREE01 b/tools/testing/selftests/rcutorture/configs/rcu/TREE01
index 04831ef1f9b5..8ae41d5f81a3 100644
--- a/tools/testing/selftests/rcutorture/configs/rcu/TREE01
+++ b/tools/testing/selftests/rcutorture/configs/rcu/TREE01
@@ -15,4 +15,3 @@ CONFIG_DEBUG_LOCK_ALLOC=n
CONFIG_RCU_BOOST=n
CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
CONFIG_RCU_EXPERT=y
-CONFIG_BOOTPARAM_HOTPLUG_CPU0=y
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 236+ messages in thread
end of thread, other threads:[~2023-06-21 16:51 UTC | newest]
Thread overview: 236+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-04-14 23:44 [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` [patch 01/37] x86/smpboot: Cleanup topology_phys_to_logical_pkg()/die() Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` [patch 02/37] cpu/hotplug: Mark arch_disable_smp_support() and bringup_nonboot_cpus() __init Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` [patch 03/37] x86/smpboot: Avoid pointless delay calibration is TSC is synchronized Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` [patch 04/37] x86/smpboot: Rename start_cpu0() to soft_restart_cpu() Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` [patch 05/37] x86/topology: Remove CPU0 hotplug option Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-06-21 16:50 ` Paul E. McKenney
2023-06-21 16:50 ` Paul E. McKenney
2023-06-21 16:50 ` Paul E. McKenney
2023-04-14 23:44 ` [patch 06/37] x86/smpboot: Remove the CPU0 hotplug kludge Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` [patch 07/37] x86/smpboot: Restrict soft_restart_cpu() to SEV Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` [patch 08/37] x86/smpboot: Split up native_cpu_up() into separate phases and document them Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` [patch 09/37] x86/smpboot: Get rid of cpu_init_secondary() Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` [patch 10/37] x86/cpu/cacheinfo: Remove cpu_callout_mask dependency Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` [patch 11/37] x86/smpboot: Move synchronization masks to SMP boot code Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` [patch 12/37] x86/smpboot: Make TSC synchronization function call based Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` [patch 13/37] x86/smpboot: Remove cpu_callin_mask Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` [patch 14/37] cpu/hotplug: Rework sparse_irq locking in bringup_cpu() Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` [patch 15/37] x86/smpboot: Remove wait for cpu_online() Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` [patch 16/37] x86/xen/smp_pv: Remove wait for CPU online Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-17 20:46 ` Boris Ostrovsky
2023-04-17 20:46 ` Boris Ostrovsky
2023-04-17 20:46 ` Boris Ostrovsky
2023-04-14 23:44 ` [patch 17/37] x86/xen/hvm: Get rid of DEAD_FROZEN handling Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` [patch 18/37] cpu/hotplug: Add CPU state tracking and synchronization Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` [patch 19/37] x86/smpboot: Switch to hotplug core state synchronization Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-15 12:58 ` Brian Gerst
2023-04-15 12:58 ` Brian Gerst
2023-04-15 12:58 ` Brian Gerst
2023-04-15 21:04 ` Thomas Gleixner
2023-04-15 21:04 ` Thomas Gleixner
2023-04-15 21:04 ` Thomas Gleixner
2023-04-14 23:44 ` [patch 20/37] cpu/hotplug: Remove cpu_report_state() and related unused cruft Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` [patch 21/37] ARM: smp: Switch to hotplug core state synchronization Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` [patch 22/37] arm64: " Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-17 15:50 ` Mark Rutland
2023-04-17 15:50 ` Mark Rutland
2023-04-17 15:50 ` Mark Rutland
2023-04-25 19:51 ` Thomas Gleixner
2023-04-25 19:51 ` Thomas Gleixner
2023-04-25 19:51 ` Thomas Gleixner
2023-04-26 7:59 ` Mark Rutland
2023-04-26 7:59 ` Mark Rutland
2023-04-26 7:59 ` Mark Rutland
2023-04-26 8:15 ` Thomas Gleixner
2023-04-26 8:15 ` Thomas Gleixner
2023-04-26 8:15 ` Thomas Gleixner
2023-04-14 23:44 ` [patch 23/37] csky/smp: " Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` [patch 24/37] MIPS: SMP_CPS: " Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` [patch 25/37] parisc: " Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` [patch 26/37] riscv: " Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-05-01 23:55 ` Palmer Dabbelt
2023-05-01 23:55 ` Palmer Dabbelt
2023-05-01 23:55 ` Palmer Dabbelt
2023-04-14 23:44 ` [patch 27/37] cpu/hotplug: Remove unused state functions Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` [patch 28/37] cpu/hotplug: Reset task stack state in _cpu_up() Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:44 ` Thomas Gleixner
2023-04-14 23:45 ` [patch 29/37] cpu/hotplug: Provide a split up CPUHP_BRINGUP mechanism Thomas Gleixner
2023-04-14 23:45 ` Thomas Gleixner
2023-04-14 23:45 ` Thomas Gleixner
2023-04-14 23:45 ` [patch 30/37] x86/smpboot: Enable split CPU startup Thomas Gleixner
2023-04-14 23:45 ` Thomas Gleixner
2023-04-14 23:45 ` Thomas Gleixner
2023-04-14 23:45 ` [patch 31/37] x86/apic: Provide cpu_primary_thread mask Thomas Gleixner
2023-04-14 23:45 ` Thomas Gleixner
2023-04-14 23:45 ` Thomas Gleixner
2023-04-14 23:45 ` [patch 32/37] cpu/hotplug: Allow "parallel" bringup up to CPUHP_BP_KICK_AP_STATE Thomas Gleixner
2023-04-14 23:45 ` Thomas Gleixner
2023-04-14 23:45 ` Thomas Gleixner
2023-04-14 23:45 ` [patch 33/37] x86/topology: Store extended topology leaf information Thomas Gleixner
2023-04-14 23:45 ` Thomas Gleixner
2023-04-14 23:45 ` Thomas Gleixner
2023-04-14 23:45 ` [patch 34/37] x86/cpu/amd; Invoke detect_extended_topology_early() on boot CPU Thomas Gleixner
2023-04-14 23:45 ` Thomas Gleixner
2023-04-14 23:45 ` Thomas Gleixner
2023-04-14 23:45 ` [patch 35/37] x86/smpboot: Support parallel startup of secondary CPUs Thomas Gleixner
2023-04-14 23:45 ` Thomas Gleixner
2023-04-14 23:45 ` Thomas Gleixner
2023-04-15 13:22 ` Brian Gerst
2023-04-15 13:22 ` Brian Gerst
2023-04-15 13:22 ` Brian Gerst
2023-04-15 21:06 ` Thomas Gleixner
2023-04-15 21:06 ` Thomas Gleixner
2023-04-15 21:06 ` Thomas Gleixner
2023-04-24 17:58 ` Thomas Gleixner
2023-04-24 17:58 ` Thomas Gleixner
2023-04-24 17:58 ` Thomas Gleixner
2023-04-14 23:45 ` [patch 36/37] x86/smpboot/64: Implement arch_cpuhp_init_parallel_bringup() and enable it Thomas Gleixner
2023-04-14 23:45 ` Thomas Gleixner
2023-04-14 23:45 ` Thomas Gleixner
2023-04-14 23:45 ` [patch 37/37] x86/smpboot: Allow parallel bringup for SEV-ES Thomas Gleixner
2023-04-14 23:45 ` Thomas Gleixner
2023-04-14 23:45 ` Thomas Gleixner
2023-04-17 8:35 ` [patch 00/37] cpu/hotplug, x86: Reworked parallel CPU bringup Juergen Gross
2023-04-17 8:35 ` Juergen Gross
2023-04-17 8:35 ` Juergen Gross
2023-04-17 10:30 ` Peter Zijlstra
2023-04-17 10:30 ` Peter Zijlstra
2023-04-17 10:30 ` Peter Zijlstra
2023-04-17 10:44 ` Andrew Cooper
2023-04-17 10:44 ` Andrew Cooper
2023-04-17 10:44 ` Andrew Cooper
2023-04-17 11:19 ` Paul Menzel
2023-04-17 11:19 ` Paul Menzel
2023-04-17 11:19 ` Paul Menzel
2023-04-17 11:24 ` Paul Menzel
2023-04-17 11:24 ` Paul Menzel
2023-04-17 11:24 ` Paul Menzel
2023-04-17 14:48 ` Thomas Gleixner
2023-04-17 14:48 ` Thomas Gleixner
2023-04-17 14:48 ` Thomas Gleixner
2023-04-17 17:40 ` Paul Menzel
2023-04-17 17:40 ` Paul Menzel
2023-04-17 17:40 ` Paul Menzel
2023-04-18 6:58 ` Thomas Gleixner
2023-04-18 6:58 ` Thomas Gleixner
2023-04-18 6:58 ` Thomas Gleixner
2023-04-18 8:40 ` Thomas Gleixner
2023-04-18 8:40 ` Thomas Gleixner
2023-04-18 8:40 ` Thomas Gleixner
2023-04-18 20:10 ` Paul Menzel
2023-04-18 20:10 ` Paul Menzel
2023-04-18 20:10 ` Paul Menzel
2023-04-19 9:38 ` Thomas Gleixner
2023-04-19 9:38 ` Thomas Gleixner
2023-04-19 9:38 ` Thomas Gleixner
2023-04-19 12:38 ` Thomas Gleixner
2023-04-19 12:38 ` Thomas Gleixner
2023-04-19 12:38 ` Thomas Gleixner
2023-04-19 13:32 ` David Woodhouse
2023-04-19 13:32 ` David Woodhouse
2023-04-19 13:32 ` David Woodhouse
2023-04-19 13:43 ` Thomas Gleixner
2023-04-19 13:43 ` Thomas Gleixner
2023-04-19 13:43 ` Thomas Gleixner
2023-04-19 13:50 ` Andrew Cooper
2023-04-19 13:50 ` Andrew Cooper
2023-04-19 13:50 ` Andrew Cooper
2023-04-19 16:21 ` Andrew Cooper
2023-04-19 16:21 ` Andrew Cooper
2023-04-19 16:21 ` Andrew Cooper
2023-04-20 8:32 ` Thomas Gleixner
2023-04-20 8:32 ` Thomas Gleixner
2023-04-20 8:32 ` Thomas Gleixner
2023-04-20 9:23 ` Andrew Cooper
2023-04-20 9:23 ` Andrew Cooper
2023-04-20 9:23 ` Andrew Cooper
2023-04-20 11:17 ` Thomas Gleixner
2023-04-20 11:17 ` Thomas Gleixner
2023-04-20 11:17 ` Thomas Gleixner
2023-04-20 14:51 ` Sean Christopherson
2023-04-20 14:51 ` Sean Christopherson
2023-04-20 14:51 ` Sean Christopherson
2023-04-20 15:57 ` Thomas Gleixner
2023-04-20 15:57 ` Thomas Gleixner
2023-04-20 15:57 ` Thomas Gleixner
2023-04-20 16:47 ` Paul Menzel
2023-04-20 16:47 ` Paul Menzel
2023-04-20 16:47 ` Paul Menzel
2023-04-20 19:10 ` Thomas Gleixner
2023-04-20 19:10 ` Thomas Gleixner
2023-04-20 19:10 ` Thomas Gleixner
2023-04-21 16:36 ` Thomas Gleixner
2023-04-21 16:36 ` Thomas Gleixner
2023-04-21 16:36 ` Thomas Gleixner
2023-04-24 18:46 ` Paul Menzel
2023-04-24 18:46 ` Paul Menzel
2023-04-24 18:46 ` Paul Menzel
2023-04-25 20:07 ` Thomas Gleixner
2023-04-25 20:07 ` Thomas Gleixner
2023-04-25 20:07 ` Thomas Gleixner
2023-04-19 16:45 ` Paul Menzel
2023-04-19 16:45 ` Paul Menzel
2023-04-27 14:48 ` Michael Kelley (LINUX)
2023-04-27 14:48 ` Michael Kelley (LINUX)
2023-04-27 14:48 ` Michael Kelley (LINUX)
2023-05-04 18:46 ` Thomas Gleixner
2023-05-04 18:46 ` Thomas Gleixner
2023-05-04 18:46 ` Thomas Gleixner
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.