On Tue, 2023-02-21 at 11:54 +0000, Usama Arif wrote:
> 
> 
> On 21/02/2023 11:42, Oleksandr Natalenko wrote:
> > On 21.02.2023 11:47, Usama Arif wrote:
> > > On 21/02/2023 10:27, David Woodhouse wrote:
> > > > 
> > > > On 21 February 2023 09:49:51 GMT, Oleksandr Natalenko 
> > > > <oleksandr@natalenko.name> wrote:
> > > > > On 21.02.2023 10:06, David Woodhouse wrote:
> > > > > > Why does arch/x86/kernel/acpi/sleep.c::x86_acpi_suspend_lowlevel() set
> > > > > > 
> > > > > >      initial_gs = per_cpu_offset(smp_processor_id()) ?
> > > > > > 
> > > > > > Would it not be CPU#0 that comes back up, and should it not get
> > > > > > per_cpu_offset(0) ?
> > > > > 
> > > > > Wanna me try `initial_gs = per_cpu_offset(0);` too?
> > > > 
> > > 
> > > I think it might be smp_processor_id() and not 0 incase CPU0 was 
> > > offline at the point the system was suspended?
> > 
> > Is it even possible for CPU 0 to be offline, at least on x86?
> > 
> 
> It is possible on x86 (using BOOTPARAM_HOTPLUG_CPU0), but I just read
> the Kconfig option and it says:
> 
> "resume from hibernate or suspend always starts from CPU0.
> So hibernate and suspend are prevented if CPU0 is offline."
> 
> so I guess switching to 0 should be ok.

The interesting question is who ends up using whose stack?

Leaving smpboot_control alone (for the parallel case only; we can't
just leave it with the APIC ID of the last CPU started in the serial
case) will make each CPU find the same CPU# and stack it used to have,
from its own APIC ID and the cpuid_to_apicid[] table.

I don't really grok what's happening in the non-parallel case. We leave
behind the stack for the CPU that happens to be running the suspend
function. And then whichever CPU comes back, it'll get *that* stack.

I don't understand why the parallel bringup changes this. Does the
cpuid to apicid mapping *change* on resume, if the suspending and
resuming CPU are different? Do they swap stacks and CPU# somehow?