On 28.09.22 13:22, Borislav Petkov wrote:
> On Wed, Sep 28, 2022 at 01:14:11PM +0200, Juergen Gross wrote:
>> No, we don't.
>>
>> Using basically your patch, but with
>>
>> +	mtrr_online = cpuhp_setup_state_nocalls(CPUHP_AP_ONLINE_DYN,
>> +						"x86/mtrr:online",
>> +						mtrr_ap_init, NULL);
>>
>> moved to the end of mtrr_aps_init(), and:
>>
>> +void mtrr_aps_thaw(void)
>> +{
>> +	cpuhp_remove_state_nocalls(mtrr_online);
>> +}
> 
> Yes, so you said. I'm not sure I like this toggling of notifier
> registration like that.

Yeah, especially with having to remember the slot.

Would you feel better with adding a new enum member CPUHP_AP_CACHECTRL_ONLINE?

This would avoid a possible source of failure during resume in case no slot
for CPUHP_AP_ONLINE_DYN is found (quite improbable, but in theory possible).

> Optimally, I'd like to be able to query the suspend code whether it is
> in the process of resuming.
> 
> This here:
> 
> 
> static int resume_target_kernel(bool platform_mode)
> {
> 
> ...
> 
>   Enable_irqs:
>          system_state = SYSTEM_RUNNING;
>          local_irq_enable();
>   
>   Enable_cpus:
>          pm_sleep_enable_secondary_cpus();
> 
> 
> but being able to do:
> 
>          pm_sleep_enable_secondary_cpus();
> 	system_state = SYSTEM_RUNNING | SYSTEM_RUNNING_APS_UP;
> 
> which can't work, obviously. But something like that.
> 

You wouldn't want to do that there, as there are multiple places where
pm_sleep_enable_secondary_cpus() is being called. Additionally not all
cases are coming in via pm_sleep_enable_secondary_cpus(), as there is
e.g. a call of suspend_enable_secondary_cpus() from kernel_kexec(),
which wants to have the same handling.

arch_thaw_secondary_cpus_begin() and arch_thaw_secondary_cpus_end() are
the functions to mark start and end of the special region where the
delayed MTRR setup should happen.


Juergen