Re: [PATCH v1 2/2] x86, apic: Disable BSP if boot cpu is AP

[ebiederm@xmission.com (Eric W. Biederman)]

HATAYAMA Daisuke <d.hatayama@jp.fujitsu.com> writes:

> From: "H. Peter Anvin" <hpa@zytor.com>
> Subject: Re: [PATCH v1 2/2] x86, apic: Disable BSP if boot cpu is AP
> Date: Mon, 22 Oct 2012 17:35:47 -0700
>
>> On 10/22/2012 02:29 PM, Eric W. Biederman wrote:
>>>>
>>>> As I said, I thought Fenghua tried that but it didn't work,
>>>> experimentally.
>>>
>>> Fair enough.  You described the problem with clearing bit 8 in a weird
>>> way.
>>>
>>> If the best we can muster are fuzzy memories it may be worth
>>> revisiting.
>>> Perhaps it works on enough cpu models to be interesting.
>>>
>> 
>> It isn't fuzzy memories... this was done as late as 1-2 months ago.  I
>> just don't know the details.
>> 
>> Fenghua, could you help fill us in?
>> 
>
> I overlooked completely the fact that BSP flag is rewritable.
>
> I tried Eric's suggestion using attached test programs and saw it
> worked fine at least on the three cpus around me below:
>
> - Intel(R) Xeon(R) CPU E7- 4820  @ 2.00GHz
> - Intel(R) Xeon(R) CPU E7- 8870  @ 2.40GHz
> - Intel(R) Xeon(TM) CPU 1.80GHz
>   - 32 bits CPU
>
> Next I found the description about this in 8.4.2, IASDM Vol.3:
>
>   The MP initialization protocol imposes the following requirements
>   and restrictions on the system:
>
>   * The MP protocol is executed only after a power-up or RESET. If the
>     MP protocol has completed and a BSP is chosen, subsequent INITs
>     (either to a specific processor or system wide) do not cause the
>     MP protocol to be repeated. Instead, each logical processor
>     examines its BSP flag (in the IA32_APIC_BASE MSR) to determine
>     whether it should execute the BIOS boot-strap code (if it is the
>     BSP) or enter a wait-for-SIPI state (if it is an AP).
>
> So this is no longer undocumented behaviour for recent cpus, I think.

The underdocumented bit is the ability to clear the flag.
And of course these are processor specific registers.

> Considering these, I'll make a patch to clear BSP flag at appropreate
> position in kernel boot-up code. OTOH, according to the discussion, it
> was reported that clearing BSP flag affected some BIOSes. To deal with
> this, I'll prepare a kernel option to decide whether to clear BSP flag
> or not.
>
> Does anyone have any comments now? Or please comment after I submit a
> new patch.

I think you are on right track with preparing some patches, and this
certainly looks like worth experimenting with.

At least for i386 the code need to verify you have a cpu new enough to
have an APIC_BASE_MSR, but I don't think that is going to be hard.

Eric