From: "M. Vefa Bicakci" <m.v.b@runbox.com>
To: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: "Charles (Chas) Williams" <ciwillia@Brocade.com>,
Thomas Gleixner <tglx@linutronix.de>,
Sebastian Andrzej Siewior <bigeasy@linutronix.de>,
"x86@kernel.org" <x86@kernel.org>,
LKML <linux-kernel@vger.kernel.org>,
Peter Zijlstra <peterz@infradead.org>,
Borislav Petkov <bp@alien8.de>,
David Vrabel <david.vrabel@citrix.com>,
Juergen Gross <jgross@suse.com>,
xen-devel <xen-devel@lists.xen.org>
Subject: Re: [PATCH] x86/cpuid: Deal with broken firmware once more
Date: Mon, 14 Nov 2016 02:42:43 +0300 [thread overview]
Message-ID: <41978b7b-2880-4ea5-14c3-7185422261e7@runbox.com> (raw)
In-Reply-To: <68840c0b-44c9-ddd8-bfab-f4fd8bacbaf0@oracle.com>
On 11/13/2016 09:04 PM, Boris Ostrovsky wrote:
> On 11/12/2016 05:05 PM, M. Vefa Bicakci wrote:
>> On 11/10/2016 06:31 PM, Boris Ostrovsky wrote:
>>> On 11/10/2016 10:05 AM, Charles (Chas) Williams wrote:
>>>>
>>>> On 11/10/2016 09:02 AM, Boris Ostrovsky wrote:
>>>>> On 11/10/2016 06:13 AM, Thomas Gleixner wrote:
>>>>>> On Thu, 10 Nov 2016, M. Vefa Bicakci wrote:
>>>>>>
>>>>>>> I have found that your patch unfortunately does not improve the
>>>>>>> situation
>>>>>>> for me. Here is an excerpt obtained from the dmesg of a kernel
>>>>>>> compiled
>>>>>>> with this patch *as well as* Sebastian's patch:
>>>>>>> [ 0.002561] CPU: Physical Processor ID: 0
>>>>>>> [ 0.002566] CPU: Processor Core ID: 0
>>>>>>> [ 0.002572] [Firmware Bug]: CPU0: APIC id mismatch. Firmware:
>>>>>>> ffff CPUID: 2
>>>>>> So apic->cpu_present_to_apicid() gives us a completely bogus APIC id
>>>>>> which
>>>>>> translates to a bogus package id. And looking at the XEN code:
>>>>>>
>>>>>> xen_pv_apic.cpu_present_to_apicid = xen_cpu_present_to_apicid,
>>>>>>
>>>>>> and xen_cpu_present_to_apicid does:
>>>>>>
>>>>>> static int xen_cpu_present_to_apicid(int cpu)
>>>>>> {
>>>>>> if (cpu_present(cpu))
>>>>>> return xen_get_apic_id(xen_apic_read(APIC_ID));
>>>>>> else
>>>>>> return BAD_APICID;
>>>>>> }
>>>>>>
>>>>>> So independent of which present CPU we query we get just some random
>>>>>> information, in the above case we get BAD_APICID from
>>>>>> xen_apic_read() not
>>>>>> from the else path as this CPU _IS_ present.
>>>>>>
>>>>>> What's so wrong with storing the fricking firmware supplied APICid as
>>>>>> everybody else does and report it back when queried?
>>>>> By firmware you mean ACPI? It is most likely not available to PV guests.
>>>>> How about returning cpu_data(cpu).initial_apicid?
>>>>>
>>>>> And what was the original problem?
>>>> The original issue I found was that VMware was returning a different set
>>>> of APIC id's in the ACPI tables than what it advertised on the CPU's.
>>>>
>>>> http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1266716.html
>>> For Xen, we recently added a6a198bc60e6 ("xen/x86: Update topology map
>>> for PV VCPUs") to at least temporarily work around some topology map
>>> problems that PV guests have with RAPL (which I think is what Vefa's
>>> problem was).
>> Hello Boris,
>>
>> (Sorry for the delay!)
>>
>> It appears that the problem is a bit different compared to the one
>> corrected by a6a198bc60e6, because my kernel tree -- based on 4.8.6 --
>> already includes the -stable backport of that commit, i.e.
>> 88540ad0820ddfb05626e0136c0e5a79cea85fd1
>>
>> The patch I included in my previous e-mail (dated 2016-11-10) corrects
>> root cause of the issue I am having with 4.8.6. Sebastian's original
>> patch adding error checking to the RAPL module prevents the RAPL module
>> from causing a kernel oops without my patch.
>
> I don't see any messages from you on that date. Can you provide a link
> to it (and to Sebastian's patch)?
>
> (BTW, generally it's a good idea to copy xen-devel list on any
> Xen-related issues).
As I explain below, it turns out that my issue was 'only' a kernel
configuration issue.
For reference, I had unknowingly solved my kernel-configuration-induced
issue via the patch at:
https://marc.info/?l=linux-kernel&m=147875027314638&w=2
Sebastian's patch (which adds error handling to the RAPL module) is at:
https://marc.info/?l=linux-kernel&m=147739814217598&w=2
>> The issue I am experiencing is caused by the boot-up code in the
>> 'init_apic_mappings' function switching the APIC ops structure from
>> Xen's structure to a no-op structure by calling the 'apic_disable'
>> function. Please let me know if I can clarify or elaborate.
>
> apic_disable() is only invoked if there is no APIC present (i.e.
> detect_init_APIC() returns a non-zero value) and I don't think this can
> happen. Is your CPUID[1].edx[9] not set?
I found out that my domU kernels invoke the 'apic_disable' function
because CONFIG_X86_MPPARSE was not enabled in my kernel configuration,
which would cause the 'smp_found_config' bit to be unset at boot-up.
This would cause 'init_apic_mappings' to call 'apic_disable', which
would cause Xen's 'apic' ops structure pointer to be replaced with the
no-op APIC ops structure's pointer.
The use of the no-op APIC ops structure would in turn cause invalid
virtual CPU package identifiers to be generated. Invalid CPU package
identifiers would in turn cause the RAPL module to produce a kernel oops
due to potentially missing error handling.
It looks like I have been ignoring the following kernel warning which I
should have noticed a long time ago:
MPS support code is not built-in.
Using acpi=off or acpi=noirq or pci=noacpi may have problem
To all on this e-mail thread, I learned a bit through this exercise, but
I have also taken a lot of everyone's time and created quite a bit of
e-mail traffic because of a kernel configuration issue on my end.
My apologies.
Vefa
next prev parent reply other threads:[~2016-11-13 23:43 UTC|newest]
Thread overview: 43+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-11-02 12:25 [RFC PATCH] perf/x86/intel/rapl: avoid access unallocate memory Sebastian Andrzej Siewior
2016-11-02 22:47 ` Charles (Chas) Williams
2016-11-03 17:47 ` Sebastian Andrzej Siewior
2016-11-04 12:20 ` Charles (Chas) Williams
2016-11-04 18:03 ` Sebastian Andrzej Siewior
2016-11-04 20:42 ` Charles (Chas) Williams
2016-11-04 20:57 ` Sebastian Andrzej Siewior
2016-11-07 16:19 ` Thomas Gleixner
2016-11-07 16:59 ` Charles (Chas) Williams
2016-11-07 20:20 ` Thomas Gleixner
2016-11-08 14:20 ` Charles (Chas) Williams
2016-11-08 14:31 ` Thomas Gleixner
2016-11-08 14:57 ` Charles (Chas) Williams
2016-11-08 16:22 ` Thomas Gleixner
2016-11-09 15:35 ` [PATCH] x86/cpuid: Deal with broken firmware once more Thomas Gleixner
2016-11-09 15:37 ` Thomas Gleixner
2016-11-09 16:03 ` Peter Zijlstra
2016-11-09 16:34 ` Charles (Chas) Williams
2016-11-09 18:37 ` Thomas Gleixner
2016-11-09 18:15 ` Charles (Chas) Williams
2016-11-09 20:27 ` [tip:x86/urgent] x86/cpu: Deal with broken firmware (VMWare/XEN) tip-bot for Thomas Gleixner
2016-11-11 5:49 ` Alok Kataria
2016-11-10 3:57 ` [PATCH] x86/cpuid: Deal with broken firmware once more M. Vefa Bicakci
2016-11-10 10:50 ` Charles (Chas) Williams
2016-11-10 11:14 ` Thomas Gleixner
2016-11-12 22:05 ` M. Vefa Bicakci
2016-11-10 11:13 ` Thomas Gleixner
2016-11-10 11:39 ` Peter Zijlstra
2016-11-10 14:02 ` Boris Ostrovsky
2016-11-10 15:05 ` Charles (Chas) Williams
2016-11-10 15:31 ` Boris Ostrovsky
2016-11-10 15:54 ` Sebastian Andrzej Siewior
2016-11-10 17:15 ` Thomas Gleixner
2016-11-12 22:05 ` M. Vefa Bicakci
2016-11-13 18:04 ` Boris Ostrovsky
2016-11-13 23:42 ` M. Vefa Bicakci [this message]
2016-11-15 1:21 ` Boris Ostrovsky
2016-11-18 11:16 ` Thomas Gleixner
2016-11-18 14:22 ` Boris Ostrovsky
2016-11-10 15:12 ` Thomas Gleixner
2016-11-10 15:38 ` Boris Ostrovsky
2016-11-10 17:13 ` Thomas Gleixner
2016-11-10 18:01 ` Boris Ostrovsky
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=41978b7b-2880-4ea5-14c3-7185422261e7@runbox.com \
--to=m.v.b@runbox.com \
--cc=bigeasy@linutronix.de \
--cc=boris.ostrovsky@oracle.com \
--cc=bp@alien8.de \
--cc=ciwillia@Brocade.com \
--cc=david.vrabel@citrix.com \
--cc=jgross@suse.com \
--cc=linux-kernel@vger.kernel.org \
--cc=peterz@infradead.org \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
--cc=xen-devel@lists.xen.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).