Re: [RFC PATCH] perf/x86/intel/rapl: avoid access unallocate memory

["Charles (Chas) Williams" <ciwillia@brocade.com>]

On 11/04/2016 02:03 PM, Sebastian Andrzej Siewior wrote:
> On 2016-11-04 08:20:37 [-0400], Charles (Chas) Williams wrote:
>> The initial CPU boots and is identified:
>>
>> [    0.009018] identify_boot_cpu
>> [    0.009174] generic_identify: phys_proc_id is now 0
>> ...
>> [    0.009427] identify_cpu: before c ffffffff81ae2680  logical_proc_id 0  c->phys_proc_id 0
>> [    0.009506] identify_cpu: after c ffffffff81ae2680  logical_proc_id 65535  c->phys_proc_id 0
>>
>> So, this is fine because the APIC hasn't been scanned yet.  APIC
>> now gets scanned:
>>
>> [    0.015789] smpboot: APIC(0) Converting physical 0 to logical package 0, cpu 0 (ffff88023fc0a040)
>> [    0.015794] smpboot: APIC(1) Converting physical 1 to logical package 1, cpu 1 (ffff88023fd0a040)
>> [    0.015797] smpboot: Max logical packages: 2
>
> where is the APICID here is comming from?

This comes from here:

                 unsigned int apicid = apic->cpu_present_to_apicid(cpu);

                 if (apicid == BAD_APICID || !apic->apic_id_valid(apicid))
                         continue;
                 if (!topology_update_package_map(apicid, cpu))

And I think this is the part that is "wrong".  The apicid appears to
be a logical CPU id.  I believe that in most cases this mapping comes
from x86_bios_cpu_apicid (or x86_cpu_to_apicid) which is generated in
generic_processor_info() which maps apicid's to logical cpu indexes.

Note that apic->cpu_present_to_apicid() is using just the cpu_index.

         for_each_present_cpu(cpu) {
                 unsigned int apicid = apic->cpu_present_to_apicid(cpu);

                 if (apicid == BAD_APICID || !apic->apic_id_valid(apicid))
                         continue;
                 if (!topology_update_package_map(apicid, cpu))
                         continue;
                 pr_warn("CPU %u APICId %x disabled\n", cpu, apicid);
                 per_cpu(x86_bios_cpu_apicid, cpu) = BAD_APICID;
                 set_cpu_possible(cpu, false);
                 set_cpu_present(cpu, false);
         }

>> So, at this point, I think everything is correct.  But now the secondary
>> CPU's "boot":
>>
>> [    0.236569] identify_secondary_cpu
>> [    0.236620] generic_identify: phys_proc_id is now 2
>
> so here is where fun starts. Xen has also
> arch/x86/xen/smp.c::cpu_bringup() where the phys_proc_id is changed. But
> isn't done for vmware but it might a place where they duct tape things.
>
> How is this APIC id different from the earlier? I guess based on your
> output that generic_identify() changes the content of phys_proc_id.
>
>> [    0.236745] identify_cpu: before c ffff88023fd0a040  logical_proc_id 65535  c->phys_proc_id 2
>> [    0.236747] identify_cpu: after c ffff88023fd0a040  logical_proc_id 65535  c->phys_proc_id 2
>>
>> So, APIC discovered I have a cpu 0 and 1 but generic_identify() is called
>> my second CPU, 2.  This is >= max_physical_pkg_id, so it is going to get
>> set to -1.
>
> Now. max_physical_pkg_id is huge. The physical_to_logical_pkg array is
> set to -1 on init so slot two has the value -1. That is what you see -
> not the -1 because of ">= max_physical_pkg_id".
>
>> The comment at the end of identfy_cpu() says:
>>
>>         /* The boot/hotplug time assigment got cleared, restore it */
>>
>> So, logical_proc_id being wrong here before restoration doesn't bother
>> me since I assume something in booting the secondary CPU's clears any
>> existing cpu data.
>>
>> I know detect_extended_topology() is likely being called for both CPU's
>> and getting the right values (checking this now).  I don't know why
>> generic_identify() is resetting this value.
>
> I don't know either. But it is clearly reading the apic id twice and
> second approach is different from the first which leads to different
> results. So if you figure out how the first APICID for the second CPU is
> retrieved and then you see how it happens for the second time. There
> must be a difference.

The phys core id from generic_identify() comes from the CPU's EBX register
so we _know_ this is right.

	       if (c->cpuid_level >= 0x00000001) {
			c->initial_apicid = (cpuid_ebx(1) >> 24) & 0xFF;
	#ifdef CONFIG_X86_32
	# ifdef CONFIG_SMP
			c->apicid = apic->phys_pkg_id(c->initial_apicid, 0);
	# else
			c->apicid = c->initial_apicid;
	# endif
	#endif
			c->phys_proc_id = c->initial_apicid;
		}

The intel docs http://x86.renejeschke.de/html/file_module_x86_id_45.html
claims this is the Local APIC ID.  So it seems likely this is correct
value.  It's not clear it matter if this is the right value or not
though.  Even if this is the correct apicid, nothing knows about it.

An argument could be made that instead of checking the cpuid level, we
could just use the apicid based on the cpu index just like the other code.
It would be consistent at least then.