Re: smp_call_function_single lockups

[Rafael David Tinoco <inaddy@ubuntu.com>]

I could only find an advisory (regarding sr-iov and irq remaps) from
HP to RHEL6.2 users stating that Gen8 firmware does not enable it by
default.

http://h20564.www2.hp.com/hpsc/doc/public/display?docId=emr_na-c03645796

"""
The interrupt remapping capability depends on x2apic enabled in the
BIOS and HP ProLiant Gen8 systems do not enable x2apic; therefore, the
following workaround is required for device assignment: Edit the
/etc/grub.conf and add intremap=no_x2apic_optout option to the kernel
command line options.
"""

Probably for backwards compatibility... not sure if there is an option
to enable/disable it in firmware (like DL390 seems to have).
I don't think so... but I was told by HP team that I should use x2apic
for >= Gen8.


On Thu, Feb 19, 2015 at 2:59 PM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
> On Thu, Feb 19, 2015 at 8:32 AM, Rafael David Tinoco <inaddy@ubuntu.com> wrote:
>> Feb 19 08:21:28 derain kernel: [    3.637682] Switched APIC routing to
>> cluster x2apic.
>
> Ok. That "cluster x2apic" mode is just about the nastiest mode when it
> comes to sending a single ipi. We do that insane dance where we
>
>  - turn single cpu number into cpumask
>  - copy the cpumask to a percpu temporary storage
>  - walk each cpu in the cpumask
>  - for each cpu, look up the cluster siblings
>  - for each cluster sibling that is also in the cpumask, look up the
> logical apic mask and add it to the actual ipi destination mask
>  - send an ipi to that final mask.
>
> which is just insane. It's complicated, it's fragile, and it's unnecessary.
>
> If we had a simple "send_IPI()" function, we could do this all with
> something much saner, and it would look sopmething like
>
>   static void x2apic_send_IPI(int cpu, int vector)
>   {
>         u32 dest = per_cpu(x86_cpu_to_logical_apicid, cpu);
>         x2apic_wrmsr_fence();
>         __x2apic_send_IPI_dest(dest, vector, APIC_DEST_LOGICAL);
>   }
>
> and then 'void native_send_call_func_single_ipi()' would just look like
>
>   void native_send_call_func_single_ipi(int cpu)
>   {
>         apic->send_IPI(cpu, CALL_FUNCTION_SINGLE_VECTOR);
>   }
>
> but I might have missed something (and we might want to have a wrapper
> that says "if the apic doesn't have a 'send_IPI' function, use
> "send_IPI_mask(cpumask_of(cpu, vector) instead"
>
> The fact that you need that no_x2apic_optout (which in turn means that
> your ACPI tables seem to say "don't use x2apic") also makes me worry.
>
> Are there known errata for the x2apic?
>
>                          Linus